PureLytics Stream

The PureLytics stream sends real time UEM data from the AppMon Server to an external data source like Elasticsearch so you can you leverage UEM data as part of big data analysis. Streamed data includes user visits, user actions and client errors.

Use Elasticsearch to do ad-hoc analysis on large amounts of visit or user action data, and combine the data with analytics mined from other sources. You can then consume the combined data stored in the external data source using a visualization tool like Kibana for detailed charting and other data visualization, providing detailed UEM analysis.

When enabled, the PureLytics stream automatically sends JSON data for all visits and user actions to the configured HTTP endpoint. An export can also be triggered on demand for a specific time frame or session.

The JSON document format

Visit, User Action, and Client Error data export as JSON documents. Data for any Business Transactions defined for Visits and User Actions also export as part of the documents.

Using the PureLytics Stream with ElasticSearch and Kibana

This video shows an example of how to visualize data from the PureLytics stream in Kibana.

The following sections describe the general process to visualize PureLytics Stream data using ElasticSearch and Kibana.

Satisfy requirements and prerequisites

Install streaming target software and visualization software

If not yet installed, install Elasticsearch and install Kibana.

Start ElasticSearch and set up a cluster

You must have Elasticsearch started and have at least one cluster set up in one node to interact with it and create JSON documents. Run an Elasticsearch cluster with at least 2 nodes for optimal results.

See the guide on the Elasticsearch Web site for more information on starting and using Elasticsearch.

Apply a dynamic template

Apply a dynamic template to the cluster. Adapt the number of shards to the number of your data nodes.

The template contains the JSON data for the Elastic, and the command to apply it.

The following shows an example cURL -X PUT command used to send the template to the Elasticsearch cluster.

curl -X PUT 'http://localhost:9200/_template/dynatrace' -d '{"order": 0, "template": "dt_*"}'

You can use dt_* as an index pattern get the streamed data. If you're sending the JSON data as a cURL string, escape the double quotes. Otherwise, it is not needed.

Configure the PureLytics Stream

See the Configure the PureLytics Stream section on the Configure Realtime Streaming page to learn how to configure AppMon to stream PureLytics data.

On-demand export

You can export data on-demand for a specific session in the AppMon Client, or use the PureLytics Stream REST API to export on-demand.

Visualize data in Kibana

You must configure at least one index pattern to use Kibana. An index pattern identifies the index for which you want to search and visualize PureLytics data. Each index pattern you specify must match the name of one of your Elasticsearch indices.

Technical details and limitations

  • Documents are exported after they are analyzed. For visits data, this means documents export after the visit completes. It can take some time until a visit is visible in Elasticsearch, depending on the visit length and the configured visit timeout.
  • Documents of every configured system profile are stored in indices per time frame. An index is created for every month. and named accordingly. For example, visits, user actions and client errors which started in January 2016 (using the document field data.startTime) are stored in index named dt_2016-01.
  • PureLytics streaming uses the Elasticsearch bulk API (https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html). AppMon UEM appends /_bulk to the configured URL that was configured (only if the target Elasticsearch is configured). A single request contains multiple documents of different types. The bulk API also requires additional meta data alongside the actual data containing various information, including the used index, document type, and the document ID.
{"index":{"_index":"dt_2016-03","_type":"visit","_id":"1860046861_14036"}}
{"serverID":1860046861,"serverName":"lnz124742d03","systemProfile":"easyTravel","data":{"visitId":14036,"startTime":1457606705739,"endTime":1457606729349,"duration":23610,"application":"easyTravel mobile","appVersion":"1.9","visitTag":null,"additionalTags":{},"userExperienceIndex":1.0,"userExperience":"satisfied","userExperienceReason":"Satisfied","connectionType":"WIFI","bandwidth":5496,"isConverted":0,"isBounced":0,"isCrashed":0,"pageViews":2,"userActions":7,"failedActions":0,"landingPage":{"title":"SearchJourneyActivity","responseTime":1909,"isFailed":0},"exitPage":{"title":"easyTravel - Terms of Use","responseTime":289,"isFailed":0},"clientType":"mobile","clientDetails":{"osFamily":"Android","osVersion":"Android 2.3.3","manufacturer":"Samsung","deviceName":"Galaxy S","modelId":"GT-I9000","resolution":"480x800","cpuInfo":"armv7","isRooted":0,"adkVersion":"6.1.1234","gpsCoordinates":{"lat":48.85832,"lon":2.29436},"applicationBuildVersion":"1234"},"ipAddress":"159.108.218.157","location":{"continent":"Europe","country":"France","region":"Ile-de-France","city":"Neuilly"},"convertedBy":null,"businessTransactions":{"Conversion+visits":{"measures":{},"splittings":null},"Visits+by+connection+type":{"measures":{},"splittings":{"Connection+Type+of+Visits":"WIFI"}}},"carrier":"Orange"}}
{"index":{"_index":"dt_2016-03","_type":"visit","_id":"1860046861_14037"}}
{"serverID":1860046861,"serverName":"lnz124742d03","systemProfile":"easyTravel","data":{"visitId":14037,"startTime":1457606708560,"endTime":1457606709023,"duration":463,"application":"easyTravel portal","appVersion":"1.8","visitTag":null,"additionalTags":{"Visits - App Version":"1.8"},"userExperienceIndex":0.0,"userExperience":"frustrated","userExperienceReason":"DueToErrorInLastAction","connectionType":"Broadband","bandwidth":5496,"isConverted":0,"isBounced":1,"isCrashed":0,"pageViews":1,"userActions":1,"failedActions":1,"landingPage":{"title":"easyTravel - Terms of Use","responseTime":463,"isFailed":1},"exitPage":{"title":"easyTravel - Terms of Use","responseTime":463,"isFailed":1},"clientType":"browser","clientDetails":{"osFamily":"Android","osVersion":"Android 4.0.x Ice Cream Sandwich","browserFamily":"Chrome Mobile","browserVersion":"38.0"},"ipAddress":"128.133.96.84","location":{"continent":"North America","country":"United States","region":"Alabama","city":"Montgomery"},"convertedBy":null,"businessTransactions":{"Conversion+visits":{"measures":{},"splittings":null},"Visits+by+connection+type":{"measures":{},"splittings":{"Connection+Type+of+Visits":"Broadband (>1500 kb/s)"}}},"browserErrors":1,"videoStreams":0,"audioStreams":0,"isp":"-"}}
{"index":{"_index":"dt_2016-03","_type":"useraction","_id":"1860046861_1385397273_1","parent":"1860046861_14036"}}
{"serverID":1860046861,"serverName":"lnz124742d03","systemProfile":"easyTravel","data":{"visitId":14036,"tagId":1,"agentId":1385397273,"startTime":1457606705739,"endTime":1457606707648,"visitTag":null,"additionalTags":{},"name":"searchJourney","prettyName":"searchJourney","type":"ActionEvent","actionGroup":"Mobile ADK Actions","actionGroupPerformanceBaseline":4000,"apdex":1.0,"userExperience":"satisfied","isFailed":0,"failingReasons":null,"responseTime":1909.0,"networkContributionTime":1665.6201,"serverContributionTime":26.379906,"thirdPartyContribution":-1.0,"cdnContribution":-1.0,"application":"easyTravel mobile","appVersion":"1.9","location":{"continent":"Europe","country":"France","region":"Ile-de-France","city":"Neuilly"},"clientDetails":{"osFamily":"Android","osVersion":"Android 2.3.3","manufacturer":"Samsung","deviceName":"Galaxy S","modelId":"GT-I9000","resolution":"480x800","cpuInfo":"armv7","isRooted":0,"adkVersion":"6.1.1234","gpsCoordinates":{"lat":48.85832,"lon":2.29436},"applicationBuildVersion":"1234","isPortrait":0,"batteryStatus":0.33563203,"totalMemory":790,"freeMemory":0.56,"runningProcesses":26,"networkTechnology":"802.11x","signalStrength":-115},"xhrUrl":null,"clientErrors":0,"serverErrors":0,"perceivedRenderTime":-1,"perceivedRenderTimeSlowestImageSrc":"-","source":{"name":"SearchJourneyActivity","viewDuration":3593},"target":{"name":"SearchJourneyActivity"},"resourceSummary":null,"navTiming":null,"onLoad":-1,"domready":0,"metaData":null,"adkStrings":{},"adkValues":{},"businessTransactions":{"Pageview+Apdex+by+Country":{"measures":{"Apdex":1.0},"splittings":{"Country+of+Visits":"France"}},"Pageview+Apdex+by+Application":{"measures":{"Apdex":1.0},"splittings":null}}}}