Configure user session exports
Dynatrace can continually send user session data to a provided webhook endpoint. The server that provides the webhook needs to listen for PUT or POST requests on a URL specified as part of the configuration.
The data for completed user sessions is sent in bulk to improve performance, with a flush every few seconds to export the data as soon as the user session is marked as complete.
To configure user session exports, go to Settings > Integration > User session exports. See below for details.
1. Define your endpoints
Endpoint setup
You can configure up to three HTTPS endpoints. Finished user sessions are sent in bulk to all defined endpoints.
For security reasons, Dynatrace only allows HTTPS endpoints.
If you aren't using an Elasticsearch installation as destination, the data is sent to an endpoint via a PUT call with a bulk JSON payload.
- Operation:
PUT(orPOSTif configured). - Path: <as configured>.
- MIME type:
application/jsonby default, but sent as multiple JSON documents in bulk line-by-line.When sending to Elasticsearch, set the MIME type to
application/x-ndjson. - Response body: The handler should respond with
HTTP 200 "OK". - Description: Dynatrace sends user sessions in bulk as soon as one of the following conditions is met:
- 1000 user sessions have finished
- The bulk size exceeds 896,000 bytes
- No user session has finished in the last 30 seconds
To prevent system overload, Dynatrace cancels the request when your endpoint doesn't respond within 30 seconds. Dynatrace retries three more times before it finally discards the request and sends an alert notification with the Request timeout error message.
Sample code when using the Jersey REST framework
The following sample code uses the open source Jersey REST framework to provide the necessary endpoint for receiving the data.
@Path("/export/")
public class ExportREST {
...
@PUT
@Produces(MediaType.APPLICATION_JSON)
@Path("events")
public JResponse<String> jsonEvents(final String data) {
...
// split the bulk into single documents
final String[] lines = StringUtils.split(data, '\n');
for(String line : lines) {
...
// handle the JSON-data
}
return JResponse.ok("")
.header(HttpHeaders.SERVER, "Endpoint for Dynatrace session data export")
.build();
}
}2. Configure basic authentication
Basic authentication enables you to secure your endpoints.
To configure user session export for basic authentication
- On the User session exports page, under Authentication, turn on Activate.
- Select Basic authentication
- Enter your username and password.
The password is encrypted and masked in your environment.
You'll need to re-enter the basic authentication password when testing your configuration.
3. Send data directly to Elasticsearch
To send data directly to your Elasticsearch installation
- Make sure your Elasticsearch instance is reachable from the cluster server. For Dynatrace SaaS deployments, this means it needs to be publicly available. You can use an SSL certificate and basic authentication to secure access.
- Make sure your export URL has the following format:
https://<your_host>:9200/_bulk. Replace<your_host>with your actual value. - On the User session exports page, under Send data directly to Elasticsearch section, turn on Activate.
- Enter the name of the index where data is sent and the type of documents in the Elasticsearch index.
Elasticsearch index
Create an index where you want to send data and define the mapping for your index. See Download a sample mapping for details on how to download a sample mapping for your configuration.
If you don't create the index before enabling user session export, your Elasticsearch installation automatically creates the mappings for the fields. This automatic mapping doesn't create the appropriate field mappings in all cases. For example, date fields are mapped as long.
Elasticsearch index type
Elasticsearch is currently removing support for mapping types. The way you create indexes and configure types depends on your Elasticsearch version. See below for details.
We recommend that you set _doc as the type of document regardless of the Elasticsearch version you're running.
- Elasticsearch version 6
- You can specify one single type per index.
- When creating an index, you can specify an
include_type_namequery string parameter to indicate that requests and responses should include a type name. This parameter defaults totrue. If you don't set it, you get a deprecation warning. If you don't specify any type when creating an index,_docis used.
- Elasticsearch version 7
- Specifying types is deprecated in this Elasticsearch version. The
include_type_nameparameter defaults tofalsenow. - You can force Elasticsearch to use a type name by setting the parameter to
true, which results in a deprecation warning.
- Specifying types is deprecated in this Elasticsearch version. The
- Elasticsearch version 8
- Specifying types is no longer supported. Starting with this version, you are required to use
_docas the document type. For more information, see Removal of mapping types.
- Specifying types is no longer supported. Starting with this version, you are required to use
Type refers to the "index type" used in Elasticsearch and doesn't restrict which user sessions are exported. Regardless of the type you choose to configure, all user session data—including user actions, events, and errors—is exported.
To restrict the user sessions you export, you can define a management zone. See Export scope for details.
4. Export scope
You can define the scope of your export by selecting a management zone in the Export scope, alerting, and advanced configuration section.
After selecting a management zone, only user sessions that have at least one user action with a matching application are included in the export.
Restricting the export to synthetic only user sessions isn't currently possible.
Advanced configuration
You can configure additional details of your user-session export under Export scope, alerting, and advanced configuration.
- You can turn on Disable notification if you don't want to receive notifications when your user session export fails.
- You can specify Custom configuration properties to further tweak the export configuration. Contact Dynatrace support before changing this field.
- If you experience any issues with user session export, contact Dynatrace support before entering any additional properties.
5. Test export
To test your configuration, select Test export. Dynatrace uses the current configuration to perform a user session export of up to 50 documents from the last seven days. If no data is available during this timeframe, the test export isn't available. As soon as the test export finishes, you are notified of the results.
- You don't need to save a configuration to test your endpoint.
- If you set up an endpoint that is secured via basic authentication, you need to test your configuration before saving it.
- You need to re-enter your basic authentication password when testing your configuration.
6. Download a sample dataset
To see what data export will look like when you export it to your endpoints, select Download sample export data.
- The sample dataset consists of up to 50 user sessions tracked in your environment for the last seven days. If there aren't any user sessions during this timeframe, downloading sample data isn't available.
- If you configured a regular endpoint for your user session export, the sample data will contain user sessions in JSON format, separated by newline characters.
If you configured your endpoint to send data directly to Elasticsearch, the sample data also contains header lines as shown in the example below. These are used to tell Elasticsearch what to do with the data.
7. Download a sample mapping
To export user session data directly to your own Elasticsearch instance, you can download a sample mapping for your indexes by selecting Download mapping. The downloaded template mapping file contains a mapping for each exported field.
The created sample mapping reflects your current settings and can immediately be used when creating the index where user session data are to be exported:
PUT /my-usersession-index
{
"settings": {
"index": {
"number_of_shards": 3,
"number_of_replicas": 1
}
},
"mappings": {
"properties": {
// add the mappings from the downloaded file here ...
}
}
}Example:
{
"mappings" : {
"dynamic_templates" : [ {
"string_fields" : {
"match" : "*",
"match_mapping_type" : "string",
"mapping" : {
"norms" : "false",
"type" : "keyword"
}
}
} ],
"properties" : {
"applicationType" : {
"type" : "keyword"
},
"appVersion" : {
"type" : "keyword"
},
// ...
}
}
}Use Settings 2.0 REST API
The user session export configuration is stored using the Settings 2.0 framework. This provides a REST API that you can use to create, read, update, and delete your endpoint configurations. See Settings API for details.
-
Endpoints to access the Settings API:
- For Dynatrace SaaS deployments:
https://{your-environment-id}.live.dynatrace.com/api/v2/settings/objects/{objectId} - For Dynatrace Managed deployments:
https://{your-domain}/e/{your-environment-id}/api/v2/settings/objects/{objectId} - For Environment ActiveGate:
https://{your-activegate-domain}/e/{your-environment-id}/api/v2/settings/objects/{objectId}
- For Dynatrace SaaS deployments:
-
Address for OpenAPI documentation:
https://{your-environment-id}.live.dynatrace.com/rest-api-doc/index.jsp?urls.primaryName=Environment%20API%20v2#/Settings - Objects. -
Schema ID for the user session export settings:
builtin:elasticsearch.user-session-export-settingsWith this schema ID, you can, for example, read your currently configured user session export settings using any REST client:GET /api/v2/settings/object?schemaIds=builtin:elasticsearch.user-session-export-settings&scopes=tenant
To add an endpoint via the REST API:
PUT /api/v2/settings/object?schemaIds=builtin:elasticsearch.user-session-export-settings&scopes=tenant
{
"value": {
"values": [{
"basicAuth": {
"active": "false"
},
"endpointDefinition": {
"contentType": "application/json",
"enableUserSessionExport": true,
"endpointUrl": "https://my-endpoint.com",
"usePost": false
},
"exportBehavior": {
"disableNotification": false
},
"sendDirect": {
"active": false
}
}]
}
}For details on how to update a settings object, see Settings API - PUT an object.
Troubleshoot
To verify sending data, run the command below. You can set the following additional flags as needed:
--insecure- to disable the SSL check--http1.1- if the command returns aREFUSED_STREAMerror
curl -v -H "Content-Type: application/json" -X PUT -d '{"visitorId":"14804637803609BCTKP776NMJBOIF3R8OD6R0E4NQALJO","visitId":"16229530","startTime":1480463779085,"endTime":1480463784889,"visitType":"SYNTHETIC"}' http://localhost:3000/export/events