OpenTelemetry instrumentation guide

You can send OpenTelemetry trace data (traces and spans) in OTLP format to Dynatrace via an API available on Dynatrace ActiveGate. The ingested spans are integrated into PurePath® distributed traces.

Use this approach:

For services that cannot be instrumented by OneAgent
When the component you want to monitor already exposes trace data in OpenTelemetry format (OTLP)

If the service you want to monitor is already instrumented by OneAgent and you want to gain additional insight with OpenTelemetry, have a look at OneAgent OpenTracing and OpenTelemetry support instead.

Prerequisites

The following prerequisites and limitations apply:

Dynatrace version 1.222+
Only export format OTLP/HTTP binary Protobuf from OpenTelemetry is supported
W3C Trace Context is enabled
1. From the Dynatrace menu, go to Settings > Server-side service monitoring > Deep monitoring > Distributed tracing.
2. Turn on Send W3C Trace Context HTTP headers.

How to instrument and send trace data to Dynatrace

Follow the steps below.

Instrument your service with OpenTelemetry
(skip this step if your service is already instrumented)

Create an authentication token

Configure the OpenTelemetry exporter

Verify that the traces are ingested into Dynatrace

Configure data capture to meet privacy requirements

Check volume of ingested traces via metrics

Instrument your service with OpenTelemetry

If your service is not yet instrumented and exporting traces, follow the following quickstart guides:

Create an authentication token

The trace ingest API requires an API token with the Ingest OpenTelemetry traces (openTelemetryTrace.ingest) scope.

Configure the OpenTelemetry exporter

The Dynatrace API expects the trace data to be sent in the OTLP/HTTP binary Protobuf export format from OpenTelemetry. Currently, the OpenTelemetry SDK provides an exporter in that format for C++, Erlang/Elixir, Go, Java, Node.js, Python, Ruby, Rust, and .NET.

To configure the OTLP/HTTP OpenTelemetry exporter in your code

Set the URL as
- Dynatrace SaaS https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/traces
- Dynatrace Managed https://{your-domain}/e/{your-environment-id}/api/v2/otlp/v1/traces
Set the Authorization HTTP header as Api-Token <TOKEN>, where <TOKEN> is the API token you created earlier.

Follow our guide to configure the OTLP/HTTP exporter:

For other languages (such as PHP), you need to deploy an additional OpenTelemetry collector between the exporter and Dynatrace.

Sample collector configuration

plaintext
receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
exporters:
  otlphttp:
    endpoint: "https://mySampleEnv.live.dynatrace.com/api/v2/otlp"
    headers: {"Authorization": "Api-Token abcdevjhij1234567890"}
  logging:
    loglevel: debug
    sampling_initial: 5
    sampling_thereafter: 200
service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: []
      exporters: [otlphttp,logging]

Verify that the traces are ingested into Dynatrace

To view your OpenTelemetry traces, in the Dynatrace menu, go to Distributed traces and look for the traces.

The dsfm:server.spans.persisted metrics via the Data explorer or Metrics v2 API show you how many spans are being ingested by Dynatrace.

Configure data capture to meet privacy requirements optional

While Dynatrace automatically captures all OpenTelemetry resource and span attributes, only attribute values specified in the allowlist are stored and shown in the web UI. This prevents accidental storage of personal data, so you can meet your privacy requirements and control the amount of monitoring data that's being stored.

To view and edit the allowlists:

Span attributes In the Dynatrace menu, go to Settings > Server-side service monitoring > Span attributes.
Resource attributes In the Dynatrace menu, go to Settings > Server-side service monitoring > Resource attributes.

Check volume of ingested traces via metrics optional

To gain insight into the number of ingested OpenTelemetry traces, you have these metrics:

dsfm:server.spans.received—the number of spans received by the cluster from our OpenTelemetry endpoint.
dsfm:server.spans.persisted—the number of spans preserved by Dynatrace; only preserved spans are available for distributed traces analysis.

Frequently asked questions

There is a warning sign next to the spans. What does it mean?

There's a time sync issue between the exporter and Dynatrace, and the span had a time that preceded its parent.

Is there an additional cost for custom trace ingestion via the trace API?

The ingestion of spans via the trace API endpoint from a host consumes Full-Stack host units. When ingestion of spans is used in cloud services or environments/platforms not supported by OneAgent, ingestion consumes Davis data units.