Instrument Rust applications with OpenTelemetry

This guide shows how to instrument your Rust application with OpenTelemetry and export the traces to Dynatrace. To learn more about how Dynatrace works with OpenTelemetry, see Send data to Dynatrace with OpenTelemetry.

Prerequisites

The following prerequisites and limitations apply:

  • Dynatrace version 1.222+
  • W3C Trace Context is enabled
    1. From the Dynatrace menu, go to Settings and select Server-side service monitoring > Deep monitoring > Distributed tracing.
    2. Turn on Send W3C Trace Context HTTP headers.

Instrument the application

Dynatrace uses OpenTelemetry Trace Ingest to provide end-to-end visibility into your Rust applications. To instrument the application:

  1. Install the required dependencies
  2. Get the endpoint and authentication token
  3. Configure the OpenTelemetry Exporter

1. Add the following crates to your Cargo.toml file

opentelemetry = { version = "0.16.0", features =  ["rt-tokio"] }
opentelemetry-otlp = { version = "0.9.0", features = ["http-proto", "reqwest-client", "reqwest-rustls"] }
opentelemetry-http = { version =  "0.5.0" }
opentelemetry-semantic-conventions = { version = "0.8.0" }

2. Get the endpoint and authentication token

To determine the endpoint to which your app will send traces

  1. Open Dynatrace.
  2. Check the address line of your browser. The URL will match one of the following patterns:
    • Dynatrace SaaS: https://{your-environment-id}.live.dynatrace.com/...
    • Dynatrace Managed: https://{your-domain}/e/{your-environment-id}/...
  3. Replace the ... part with api/v2/otlp/v1/traces to get the URL you will need in section 3 below.
    • Dynatrace SaaS: https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/traces
    • Dynatrace Managed: https://{your-domain}/e/{your-environment-id}/api/v2/otlp/v1/traces

To create an authentication token

  1. In the Dynatrace menu, go to Access tokens and select Generate new token.
  2. Provide a Token name.
  3. In the Search scopes box, search for Ingest OpenTelemetry traces and select the checkbox.
  4. Select Generate token.
  5. Select Copy to copy the token to your clipboard.
  6. Save the token in a safe place; you can't display it again, and you will need it in section 3 below.

3. Add the OpenTelemetry tracer to your code

To make sure all the requests are linked together in your PurePath®, you need to configure the context propagation (learn more about W3C Trace Context).

  • First you need to create an init_tracer function:
use std::collections::HashMap;

use opentelemetry::{
    global,
    sdk::{propagation::TraceContextPropagator, trace as sdktrace, Resource},
    trace::{Span, TraceContextExt, TraceError, Tracer},
    Context, Key,
};

use opentelemetry_http::{HeaderExtractor, HeaderInjector};
use opentelemetry_otlp::WithExportConfig;
use opentelemetry_semantic_conventions as semcov;

fn init_tracer() -> Result<sdktrace::Tracer, TraceError> {
    global::set_text_map_propagator(TraceContextPropagator::new());

    let mut map = HashMap::new();
    //TODO replace <TOKEN> with the authentication token created in section 2 above
    map.insert("Authorization".to_string(), "Api-Token <TOKEN>".to_string());

    opentelemetry_otlp::new_pipeline()
        .tracing()
        .with_exporter(
            opentelemetry_otlp::new_exporter()
                .http()
                .with_endpoint("<URL>") //TODO replace <URL> with the URL as determined in section 2 above
                .with_headers(map),
        )
        .with_trace_config(sdktrace::config().with_resource(Resource::new(vec![
            //customizable resource attributes
            semcov::resource::SERVICE_NAME.string("rust-quickstart"),
            semcov::resource::SERVICE_VERSION.string("1.0.1"),
        ])))
        .install_batch(opentelemetry::runtime::Tokio)
}
  • In your main method, make sure to initialize the function and shut it down at the end of your method:
#[tokio::main]
pub async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
    let _ = init_tracer()?;
    
    //Your code goes here

    global::shutdown_tracer_provider();
}
  • Whenever receiving an incoming request, you need to ensure that the parent context is extracted and the span is created as a child of it. Here is how:
//Method to extract the parent context from the request
fn get_parent_context(req: Request<Body>) -> Context {
    global::get_text_map_propagator(|propagator| {
        propagator.extract(&HeaderExtractor(req.headers()))
    })
}

async fn incoming_request(req: Request<Body>) -> Result<Response<Body>, Infallible> {
    let parent_cx = get_parent_context(req);

    let mut span = global::tracer("server").start_with_context("GET /", parent_cx.clone().into());

    //Use span.set_attribute to add attributes to your span
    span.set_attribute(semcov::trace::HTTP_URL.string("HTTP_URL"));
    span.set_attribute(semcov::trace::HTTP_METHOD.string("GET"));
    span.set_attribute(semcov::trace::HTTP_STATUS_CODE.i64(200));
    span.set_attribute(Key::new("custom_key_1").string("custom_attribute_1"));
    span.set_attribute(Key::new("custom_key_N").string("custom_attribute_N"));

    //Your code goes here
}
  • If your application calls another service, you need to ensure that you propagate the context, adding it to your outgoing request. Here is how:
async fn outgoing_request(
    context: Context,
) -> std::result::Result<(), Box<dyn std::error::Error + Send + Sync + 'static>> {
    let client = Client::new();
    let span = global::tracer("client").start_with_context("outgoing_request", context.clone().into());
    let cx = Context::current_with_span(span);

    let mut req = hyper::Request::builder().uri("<HTTP_URL>");

    //Method to inject the current context in the request
    global::get_text_map_propagator(|propagator| {
        propagator.inject_context(&cx, &mut HeaderInjector(&mut req.headers_mut().unwrap()))
    });

    //Use set_attribute to add attributes to your span
    cx.span()
        .set_attribute(Key::new("child_key").string("child_attribute"));

    //Your code goes here
}

Verify that the traces are ingested into Dynatrace

A few minutes after restarting your app, look for your spans:

  • In the Dynatrace menu, go to Distributed traces and select the Ingested traces tab.
  • Your spans will be part of an existing PurePath, if the root of your call is already being monitored by the OneAgent.

If your app is not getting any traffic, there will be no traces.

(Optional) Configure data capture to meet privacy requirements

While Dynatrace automatically captures all OpenTelemetry resource and span attributes, only attribute values specified in the allowlist are stored and displayed in the Dynatrace web UI. This prevents accidental storage of personal data, so you can meet your privacy requirements and control the amount of monitoring data that's stored.

To view your custom span attributes, you need to allow them in the Dynatrace web UI first:

  • Span attributes In the Dynatrace menu, go to Settings and select Server-side service monitoring > Span attributes.
  • Resource attributes In the Dynatrace menu, go to Settings and select Server-side service monitoring > Resource attributes.