Instrument C++ applications with OpenTelemetry

This guide shows how to instrument your C++ application with OpenTelemetry and export the traces to Dynatrace.

To learn more about how Dynatrace works with OpenTelemetry, see Send data to Dynatrace with OpenTelemetry.

Prerequisites

Dynatrace version 1.222+
W3C Trace Context is enabled
1. From the Dynatrace menu, go to Settings > Preferences > OneAgent features.
2. Turn on Send W3C Trace Context HTTP headers.

OpenTelemetry-cpp prerequisites

A supported development platform.
A compatible C++ compiler.
Git for fetching opentelemetry-cpp source code from repository. For details, see how to set up git.
CMake for building opentelemetry-cpp API and SDK. For details, see how to install CMake.

OpenTelemetry-cpp OTLP exporter prerequisites

Protobuf library to serialize structured data. For details, see how to install Protobuf.
libcurl the multiprotocol file transfer library. For details, see (libcurl documentation).
nlohmann/json the JSON for Modern C++ (you can use cmake to build and install nlohmann/json from its github repository).

Build and install the OpenTelemetry-cpp

Now that you have all the dependencies, let's build the opentelemetry-cpp using CMake.
We will follow the steps from OpenTelemetry, with some adjustments to also build the OTLP_HTTP exporter.

Get the opentelemetry-cpp source in the directory where you want to create the code repository:
```
shell
git clone https://github.com/open-telemetry/opentelemetry-cpp
```
Set an environment variable with the OpenTelemetry C++ version:
```
shell
export OPENTELEMETRY_CPP_VERSION=1.8.3
```

Navigate to the cloned repository and create the CMake build configuration:

shell
cd opentelemetry-cpp
git checkout tags/v${OPENTELEMETRY_CPP_VERSION} -b v${OPENTELEMETRY_CPP_VERSION}
mkdir build && cd build
cmake .. -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_BUILD_TYPE=Release \
        -DBUILD_TESTING=OFF -DWITH_EXAMPLES=OFF -DWITH_OTLP=ON \
        -DWITH_OTLP_HTTP=ON -DWITH_OTLP_GRPC=OFF -DOPENTELEMETRY_INSTALL=ON

Install C++ OpenTelemetry API and SDK:

shell
make -j$(nproc || sysctl -n hw.ncpu || echo 1) install

Overview

To monitor your C++ application with OpenTelemetry

Instrument your application

Send the data to Dynatrace

Configure context propagation

Restart your application and verify the data in Dynatrace

Configure data capture to meet privacy requirements

Instrument your application

C++ instrumentation requires some manual steps. Add the following to your CMakeLists.txt file and add the code snippet below to any C++ method you want to monitor.

cpp
find_package(CURL REQUIRED)
find_package(Protobuf REQUIRED)
find_package(opentelemetry-cpp CONFIG REQUIRED)

include_directories("${OPENTELEMETRY_CPP_INCLUDE_DIRS}")

target_link_libraries(
     <YOUR_EXE_NAME> ${OPENTELEMETRY_CPP_LIBRARIES} opentelemetry_trace
     opentelemetry_common opentelemetry_http_client_curl
     opentelemetry_exporter_otlp_http opentelemetry_exporter_otlp_http_client
     opentelemetry_otlp_recordable opentelemetry_resources
     opentelemetry_metrics opentelemetry_exporter_otlp_http_metric)

Set names for the tracer and the span, and add attributes as you see fit.

cpp
#include "tracer_common.h" // header file that will be created on the next step

using namespace opentelemetry::trace;

// your method
{
  StartSpanOptions options;
  options.kind          = SpanKind::kServer; // TODO Set the Span Kind
  std::string span_name = "my-span" // TODO Replace with the name of your span

  auto span = get_tracer("my-tracer") // TODO Replace with the name of your tracer
              ->StartSpan(span_name, {
                {"my-key-1", "my-value-1"} // TODO Add attributes
              }, options);

  auto scope = get_tracer("my-span")->WithActiveSpan(span);

  // your code goes here

  span->End();
}

Send data to Dynatrace

With OneAgentWithout OneAgent

To send data to Dynatrace with OneAgent, you have to create and configure the following header file, and add it to your C++ application:

cpp
// tracer_common.h

#pragma once

#include "opentelemetry/exporters/otlp/otlp_http_exporter_factory.h"
#include "opentelemetry/exporters/otlp/otlp_http_exporter_options.h"
#include "opentelemetry/context/propagation/global_propagator.h"
#include "opentelemetry/context/propagation/text_map_propagator.h"
#include "opentelemetry/exporters/ostream/span_exporter_factory.h"
#include "opentelemetry/nostd/shared_ptr.h"
#include "opentelemetry/sdk/trace/simple_processor_factory.h"
#include "opentelemetry/sdk/trace/tracer_context_factory.h"
#include "opentelemetry/sdk/trace/tracer_provider_factory.h"
#include "opentelemetry/trace/propagation/http_trace_context.h"
#include "opentelemetry/trace/provider.h"

#include <cstring>
#include <iostream>
#include <vector>
#include <fstream>
#include <list>

using namespace std;
namespace nostd    = opentelemetry::nostd;
namespace otlp     = opentelemetry::exporter::otlp;
namespace sdktrace = opentelemetry::sdk::trace;
namespace resource = opentelemetry::sdk::resource;

namespace
{
  template <typename T>
  class HttpTextMapCarrier : public opentelemetry::context::propagation::TextMapCarrier
  {
  public:
    HttpTextMapCarrier<T>(T &headers) : headers_(headers) {}
    HttpTextMapCarrier() = default;
    virtual nostd::string_view Get(nostd::string_view key) const noexcept override
    {
      std::string key_to_compare = key.data();
      if (key == opentelemetry::trace::propagation::kTraceParent)
      {
        key_to_compare = "Traceparent";
      }
      else if (key == opentelemetry::trace::propagation::kTraceState)
      {
        key_to_compare = "Tracestate";
      }
      auto it = headers_.find(key_to_compare);
      if (it != headers_.end())
      {
        return it->second;
      }
      return "";
    }

    virtual void Set(nostd::string_view key, nostd::string_view value) noexcept override
    {
      headers_.insert(std::pair<std::string, std::string>(std::string(key), std::string(value)));
    }

    T headers_;
  };

  otlp::OtlpHttpExporterOptions opts;

  void initTracer()
  {
    opts.url = "http://localhost:14499/otlp/v1/traces";

    opts.content_type  = exporter::HttpRequestContentType::kBinary;

    resource::ResourceAttributes resource_attributes = {
        {"service.name", "cpp-quickstart"}, //TODO Replace with the name of your application
        {"service.version", "1.0.1"} //TODO Replace with the version of your application
    };

    resource::ResourceAttributes dt_resource_attributes;

    try
    {
      for (string name : {"dt_metadata_e617c525669e072eebe3d0f08212e8f2.properties", "/var/lib/dynatrace/enrichment/dt_metadata.properties"})
      {
        string file_path;
        ifstream dt_file;
        dt_file.open(name);
        if (dt_file.is_open())
        {
          string dt_metadata;
          ifstream dt_properties;
          while (getline(dt_file, file_path))
          {
            dt_properties.open(file_path);
            if (dt_properties.is_open())
            {
              while (getline(dt_properties, dt_metadata))
              {
                dt_resource_attributes.SetAttribute(
                  dt_metadata.substr(0, dt_metadata.find("=")),
                  dt_metadata.substr(dt_metadata.find("=") + 1)
                );
              }
              dt_properties.close();
            }
          }
          dt_file.close();
        }
      }
    }
    catch (...) {}

    auto dt_resource = resource::Resource::Create(dt_resource_attributes);
    auto resource = resource::Resource::Create(resource_attributes);
    auto merged_resource = dt_resource.Merge(resource);

    auto exporter = otlp::OtlpHttpExporterFactory::Create(opts);
    auto processor =
        sdktrace::SimpleSpanProcessorFactory::Create(std::move(exporter));
    std::vector<std::unique_ptr<sdktrace::SpanProcessor>> processors;
    processors.push_back(std::move(processor));

    std::shared_ptr<sdktrace::TracerContext> context =
        sdktrace::TracerContextFactory::Create(std::move(processors), merged_resource);
    std::shared_ptr<opentelemetry::trace::TracerProvider> provider =
        sdktrace::TracerProviderFactory::Create(context);

    opentelemetry::trace::Provider::SetTracerProvider(provider);

    opentelemetry::context::propagation::GlobalTextMapPropagator::SetGlobalPropagator(
        opentelemetry::nostd::shared_ptr<opentelemetry::context::propagation::TextMapPropagator>(
            new opentelemetry::trace::propagation::HttpTraceContext()));
  }

  opentelemetry::nostd::shared_ptr<opentelemetry::trace::Tracer> get_tracer(std::string tracer_name)
  {
    auto provider = opentelemetry::trace::Provider::GetTracerProvider();
    return provider->GetTracer(tracer_name);
  }
}

When using OneAgent, make sure to enable the public Extension Execution Controller in your Dynatrace Settings, otherwise no data will be sent.

In the Dynatrace menu, go to Settings > Preferences > Extension Execution Controller. The toggles Enable Extension Execution Controller and Enable local PIPE/HTTP metric and Log Ingest API should be active.

To send data to Dynatrace without OneAgent, you have to create and configure the following header file, and add it to your C++ application:

cpp
// tracer_common.h

#pragma once

#include "opentelemetry/exporters/otlp/otlp_http_exporter_factory.h"
#include "opentelemetry/exporters/otlp/otlp_http_exporter_options.h"
#include "opentelemetry/context/propagation/global_propagator.h"
#include "opentelemetry/context/propagation/text_map_propagator.h"
#include "opentelemetry/exporters/ostream/span_exporter_factory.h"
#include "opentelemetry/nostd/shared_ptr.h"
#include "opentelemetry/sdk/trace/simple_processor_factory.h"
#include "opentelemetry/sdk/trace/tracer_context_factory.h"
#include "opentelemetry/sdk/trace/tracer_provider_factory.h"
#include "opentelemetry/trace/propagation/http_trace_context.h"
#include "opentelemetry/trace/provider.h"

#include <cstring>
#include <iostream>
#include <vector>
#include <fstream>
#include <list>

using namespace std;
namespace nostd    = opentelemetry::nostd;
namespace otlp     = opentelemetry::exporter::otlp;
namespace sdktrace = opentelemetry::sdk::trace;
namespace resource = opentelemetry::sdk::resource;

namespace
{
  template <typename T>
  class HttpTextMapCarrier : public opentelemetry::context::propagation::TextMapCarrier
  {
  public:
    HttpTextMapCarrier<T>(T &headers) : headers_(headers) {}
    HttpTextMapCarrier() = default;
    virtual nostd::string_view Get(nostd::string_view key) const noexcept override
    {
      std::string key_to_compare = key.data();
      if (key == opentelemetry::trace::propagation::kTraceParent)
      {
        key_to_compare = "Traceparent";
      }
      else if (key == opentelemetry::trace::propagation::kTraceState)
      {
        key_to_compare = "Tracestate";
      }
      auto it = headers_.find(key_to_compare);
      if (it != headers_.end())
      {
        return it->second;
      }
      return "";
    }

    virtual void Set(nostd::string_view key, nostd::string_view value) noexcept override
    {
      headers_.insert(std::pair<std::string, std::string>(std::string(key), std::string(value)));
    }

    T headers_;
  };

  otlp::OtlpHttpExporterOptions opts;

  void initTracer()
  {
    opts.url = "<URL>"; //TODO Replace <URL> to your SaaS/Managed-URL as mentioned in the next step

    opts.content_type  = otlp::HttpRequestContentType::kBinary;
    opts.http_headers.insert(
        std::make_pair<const std::string, std::string>("Authorization", "Api-Token <TOKEN>")); //TODO Replace <TOKEN> with your API Token as mentioned in the next step

    resource::ResourceAttributes resource_attributes = {
        {"service.name", "cpp-quickstart"}, //TODO Replace with the name of your application
        {"service.version", "1.0.1"} //TODO Replace with the version of your application
    };

    resource::ResourceAttributes dt_resource_attributes;

    try
    {
      for (string name : {"dt_metadata_e617c525669e072eebe3d0f08212e8f2.properties", "/var/lib/dynatrace/enrichment/dt_metadata.properties"})
      {
        string file_path;
        ifstream dt_file;
        dt_file.open(name);
        if (dt_file.is_open())
        {
          string dt_metadata;
          ifstream dt_properties;
          while (getline(dt_file, file_path))
          {
            dt_properties.open(file_path);
            if (dt_properties.is_open())
            {
              while (getline(dt_properties, dt_metadata))
              {
                dt_resource_attributes.SetAttribute(
                  dt_metadata.substr(0, dt_metadata.find("=")),
                  dt_metadata.substr(dt_metadata.find("=") + 1)
                );
              }
              dt_properties.close();
            }
          }
          dt_file.close();
        }
      }
    }
    catch (...) {}

    auto dt_resource = resource::Resource::Create(dt_resource_attributes);
    auto resource = resource::Resource::Create(resource_attributes);
    auto merged_resource = dt_resource.Merge(resource);

    auto exporter = otlp::OtlpHttpExporterFactory::Create(opts);
    auto processor =
        sdktrace::SimpleSpanProcessorFactory::Create(std::move(exporter));
    std::vector<std::unique_ptr<sdktrace::SpanProcessor>> processors;
    processors.push_back(std::move(processor));

    std::shared_ptr<sdktrace::TracerContext> context =
        sdktrace::TracerContextFactory::Create(std::move(processors), merged_resource);
    std::shared_ptr<opentelemetry::trace::TracerProvider> provider =
        sdktrace::TracerProviderFactory::Create(context);

    opentelemetry::trace::Provider::SetTracerProvider(provider);

    opentelemetry::context::propagation::GlobalTextMapPropagator::SetGlobalPropagator(
        opentelemetry::nostd::shared_ptr<opentelemetry::context::propagation::TextMapPropagator>(
            new opentelemetry::trace::propagation::HttpTraceContext()));
  }

  opentelemetry::nostd::shared_ptr<opentelemetry::trace::Tracer> get_tracer(std::string tracer_name)
  {
    auto provider = opentelemetry::trace::Provider::GetTracerProvider();
    return provider->GetTracer(tracer_name);
  }
}

Lastly, you need to define the correct endpoint and token, to make sure your data arrives where it should be.

To set the endpoint:
1. Use your Environment ID to set the endpoint to which your app will send traces as follows:
  - Dynatrace SaaS https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/traces
  - Dynatrace Managed https://{your-domain}/e/{your-environment-id}/api/v2/otlp/v1/traces
  - Dynatrace ActiveGate https://{your-activegate-endpoint}/e/{your-environment-id}/api/v2/otlp/v1/traces
    - You may need to include the port to your ActiveGate endpoint. For example:
      - https://{your-activegate-endpoint}:9999/e/{your-environment-id}/api/v2/otlp/v1/traces
    - If you are running a containerized ActiveGate, you need to use the FQDN of it. For example:
      - https://{your-activegate-service-name}.dynatrace.svc.cluster.local/e/{your-environment-id}/api/v2/otlp/v1/traces
2. Replace <URL> in the code snippet above with your endpoint.
To create an authentication token
1. In the Dynatrace menu, go to Access tokens and select Generate new token.
2. Provide a Token name.
3. In the Search scopes box, search for Ingest OpenTelemetry traces and select the checkbox.
4. Select Generate token.
5. Select Copy to copy the token to your clipboard.
6. Save the token in a safe place; you can't display it again.
7. Replace <TOKEN> in the code snippet above with your token.

Configure context propagation optional

If your application receives a request or calls other applications, you need to configure context propagation to make sure the spans are linked together.

Whenever receiving an incoming request, you need to extract the parent context and create the new span as a child of it. Here is how:

cpp
#include "opentelemetry/trace/context.h"
#include "tracer_common.h" // header file created in the step above

using namespace opentelemetry::trace;
namespace context = opentelemetry::context;

// your method
{
    StartSpanOptions options;
    options.kind = SpanKind::kServer;

    // extract context from http header
    HttpTextMapCarrier<http_client::Headers> carrier;
    auto prop        = context::propagation::GlobalTextMapPropagator::GetGlobalPropagator();
    auto current_ctx = context::RuntimeContext::GetCurrent();
    auto new_context = prop->Extract(carrier, current_ctx);
    options.parent   = GetSpan(new_context)->GetContext();

    // start scoped span with parent context extracted from http header
    auto span = get_tracer("manual-server")
                  ->StartSpan("my-server-span", { //TODO Replace with the name of your span
                              {"my-server-key-1", "my-server-value-1"} //TODO Add attributes
                            }, options);

    auto scope = get_tracer("http_server")->WithActiveSpan(span);

    // your code goes here

    span->End();
}

If your application calls another service, you need to ensure that you propagate the context, adding it to your outgoing request. Here is how:

cpp
#include "opentelemetry/ext/http/client/http_client_factory.h"
#include "opentelemetry/ext/http/common/url_parser.h"
#include "opentelemetry/trace/context.h"
#include "tracer_common.h" // header file created in the step above

using namespace opentelemetry::trace;
namespace context     = opentelemetry::context;
namespace http_client = opentelemetry::ext::http::client;

// your method
{
    auto http_client = http_client::HttpClientFactory::CreateSync();
    std::string url  = "<HTTP_URL>";

    // start scoped active span
    StartSpanOptions options;
    options.kind = SpanKind::kClient;
    opentelemetry::ext::http::common::UrlParser url_parser(url);

    std::string span_name = url_parser.path_; // Fetch the URL path to be used as span name
    auto span = get_tracer("http-client")
                  ->StartSpan(span_name, {
                              {"my-client-key-1", "my-client-value-1"} //TODO Add attributes
                            }, options);

    auto scope = get_tracer("http-client")->WithActiveSpan(span);

    // inject current context into http header
    auto current_ctx = context::RuntimeContext::GetCurrent();
    HttpTextMapCarrier<http_client::Headers> carrier;
    auto prop = context::propagation::GlobalTextMapPropagator::GetGlobalPropagator();
    prop->Inject(carrier, current_ctx);

    // send http request
    http_client::Result result = http_client->Get(url, carrier.headers_);

    //your code goes here

    span->End();
}

Verify that the traces are ingested into Dynatrace

A few minutes after restarting your app, look for your spans:

In the Dynatrace menu, go to Distributed traces and select the Ingested traces tab.
- If you use OneAgent, go to Distributed traces and select the PurePaths tab.
Your spans will be part of an existing PurePath, if the root of your call is already monitored by OneAgent.

If your application does not receive any traffic, there will be no traces.

Configure data capture to meet privacy requirements optional

While Dynatrace automatically captures all OpenTelemetry resource and span attributes, only attribute values specified in the allowlist are stored and displayed in the Dynatrace web UI. This prevents accidental storage of personal data, so you can meet your privacy requirements and control the amount of monitoring data that's stored.

To view your custom span attributes, you need to allow them in the Dynatrace web UI first:

Span attributes In the Dynatrace menu, go to Settings and select Server-side service monitoring > Span attributes.
Resource attributes In the Dynatrace menu, go to Settings and select Server-side service monitoring > Resource attributes.