Instrument C++ applications with OpenTelemetry
This guide shows how to instrument your C++ application with OpenTelemetry and export the traces to Dynatrace.
To learn more about how Dynatrace works with OpenTelemetry, see Send data to Dynatrace with OpenTelemetry.
Prerequisites
- Dynatrace version 1.222+
- W3C Trace Context is enabled
- From the Dynatrace menu, go to Settings > Preferences > OneAgent features.
- Turn on Send W3C Trace Context HTTP headers.
OpenTelemetry-cpp prerequisites
- A supported development platform.
- A compatible C++ compiler.
- Git for fetching opentelemetry-cpp source code from repository. For details, see how to set up git.
- CMake for building opentelemetry-cpp API and SDK. For details, see how to install CMake.
OpenTelemetry-cpp OTLP exporter prerequisites
- Protobuf library to serialize structured data. For details, see how to install Protobuf.
- libcurl the multiprotocol file transfer library. For details, see (libcurl documentation).
- nlohmann/json the JSON for Modern C++ (you can use cmake to build and install nlohmann/json from its github repository).
Build and install the OpenTelemetry-cpp
Now that you have all the dependencies, let's build the opentelemetry-cpp using CMake.
We will follow the steps from OpenTelemetry, with some adjustments to also build the OTLP_HTTP exporter.
-
Get the opentelemetry-cpp source in the directory where you want to create the code repository:
git clone https://github.com/open-telemetry/opentelemetry-cpp
-
Set an environment variable with the OpenTelemetry C++ version:
export OPENTELEMETRY_CPP_VERSION=1.8.3
-
Navigate to the cloned repository and create the CMake build configuration:
cd opentelemetry-cpp git checkout tags/v${OPENTELEMETRY_CPP_VERSION} -b v${OPENTELEMETRY_CPP_VERSION} mkdir build && cd build cmake .. -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_BUILD_TYPE=Release \ -DBUILD_TESTING=OFF -DWITH_EXAMPLES=OFF -DWITH_OTLP=ON \ -DWITH_OTLP_HTTP=ON -DWITH_OTLP_GRPC=OFF -DOPENTELEMETRY_INSTALL=ON
-
Install C++ OpenTelemetry API and SDK:
make -j$(nproc || sysctl -n hw.ncpu || echo 1) install
Overview
To monitor your C++ application with OpenTelemetry
Instrument your application
Send the data to Dynatrace
Configure context propagation
Restart your application and verify the data in Dynatrace
Configure data capture to meet privacy requirements
Instrument your application
C++ instrumentation requires some manual steps. Add the following to your CMakeLists.txt
file and add the code snippet below to any C++ method you want to monitor.
find_package(CURL REQUIRED)
find_package(Protobuf REQUIRED)
find_package(opentelemetry-cpp CONFIG REQUIRED)
include_directories("${OPENTELEMETRY_CPP_INCLUDE_DIRS}")
target_link_libraries(
<YOUR_EXE_NAME> ${OPENTELEMETRY_CPP_LIBRARIES} opentelemetry_trace
opentelemetry_common opentelemetry_http_client_curl
opentelemetry_exporter_otlp_http opentelemetry_exporter_otlp_http_client
opentelemetry_otlp_recordable opentelemetry_resources
opentelemetry_metrics opentelemetry_exporter_otlp_http_metric)
Set names for the tracer and the span, and add attributes as you see fit.
#include "tracer_common.h" // header file that will be created on the next step
using namespace opentelemetry::trace;
// your method
{
StartSpanOptions options;
options.kind = SpanKind::kServer; // TODO Set the Span Kind
std::string span_name = "my-span" // TODO Replace with the name of your span
auto span = get_tracer("my-tracer") // TODO Replace with the name of your tracer
->StartSpan(span_name, {
{"my-key-1", "my-value-1"} // TODO Add attributes
}, options);
auto scope = get_tracer("my-span")->WithActiveSpan(span);
// your code goes here
span->End();
}
Send data to Dynatrace
To send data to Dynatrace with OneAgent, you have to create and configure the following header file, and add it to your C++ application:
// tracer_common.h
#pragma once
#include "opentelemetry/exporters/otlp/otlp_http_exporter_factory.h"
#include "opentelemetry/exporters/otlp/otlp_http_exporter_options.h"
#include "opentelemetry/context/propagation/global_propagator.h"
#include "opentelemetry/context/propagation/text_map_propagator.h"
#include "opentelemetry/exporters/ostream/span_exporter_factory.h"
#include "opentelemetry/nostd/shared_ptr.h"
#include "opentelemetry/sdk/trace/simple_processor_factory.h"
#include "opentelemetry/sdk/trace/tracer_context_factory.h"
#include "opentelemetry/sdk/trace/tracer_provider_factory.h"
#include "opentelemetry/trace/propagation/http_trace_context.h"
#include "opentelemetry/trace/provider.h"
#include <cstring>
#include <iostream>
#include <vector>
#include <fstream>
#include <list>
using namespace std;
namespace nostd = opentelemetry::nostd;
namespace otlp = opentelemetry::exporter::otlp;
namespace sdktrace = opentelemetry::sdk::trace;
namespace resource = opentelemetry::sdk::resource;
namespace
{
template <typename T>
class HttpTextMapCarrier : public opentelemetry::context::propagation::TextMapCarrier
{
public:
HttpTextMapCarrier<T>(T &headers) : headers_(headers) {}
HttpTextMapCarrier() = default;
virtual nostd::string_view Get(nostd::string_view key) const noexcept override
{
std::string key_to_compare = key.data();
if (key == opentelemetry::trace::propagation::kTraceParent)
{
key_to_compare = "Traceparent";
}
else if (key == opentelemetry::trace::propagation::kTraceState)
{
key_to_compare = "Tracestate";
}
auto it = headers_.find(key_to_compare);
if (it != headers_.end())
{
return it->second;
}
return "";
}
virtual void Set(nostd::string_view key, nostd::string_view value) noexcept override
{
headers_.insert(std::pair<std::string, std::string>(std::string(key), std::string(value)));
}
T headers_;
};
otlp::OtlpHttpExporterOptions opts;
void initTracer()
{
opts.url = "http://localhost:14499/otlp/v1/traces";
opts.content_type = exporter::HttpRequestContentType::kBinary;
resource::ResourceAttributes resource_attributes = {
{"service.name", "cpp-quickstart"}, //TODO Replace with the name of your application
{"service.version", "1.0.1"} //TODO Replace with the version of your application
};
resource::ResourceAttributes dt_resource_attributes;
try
{
for (string name : {"dt_metadata_e617c525669e072eebe3d0f08212e8f2.properties", "/var/lib/dynatrace/enrichment/dt_metadata.properties"})
{
string file_path;
ifstream dt_file;
dt_file.open(name);
if (dt_file.is_open())
{
string dt_metadata;
ifstream dt_properties;
while (getline(dt_file, file_path))
{
dt_properties.open(file_path);
if (dt_properties.is_open())
{
while (getline(dt_properties, dt_metadata))
{
dt_resource_attributes.SetAttribute(
dt_metadata.substr(0, dt_metadata.find("=")),
dt_metadata.substr(dt_metadata.find("=") + 1)
);
}
dt_properties.close();
}
}
dt_file.close();
}
}
}
catch (...) {}
auto dt_resource = resource::Resource::Create(dt_resource_attributes);
auto resource = resource::Resource::Create(resource_attributes);
auto merged_resource = dt_resource.Merge(resource);
auto exporter = otlp::OtlpHttpExporterFactory::Create(opts);
auto processor =
sdktrace::SimpleSpanProcessorFactory::Create(std::move(exporter));
std::vector<std::unique_ptr<sdktrace::SpanProcessor>> processors;
processors.push_back(std::move(processor));
std::shared_ptr<sdktrace::TracerContext> context =
sdktrace::TracerContextFactory::Create(std::move(processors), merged_resource);
std::shared_ptr<opentelemetry::trace::TracerProvider> provider =
sdktrace::TracerProviderFactory::Create(context);
opentelemetry::trace::Provider::SetTracerProvider(provider);
opentelemetry::context::propagation::GlobalTextMapPropagator::SetGlobalPropagator(
opentelemetry::nostd::shared_ptr<opentelemetry::context::propagation::TextMapPropagator>(
new opentelemetry::trace::propagation::HttpTraceContext()));
}
opentelemetry::nostd::shared_ptr<opentelemetry::trace::Tracer> get_tracer(std::string tracer_name)
{
auto provider = opentelemetry::trace::Provider::GetTracerProvider();
return provider->GetTracer(tracer_name);
}
}
When using OneAgent, make sure to enable the public Extension Execution Controller in your Dynatrace Settings, otherwise no data will be sent.
In the Dynatrace menu, go to Settings > Preferences > Extension Execution Controller. The toggles Enable Extension Execution Controller and Enable local PIPE/HTTP metric and Log Ingest API should be active.
To send data to Dynatrace without OneAgent, you have to create and configure the following header file, and add it to your C++ application:
// tracer_common.h
#pragma once
#include "opentelemetry/exporters/otlp/otlp_http_exporter_factory.h"
#include "opentelemetry/exporters/otlp/otlp_http_exporter_options.h"
#include "opentelemetry/context/propagation/global_propagator.h"
#include "opentelemetry/context/propagation/text_map_propagator.h"
#include "opentelemetry/exporters/ostream/span_exporter_factory.h"
#include "opentelemetry/nostd/shared_ptr.h"
#include "opentelemetry/sdk/trace/simple_processor_factory.h"
#include "opentelemetry/sdk/trace/tracer_context_factory.h"
#include "opentelemetry/sdk/trace/tracer_provider_factory.h"
#include "opentelemetry/trace/propagation/http_trace_context.h"
#include "opentelemetry/trace/provider.h"
#include <cstring>
#include <iostream>
#include <vector>
#include <fstream>
#include <list>
using namespace std;
namespace nostd = opentelemetry::nostd;
namespace otlp = opentelemetry::exporter::otlp;
namespace sdktrace = opentelemetry::sdk::trace;
namespace resource = opentelemetry::sdk::resource;
namespace
{
template <typename T>
class HttpTextMapCarrier : public opentelemetry::context::propagation::TextMapCarrier
{
public:
HttpTextMapCarrier<T>(T &headers) : headers_(headers) {}
HttpTextMapCarrier() = default;
virtual nostd::string_view Get(nostd::string_view key) const noexcept override
{
std::string key_to_compare = key.data();
if (key == opentelemetry::trace::propagation::kTraceParent)
{
key_to_compare = "Traceparent";
}
else if (key == opentelemetry::trace::propagation::kTraceState)
{
key_to_compare = "Tracestate";
}
auto it = headers_.find(key_to_compare);
if (it != headers_.end())
{
return it->second;
}
return "";
}
virtual void Set(nostd::string_view key, nostd::string_view value) noexcept override
{
headers_.insert(std::pair<std::string, std::string>(std::string(key), std::string(value)));
}
T headers_;
};
otlp::OtlpHttpExporterOptions opts;
void initTracer()
{
opts.url = "<URL>"; //TODO Replace <URL> to your SaaS/Managed-URL as mentioned in the next step
opts.content_type = otlp::HttpRequestContentType::kBinary;
opts.http_headers.insert(
std::make_pair<const std::string, std::string>("Authorization", "Api-Token <TOKEN>")); //TODO Replace <TOKEN> with your API Token as mentioned in the next step
resource::ResourceAttributes resource_attributes = {
{"service.name", "cpp-quickstart"}, //TODO Replace with the name of your application
{"service.version", "1.0.1"} //TODO Replace with the version of your application
};
resource::ResourceAttributes dt_resource_attributes;
try
{
for (string name : {"dt_metadata_e617c525669e072eebe3d0f08212e8f2.properties", "/var/lib/dynatrace/enrichment/dt_metadata.properties"})
{
string file_path;
ifstream dt_file;
dt_file.open(name);
if (dt_file.is_open())
{
string dt_metadata;
ifstream dt_properties;
while (getline(dt_file, file_path))
{
dt_properties.open(file_path);
if (dt_properties.is_open())
{
while (getline(dt_properties, dt_metadata))
{
dt_resource_attributes.SetAttribute(
dt_metadata.substr(0, dt_metadata.find("=")),
dt_metadata.substr(dt_metadata.find("=") + 1)
);
}
dt_properties.close();
}
}
dt_file.close();
}
}
}
catch (...) {}
auto dt_resource = resource::Resource::Create(dt_resource_attributes);
auto resource = resource::Resource::Create(resource_attributes);
auto merged_resource = dt_resource.Merge(resource);
auto exporter = otlp::OtlpHttpExporterFactory::Create(opts);
auto processor =
sdktrace::SimpleSpanProcessorFactory::Create(std::move(exporter));
std::vector<std::unique_ptr<sdktrace::SpanProcessor>> processors;
processors.push_back(std::move(processor));
std::shared_ptr<sdktrace::TracerContext> context =
sdktrace::TracerContextFactory::Create(std::move(processors), merged_resource);
std::shared_ptr<opentelemetry::trace::TracerProvider> provider =
sdktrace::TracerProviderFactory::Create(context);
opentelemetry::trace::Provider::SetTracerProvider(provider);
opentelemetry::context::propagation::GlobalTextMapPropagator::SetGlobalPropagator(
opentelemetry::nostd::shared_ptr<opentelemetry::context::propagation::TextMapPropagator>(
new opentelemetry::trace::propagation::HttpTraceContext()));
}
opentelemetry::nostd::shared_ptr<opentelemetry::trace::Tracer> get_tracer(std::string tracer_name)
{
auto provider = opentelemetry::trace::Provider::GetTracerProvider();
return provider->GetTracer(tracer_name);
}
}
Lastly, you need to define the correct endpoint and token, to make sure your data arrives where it should be.
- To set the endpoint:
- Use your Environment ID to set the endpoint to which your app will send traces as follows:
- Dynatrace SaaS
https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/traces
- Dynatrace Managed
https://{your-domain}/e/{your-environment-id}/api/v2/otlp/v1/traces
- Dynatrace ActiveGate
https://{your-activegate-endpoint}/e/{your-environment-id}/api/v2/otlp/v1/traces
- You may need to include the port to your ActiveGate endpoint. For example:
https://{your-activegate-endpoint}:9999/e/{your-environment-id}/api/v2/otlp/v1/traces
- If you are running a containerized ActiveGate, you need to use the FQDN of it. For example:
https://{your-activegate-service-name}.dynatrace.svc.cluster.local/e/{your-environment-id}/api/v2/otlp/v1/traces
- You may need to include the port to your ActiveGate endpoint. For example:
- Dynatrace SaaS
- Replace
<URL>
in the code snippet above with your endpoint.
- Use your Environment ID to set the endpoint to which your app will send traces as follows:
- To create an authentication token
- In the Dynatrace menu, go to Access tokens and select Generate new token.
- Provide a Token name.
- In the Search scopes box, search for
Ingest OpenTelemetry traces
and select the checkbox. - Select Generate token.
- Select Copy to copy the token to your clipboard.
- Save the token in a safe place; you can't display it again.
- Replace
<TOKEN>
in the code snippet above with your token.
Configure context propagation optional
If your application receives a request or calls other applications, you need to configure context propagation to make sure the spans are linked together.
- Whenever receiving an incoming request, you need to extract the parent context and create the new span as a child of it. Here is how:
#include "opentelemetry/trace/context.h"
#include "tracer_common.h" // header file created in the step above
using namespace opentelemetry::trace;
namespace context = opentelemetry::context;
// your method
{
StartSpanOptions options;
options.kind = SpanKind::kServer;
// extract context from http header
HttpTextMapCarrier<http_client::Headers> carrier;
auto prop = context::propagation::GlobalTextMapPropagator::GetGlobalPropagator();
auto current_ctx = context::RuntimeContext::GetCurrent();
auto new_context = prop->Extract(carrier, current_ctx);
options.parent = GetSpan(new_context)->GetContext();
// start scoped span with parent context extracted from http header
auto span = get_tracer("manual-server")
->StartSpan("my-server-span", { //TODO Replace with the name of your span
{"my-server-key-1", "my-server-value-1"} //TODO Add attributes
}, options);
auto scope = get_tracer("http_server")->WithActiveSpan(span);
// your code goes here
span->End();
}
- If your application calls another service, you need to ensure that you propagate the context, adding it to your outgoing request. Here is how:
#include "opentelemetry/ext/http/client/http_client_factory.h"
#include "opentelemetry/ext/http/common/url_parser.h"
#include "opentelemetry/trace/context.h"
#include "tracer_common.h" // header file created in the step above
using namespace opentelemetry::trace;
namespace context = opentelemetry::context;
namespace http_client = opentelemetry::ext::http::client;
// your method
{
auto http_client = http_client::HttpClientFactory::CreateSync();
std::string url = "<HTTP_URL>";
// start scoped active span
StartSpanOptions options;
options.kind = SpanKind::kClient;
opentelemetry::ext::http::common::UrlParser url_parser(url);
std::string span_name = url_parser.path_; // Fetch the URL path to be used as span name
auto span = get_tracer("http-client")
->StartSpan(span_name, {
{"my-client-key-1", "my-client-value-1"} //TODO Add attributes
}, options);
auto scope = get_tracer("http-client")->WithActiveSpan(span);
// inject current context into http header
auto current_ctx = context::RuntimeContext::GetCurrent();
HttpTextMapCarrier<http_client::Headers> carrier;
auto prop = context::propagation::GlobalTextMapPropagator::GetGlobalPropagator();
prop->Inject(carrier, current_ctx);
// send http request
http_client::Result result = http_client->Get(url, carrier.headers_);
//your code goes here
span->End();
}
Verify that the traces are ingested into Dynatrace
A few minutes after restarting your app, look for your spans:
- In the Dynatrace menu, go to Distributed traces and select the Ingested traces tab.
- If you use OneAgent, go to Distributed traces and select the PurePaths tab.
- Your spans will be part of an existing PurePath, if the root of your call is already monitored by OneAgent.
If your application does not receive any traffic, there will be no traces.
Configure data capture to meet privacy requirements optional
While Dynatrace automatically captures all OpenTelemetry resource and span attributes, only attribute values specified in the allowlist are stored and displayed in the Dynatrace web UI. This prevents accidental storage of personal data, so you can meet your privacy requirements and control the amount of monitoring data that's stored.
To view your custom span attributes, you need to allow them in the Dynatrace web UI first:
- Span attributes In the Dynatrace menu, go to Settings and select Server-side service monitoring > Span attributes.
- Resource attributes In the Dynatrace menu, go to Settings and select Server-side service monitoring > Resource attributes.