Instrumentation

When you instrument your code or application, you enhance its capabilities to support, record, and aggregate telemetry data specific to your use case.

Instrumentation is done automatically, manually, or through a combination of both approaches.

Automatic instrumentation, which uses dedicated instrumentation tools and libraries, is the easiest and most convenient way of adding observability and telemetry to your application, as you only need to add the necessary tooling to your deployment and let it work essentially out of the box. However, automatic instrumentation is not supported for every use case; whether it's suited to your needs is highly dependent on your application's runtime environment.
Manual instrumentation, which is achieved by extending your codebase to record relevant data, is more complex. It requires code changes and you need to decide which data should be collected where. On the other hand, manual instrumentation also allows your telemetry to work at the most granular level and supports scenarios that are anywhere from difficult to impossible with automatic instrumentation alone.

Generally, the closer your code is to native machine code, the more difficult it is to automatically instrument your code.

Using OpenTelemetry with Kubernetes containers

If you are using a container setup with Kubernetes, you can also use OpenTelemetry's Kubernetes Operator to manage the telemetry setup and add auto-instrumentation to your container applications.

Automatic instrumentation

Instrumenting an application automatically is typically achieved by using a set of language-specific libraries and toolchains added to the runtime environment.

For example, on Java, a dedicated JVM agent hooks into pre-defined JVM call-backs, monitors the execution of your application, and generates telemetry data. With other languages (for example, Go), integration might not be as self-contained and can require small manual changes to your setup in order to load and initialize OpenTelemetry.

Whether it's fully or partially automatic, automatic instrumentation also requires support for the libraries you are using. Java may lead here, with support for more than 100 libraries and frameworks and the most common application servers.

Once instrumentation is in place, its tooling uses the interfaces provided by the language or platform to hook into relevant parts of your execution flow (for example, by looking for known classes or functions) and create pre-defined OpenTelemetry signals (traces, metrics, and logs) for those parts.

Instrumented network libraries, for example, typically create spans with details such as network addresses, ports, or request methods, whereas instrumented database libraries may record details such as the query, number of returned entries, and timing.

Another aspect that is usually handled within this scope is context propagation. This applies to recreating contexts for inbound requests as well as propagating the current context on outbound requests.

Language support

The following is a list of languages currently supported by OpenTelemetry and their state of automatic instrumentation.

Full: OpenTelemetry can be integrated without changes to the codebase (for example, standalone agents and pre-loaders)
Partial: integrating OpenTelemetry requires some changes to the codebase (for example, initialization)
N/A: there is no support for automatic instrumentation; only manual instrumentation is possible

Language	Automatic instrumentation
C++	N/A
Erlang/Elixir	Partial
Go	Partial
Java	Full
JavaScript	Partial
.NET	Full
PHP	Partial
Python	Full
Ruby	Partial
Rust	N/A
Swift	Partial

Limitations

While automatic instrumentation is the most convenient approach to introducing telemetry to your application, there are limitations.

Availability

As you can see in the list above, full automatic support is still restricted to a handful of languages; most platforms require at least some smaller changes to your codebase.
Instrumentation libraries

The predefined signals issued by the instrumentation libraries might not cover all the areas you need for your use case.

Manual instrumentation can address both issues and is not mutually exclusive to automatic instrumentation. You can use both approaches side by side.

Python example

Adding automatic instrumentation to a Python application is pretty straightforward and only requires the installation of a few packages, a setup call, and an adjusted command line invocation.

Install the packages with pip.
```
bash
pip install opentelemetry-distro opentelemetry-exporter-otlp
```
This installs the opentelemetry-distro package, which contains the API, the SDK, and the two core command line utilities, opentelemetry-bootstrap and opentelemetry-instrument.
Configure the application with opentelemetry-bootstrap.

After the package is installed, run opentelemetry-bootstrap to install and configure automatic instrumentation for your application.
```
bash
opentelemetry-bootstrap -a install
```
This checks for already installed library packages and matches them to available instrumentation libraries. For example, if you are using the flask package, it adds support package opentelemetry-instrumentation-flask.
Run the application with opentelemetry-instrument.

Once all applicable packages are installed, start the application by wrapping the usual python execution command with a call to opentelemetry-instrument and pass a couple of additional configuration settings.
```
bash
opentelemetry-instrument \
    --traces_exporter console,otlp \
    --metrics_exporter console \
    --service_name your-service-name \
    --exporter_otlp_endpoint 0.0.0.0:4317 \
    python myapp.py
```

For details on automatic instrumentation for Python, see the Python / Automatic / Agent Configuration OpenTelemetry documentation.

Manual instrumentation

When automatic instrumentation is not an option, you can always instrument your application manually.

Manual instrumentation typically involves the following steps:

Add the OpenTelemetry SDK libraries to your application
Determine which information (traces and metrics) you need
Initialize the SDK in your code
Extend the relevant code parts with telemetry calls to the OpenTelemetry SDK
Ensure that context propagation is properly working (if applicable)

Each of these steps highly depends on the language and its SDK. For concrete examples, see Integration walk-throughs.

Traces

When instrumenting for tracing, it is important to keep track of the current context. The context is either established with a tracer provider or obtained via automatic instrumentation or context propagation.

Having access to the context (or tracer) will then allow you to create spans in the right locations of your code, mark them as active (important for the span hierarchy), attach relevant attributes and events, and set their status codes.

Context propagation with manual instrumentation

If your application uses distributed services, you need to pay particular attention to context propagation when handling instrumentation manually.

It is important to ensure your code restores a trace context it received from a calling service (context extraction) and pass your own context to other instrumented services you are calling from your application (context injection).

Metrics

Adding support for metrics to your application can be a bit easier than tracing, as there is no context propagation involved and no call or function hierarchies to take into account.

For starters, you need to initialize a meter provider object to take care of the heavy lifting (configuration, exporters, views). The meter provider is used to obtain a meter object, which provides access the actual instrument objects and comes with various create methods, each for a particular instrument (counter, histogram, gauge).

Synchronous and asynchronous instruments differ in their use:

Synchronous instruments require you to use the applicable instrument method (for example, add for counters) to alter the values directly.
Asynchronous instruments use callback functions that are called during metrics collection and provide the value in question.

Logs

OpenTelemetry logs are not supposed to be used directly and are never manually instrumented.

For details, see How to use OpenTelemetry logs

Exporting data

While not part of the instrumentation itself, exporting is still a vital part of any instrumented application, as it is the step where the collected data is published to the analytics backend (for example, Dynatrace), where it is aggregated, monitored, and further analyzed according to your configuration and requirements.

OTLP plays a central role in exporting data, as it is the core communication protocol used by all involved parties to exchange telemetry data.

In the case of Dynatrace, applications can export directly to ActiveGates or use OneAgent for traces-only exports. The latter supports automatic ingestion for a variety of language stacks and also provides a dedicated OTLP endpoint for traces. For more information, see Getting started.

It is also possible to export data to an intermediary proxy service such as OpenTelemetry's Collector and configure that service to forward the data to one of the options mentioned above.