Instrument your Go application with OpenTelemetry

This walkthrough shows how to add observability to your Go application using the OpenTelemetry Go libraries and tools.

Prerequisites

• Dynatrace version 1.222+
• For tracing, W3C Trace Context is enabled
  1. From the Dynatrace menu, go to Settings > Preferences > OneAgent features.
  2. Turn on Send W3C Trace Context HTTP headers.

Choose how to ingest data into Dynatrace

Choose how you want to instrument your application

OpenTelemetry supports on Go automatic and manual instrumentation, or a combination of both.

Initialize OpenTelemetry

The initialization of OpenTelemetry varies, depending on whether you use OneAgent to send data to Dynatrace or perform an OTLP export.

Automatically instrument your application optional

1. Browse the OpenTelemetry registry and pick the instrumentation libraries matching your application libraries.

2. Install the applicable support libraries using go get.

   shellcopydownload
   go get go.opentelemetry.io/contrib/instrumentation/[PACKAGE_DETAILS]

3. Add the package to your import statements.

   gocopydownload
   import (
     "go.opentelemetry.io/[PACKAGE]"
   )

4. Wrap your existing code with calls to the support libraries.

Example for net/http

1. Install the instrumentation library for net/http.

2. Add the package to your import statements.

   gocopydownload
   import (
     // other packages
     "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
   )

3. Wrap your HTTP handler function.

   gocopydownload
   handler := http.HandlerFunc(httpHandler)
   wrappedHandler := otelhttp.NewHandler(handler, "my-span") //TODO Replace with the name of your span
   
   //Use the wrappedHandler with your handle
   http.Handle("/", wrappedHandler)

Manually instrument your application

Add tracing

1. You first need to get a tracer object.

   gocopydownload
   tracer := otel.Tracer("my-tracer")

2. With tracer, you can now use a span builder to create and start new spans.

   gocopydownload
   _, span := tracer.Start(r.Context(), "Call to /myendpoint")
   defer span.End()
   
   span.SetAttributes(attribute.String("http.method", "GET"), attribute.String("net.protocol.version", "1.1"))
   
   // TODO your code goes here

   In the code above, we:

   • Create a new span and name it "Call to /myendpoint"
   • Schedule a deferred call to End(), to ensure the span is properly closed when the function returns
   • Add two attributes, following the semantic naming convention, specific to the action of this span: information on the HTTP method and version
   • Add a TODO in place of the eventual business logic

Collect metrics

1. Obtain a meter object.

   gocopydownload
   meter := otel.Meter("my-meter")

2. With meter, we can now create individual instruments, such as a counter.

   gocopydownload
   requestCounter, _ := meter.Int64Counter("request_counter")

3. Now we can invoke the Add() method of requestCounter to record new values with the counter.

   gocopydownload
   requestCounter.Add(context.Background(), 1)

Connect logs

OpenTelemetry logging is currently not yet available for Go and is still under development.

Ensure context propagation optional

Context propagation is particularly important when network calls (for example, REST) are involved.

Extracting the context when receiving a request

In the following example, we assume that we have received a network call via the net/http library and its Request type.

To obtain a handle to the original context (which was provided by the calling service), we pass the HTTP header object (r.Header) to the Extract function of the global propagator singleton, which instantiates that context and returns in parentCtx. This allows us to continue the previous trace with our own spans.

gocopydownload
func httpHandler(w http.ResponseWriter, r *http.Request) {
  parentCtx := otel.GetTextMapPropagator().Extract(r.Context(), propagation.HeaderCarrier(r.Header))

  ctx, span := tracer.Start(
    parentCtx,
    "manual-server", //TODO Replace with the name of your span
    trace.WithAttributes(
      attribute.String("my-key-1", "my-value-1") //TODO Add attributes
    )
  )
  defer span.End()

  //TODO your code goes here
}

Injecting the context when sending requests

In the following example, we set up a new instance of Request and pass the object to the Inject call of the global propagator singleton. This adds the necessary HTTP headers to the request object, which we eventually pass to Do to execute the network request.

gocopydownload
client := http.Client{}

req, err := http.NewRequest("<method>", "<url>", <body>)
if err != nil {
  // TODO handle error
}

//Method to inject the current context in the request headers
otel.GetTextMapPropagator().Inject(ctx, propagation.HeaderCarrier(req.Header))

client.Do(req) // Your call goes here

Configure data capture to meet privacy requirements optional

While Dynatrace automatically captures all OpenTelemetry resource and span attributes, only attribute values specified in the allowlist are stored and displayed in the Dynatrace web UI. This prevents accidental storage of personal data, so you can meet your privacy requirements and control the amount of monitoring data stored.

To view your custom span attributes, you need to allow them in the Dynatrace web UI first.

• Span attributes: In the Dynatrace menu, go to Settings and select Server-side service monitoring > Span attributes.
• Resource attributes: In the Dynatrace menu, go to Settings and select Server-side service monitoring > Resource attributes.

Verify data ingestion into Dynatrace

Once you have finished the instrumentation of your application, perform a couple of test actions to create and send demo traces, metrics, and logs and verify that they were correctly ingested into Dynatrace.

To do that for traces, in the Dynatrace menu, go to Distributed traces and select the Ingested traces tab. If you use OneAgent, select PurePaths instead.

Metrics and logs can be found under their respective entries at Observe and explore.