Trace AWS Lambda .NET Core functions with OpenTelemetry .NET

To ingest gRPC via the Dynatrace Trace API, you need to use an OpenTelemetry collector between Dynatrace and the exporter. You can choose to either self-host a collector or use the AWS Distro for OpenTelemetry Collector (ADOT Collector).

If you use environment variables for setup, you need to set the following value:

• For OTEL_EXPORTER_OTLP_PROTOCOL: grpc

Add the ARN of the Lambda Layer ADOT Collector

Collector layer: aws-otel-collector-ver-0-27-0.

For a complete list of the AWS-managed OpenTelemetry Lambda layers, see AWS Distro for OpenTelemetry - AWS Lambda respository

Lambda layer ARN format is:

plaintextcopydownload
arn:aws:lambda:\<region>:901920570463:layer:<layer>:1

Configure ADOT Collector

The configuration of the ADOT Collector follows the OpenTelemetry standard.

By default, the ADOT Lambda layer uses the config.yaml file, which exports OpenTelemetry data to AWS X-Ray. To export the data to Dynatrace, you need to customize the configuration using the OpenTelemetry OTLP exporter.

To customize the collector configuration, add a configuration YAML file to your function code. After the file has been deployed with a Lambda function, create environment variable OPENTELEMETRY_COLLECTOR_CONFIG_FILE on your Lambda function and set it to /var/task/<path>/<to>/<filename>. This tells the extension where to find the collector configuration.

Here is a sample configuration file, collector.yaml, in the root directory:

1. Copy collector.yaml in the root directory
2. Set an environment variable OPENTELEMETRY_COLLECTOR_CONFIG_FILE to /var/task/<path>/<to>/<file>

yamlcopydownload
receivers:
  otlp:
    protocols:
      grpc:
exporters:
  otlphttp:
    endpoint: "https://<YOUR-TENANT-ID>.live.dynatrace.com/api/v2/otlp"
    headers: {"Authorization": "Api-Token <YOUR-DYNATRACE-API-TOKEN>"}
service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [otlphttp]

For further details on configuration, see Send OpenTelemetry trace data to Dynatrace.

You can set this via the Lambda console or the AWS CLI. With the CLI, use the following command:

shellcopydownload
aws lambda update-function-configuration --function-name Function --environment Variables={OPENTELEMETRY_COLLECTOR_CONFIG_FILE=/var/task/collector.yaml}

You can also configure environment variables via CloudFormation template:

yamlcopydownload
Function:
  Type: AWS::Serverless::Function
  Properties:
    ...
    Environment:
      Variables:
        OPENTELEMETRY_COLLECTOR_CONFIG_FILE: /var/task/collector.yaml

To ingest HTTP via the Dynatrace Trace API, you need to configure the exporter. The exporter will then directly send traces to the configured endpoint.

If you use environment variables for setup, you need to set the following values:

• For OTEL_EXPORTER_OTLP_PROTOCOL: http/protobuf

• For OTEL_EXPORTER_OTLP_ENDPOINT: the URL for export endpoint

  Notes:

  • If you set the endpoint URL via environment variables, the export endpoints for traces and metrics are automatically appended by v1/traces for traces and v1/metrics for metrics. For example, if the endpoint is set to https://<YOUR-TENANT-ID>.live.dynatrace.com/api/v2/otlp, traces will be exported to https://<YOUR-TENANT-ID>.live.dynatrace.com/api/v2/otlp/v1/traces.
  • If you set the endpoint explicitly in code, it will be used as is.

  For details, see Endpoint URLs for OTLP/HTTP.

• For OTEL_EXPORTER_OTLP_HEADERS: the authorization API token value in the following format: Authorization=Api-Token <TOKEN>.

The AWS Distro for OpenTelemetry doesn't provide a wrapper layer for .NET as it does for other languages. You need to add a tracer to your code and create a "root span."

The following sample uses an AWS Lambda Proxy Integration and gRPC transport.

Note: If you don't set the Protocol property of the OtlpExporterOptions class via environment variables or in code, it will be initialized as OtlpExportProtocol.Grpc by default.

c#copydownload
public class Functions
{
  public Functions() {}

  //Defines the OpenTelemetry resource attribute "service.name" which is mandatory
  private const string servicename = "AWS Lambda";

  //Defines the OpenTelemetry Instrumentation Scope.
  private const string activitySource = "MyCompany.MyProduct.MyLibrary";

  //Provides the API for starting/stopping activities.
  private static readonly ActivitySource MyActivitySource = new ActivitySource(activitySource);

  public async Task<APIGatewayProxyResponse> Get(APIGatewayProxyRequest request, ILambdaContext context)
  {
      AppContext.SetSwitch("System.Net.Http.SocketsHttpHandler.Http2UnencryptedSupport",true);

      //Initialize OpenTelemetry Tracer
      using (Sdk.CreateTracerProviderBuilder()
          .SetSampler(new AlwaysOnSampler())
          .AddSource(activitySource)
          .SetResourceBuilder(ResourceBuilder.CreateDefault().AddService(servicename))
          .AddAWSInstrumentation() //Add auto-instrumentation for AWS SDK
          .AddHttpClientInstrumentation() //Add auto-instrumentation for AWS SDK
          .AddOtlpExporter(otlpOptions =>
          {
              //Use a local endpoint for AWS Lambda ADOT Collector Layer
              //or an endpoint configured via environment variable.
              var collectorUrl = Environment.GetEnvironmentVariable("COLLECTOR_URL") ?? "http://localhost:55680";
              otlpOptions.Endpoint = new Uri(collectorUrl);
          })
          .Build())
      {
          //create root-span, connecting with trace-parent read from the http-header
          using (var activity = MyActivitySource.StartActivity("Invoke", ActivityKind.Server, request.Headers["traceparent"]))
          {

            //.....
            //... YOUR CODE GOES HERE
            //....
          }
      }
  }
}

The code sample using HTTP exporter is similar to the gRPC exporter sample; the only difference is in the configuration of OtlpExporterOptions:

c#copydownload
//Initialize OpenTelemetry Tracer
using (Sdk.CreateTracerProviderBuilder()
     // ... other initialization code (see code snippet for the gRPC case)
    .AddOtlpExporter(otlpOptions =>
    {
        otlpOptions.Protocol = OtlpExportProtocol.HttpProtobuf;
        otlpOptions.Headers = "Authorization=Api-Token <TOKEN>";

        //Use an explicitly set endpoint for export
        //or an endpoint configured via environment variable.
        otlpOptions.Endpoint = new Uri("https://<YOUR-TENANT-ID>.live.dynatrace.com/api/v2/otlp");
    })
    .Build())
{
    // ... span creation code and your code goes here (see code snippet for the gRPC case)
}

If configuration is done via environment variables, the code for adding an OTLP/HTTP exporter looks even simpler:

c#copydownload
//Initialize OpenTelemetry Tracer
using (Sdk.CreateTracerProviderBuilder()
     // ... other initialization code (see code snippet for the gRPC case)
    .AddOtlpExporter()
    .Build())
{
    // ... span creation code and your code goes here (see code snippet for the gRPC case)
}