Trace .NET Lambda functions
Dynatrace uses OpenTelemetry to monitor AWS Lambda invocations.
Prerequisites
Ensure that you have followed the initial configuration steps described in Configuration for monitoring AWS Lambda with OpenTelemetry before using the packages below.
The following NuGet packages can be used to cover different aspects of AWS Lambda tracing:
- recommended OpenTelemetry.Instrumentation.AWSLambda version 1.1.0-beta.2+: Contains methods to trace incoming AWS Lambda invocations.
- recommended OpenTelemetry.Contrib.Instrumentation.AWS version 1.0.1+: Traces outgoing AWS SDK calls to other AWS Lambda invocations and calls to AWS services like DynamoDB and SQS.
- required
Dynatrace.OpenTelemetry.Instrumentation.AwsLambda
: Enables linking traces from one Lambda to another over an AWS SDK Lambda invocation. This is the only way to link these kinds of requests; however, if you don't need this linking, or don't use the AWS Lambda client SDK to invoke or receive other Lambda invocations, you can leave out the package.
Installation
Any of the above-listed packages can be installed via the CLI. For example, Dynatrace.OpenTelemetry.Instrumentation.AwsLambda
can be installed using the following command:
dotnet add package Dynatrace.OpenTelemetry.Instrumentation.AwsLambda
Note: Some packages may require you to specify a version explicitly or use the --prerelease
command line flag, such as
dotnet add package --prerelease OpenTelemetry.Instrumentation.AWSLambda
.
Initialization
The initialization code for AWS Lambda tracing in your Function.cs
file could look as follows (where Function
is the configured Lambda handler class):
using System.Threading.Tasks;
using Amazon.Lambda.Core;
using Dynatrace.OpenTelemetry;
using Dynatrace.OpenTelemetry.Instrumentation.AwsLambda;
using OpenTelemetry;
using OpenTelemetry.Instrumentation.AWSLambda;
using OpenTelemetry.Trace;
[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))]
namespace Examples.AwsFunctionApp
{
public class Function
{
private static readonly TracerProvider TracerProvider;
static Function()
{
DynatraceSetup.InitializeLogging();
TracerProvider = Sdk.CreateTracerProviderBuilder()
.AddDynatrace()
// Configures AWS Lambda invocations tracing
.AddAWSLambdaConfigurations(c => c.DisableAwsXRayContextExtraction = true)
// Instrumentation for creation of span (Activity) representing AWS SDK call.
// Can be omitted if there are no outgoing AWS SDK calls to other AWS Lambdas and/or calls to AWS services like DynamoDB and SQS.
.AddAWSInstrumentation(c => c.SuppressDownstreamInstrumentation = true)
// Adds injection of Dynatrace-specific context information in certain SDK calls (e.g. Lambda Invoke).
// Can be omitted if there are no outgoing calls to other Lambdas via the AWS Lambda SDK.
.AddDynatraceAwsSdkInjection()
.Build();
}
}
}
Setting the option DisableAwsXRayContextExtraction
to true
is required to skip Amazon X-Ray parent extraction, which may conflict with the Dynatrace propagation.
Tracing incoming AWS Lambda calls
Example 1: Trace an AWS Lambda invoked via AWS SDK
In addition to the initialization part provided above, the handler method of a Lambda invoked via AWS SDK could look as follows:
using System.Threading.Tasks;
using Amazon.Lambda.Core;
using Dynatrace.OpenTelemetry;
using Dynatrace.OpenTelemetry.Instrumentation.AwsLambda;
using OpenTelemetry;
using OpenTelemetry.Instrumentation.AWSLambda;
using OpenTelemetry.Trace;
[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))]
namespace Examples.AwsFunctionApp
{
public class Function
{
private static readonly TracerProvider TracerProvider;
// Use initialization code from the "Initialization" section of the docs
public Task FunctionHandlerAsync(object input, ILambdaContext context)
{
var propagationContext = AwsLambdaHelpers.ExtractPropagationContext(context);
return AWSLambdaWrapper.TraceAsync(TracerProvider, FunctionHandlerInternalAsync, input, context, propagationContext.ActivityContext);
}
private Task FunctionHandlerInternalAsync(object input, ILambdaContext context)
{
// This is just an example of function handler and should be replaced by actual code.
return Task.CompletedTask;
}
}
}
- A parent context is extracted explicitly using the
AwsLambdaHelpers
class from theDynatrace.OpenTelemetry.Instrumentation.AwsLambda
package. - An activity tracing the incoming request and the handler is created by the
TraceAsync
method. TraceAsync
should be used when you trace an async function or a function returning a task. That way, the activity ends only when the task completes.
Example 2: Trace an AWS Lambda invoked via Amazon API Gateway (incoming HTTP request)
In addition to the initialization part provided above, the Lambda handler invoked via Amazon API Gateway could look as follows:
using Amazon.Lambda.APIGatewayEvents;
using Amazon.Lambda.Core;
using Dynatrace.OpenTelemetry;
using Dynatrace.OpenTelemetry.Instrumentation.AwsLambda;
using OpenTelemetry;
using OpenTelemetry.Instrumentation.AWSLambda;
using OpenTelemetry.Trace;
[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))]
namespace Examples.AwsFunctionApp
{
public class Function
{
private static readonly TracerProvider TracerProvider;
// Use initialization code from the "Initialization" section of the docs
public APIGatewayHttpApiV2ProxyResponse FunctionHandler(APIGatewayHttpApiV2ProxyRequest request, ILambdaContext context)
{
return AWSLambdaWrapper.Trace(TracerProvider, FunctionHandlerInternal, request, context);
}
private APIGatewayHttpApiV2ProxyResponse FunctionHandlerInternal(APIGatewayHttpApiV2ProxyRequest request, ILambdaContext context)
{
// This is just an example of function handler and should be replaced by actual code.
return new APIGatewayHttpApiV2ProxyResponse
{
StatusCode = 200,
Body = "Example function result",
};
}
}
}
- A parent context is extracted from the incoming request in the
Trace
(orTraceAsync
) method. - An activity tracing the incoming request and the handler is created by the
Trace
method. - In general, the
Trace
/TraceAsync
methods support any trigger, but extended support is available for theAPIGatewayProxyRequest
andAPIGatewayHttpApiV2ProxyRequest
trigger types. For more details about request/response types, consult the GitHub documentation. Trace
should only be used when you have a function returning something other than aTask
. For the asynchronous handler,TraceAsync
should be used instead.
Example 3: Tracing without the AwsLambda
package
If you prefer not to use the OpenTelemetry.Instrumentation.AWSLambda
package, you can manually create an activity for Lambda. Note that this involves quite a bit of work, as Dynatrace requires certain activity tags (span attributes) to detect the service (conforming to the OpenTelemetry FaaS trace conventions
and resource conventions). You also need to manually extract the parent context.
For this example, only the Dynatrace.OpenTelemetry.Instrumentation.AwsLambda
package is required.
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Reflection;
using Amazon.Lambda.APIGatewayEvents;
using Amazon.Lambda.Core;
using Dynatrace.OpenTelemetry;
using Dynatrace.OpenTelemetry.Instrumentation.AwsLambda;
using OpenTelemetry;
using OpenTelemetry.Context.Propagation;
using OpenTelemetry.Trace;
[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))]
namespace Examples.AwsFunctionApp
{
public class Function
{
private static readonly TracerProvider TracerProvider;
private static readonly ActivitySource ActivitySource;
static Function()
{
DynatraceSetup.InitializeLogging();
var activitySourceName = Assembly.GetExecutingAssembly().GetName().Name;
ActivitySource = new ActivitySource(activitySourceName);
TracerProvider = Sdk.CreateTracerProviderBuilder()
.AddSource(activitySourceName)
.AddDynatrace()
.Build();
}
public static IEnumerable<KeyValuePair<string, object>> GetFunctionTags(ILambdaContext context, string trigger)
{
return new KeyValuePair<string, object>[]
{
new("faas.name", context.FunctionName),
new("faas.id", context.InvokedFunctionArn),
new("faas.trigger", trigger),
new("cloud.platform", "aws_lambda"),
new("cloud.provider", "aws"),
new("cloud.region", Environment.GetEnvironmentVariable("AWS_REGION")),
};
}
public APIGatewayProxyResponse FunctionHandler(APIGatewayHttpApiV2ProxyRequest apiGatewayProxyEvent, ILambdaContext context)
{
try
{
var parentContext = ExtractParentContext(apiGatewayProxyEvent, context);
using var activity = ActivitySource.StartActivity(ActivityKind.Server, parentContext, GetFunctionTags(context, "http"));
return new APIGatewayProxyResponse
{
StatusCode = 200,
Body = "Example function result",
};
}
catch (Exception ex)
{
context.Logger.LogLine($"Exception occurred while handling request: {ex.Message}");
throw;
}
finally
{
TracerProvider?.ForceFlush();
}
}
private static ActivityContext ExtractParentContext(APIGatewayHttpApiV2ProxyRequest apiGatewayProxyEvent, ILambdaContext context)
{
var propagationContext = AwsLambdaHelpers.ExtractPropagationContext(context);
if (propagationContext == default)
{
propagationContext = Propagators.DefaultTextMapPropagator.Extract(default, apiGatewayProxyEvent, HeaderValuesGetter);
}
return propagationContext.ActivityContext;
}
private static IEnumerable<string> HeaderValuesGetter(APIGatewayHttpApiV2ProxyRequest apiGatewayProxyEvent, string name) =>
(apiGatewayProxyEvent.Headers != null && apiGatewayProxyEvent.Headers.TryGetValue(name.ToLowerInvariant(), out var value)) ? new[] { value } : null;
}
}
Amazon DynamoDB operations tracing
To trace DynamoDB operations, add the OpenTelemetry.Contrib.Instrumentation.AWS
dependency—the AWS client instrumentation for OpenTelemetry .NET—to your project:
dotnet add package OpenTelemetry.Contrib.Instrumentation.AWS
After you add the dependency, update the initialization code by adding a call to the AddAWSInstrumentation()
extension method:
using Dynatrace.OpenTelemetry;
using OpenTelemetry;
using OpenTelemetry.Trace;
using System.Reflection;
namespace Examples.AwsFunctionApp
{
public class Function
{
private static readonly TracerProvider TracerProvider;
static Function()
{
DynatraceSetup.InitializeLogging();
var activitySourceName = Assembly.GetExecutingAssembly().GetName().Name;
TracerProvider = Sdk.CreateTracerProviderBuilder()
.AddSource(activitySourceName)
// Instrumentation used for tracing calls to AWS services via AWS SDK (including DynamoDB)
.AddAWSInstrumentation(c => c.SuppressDownstreamInstrumentation = true)
.Build();
}
}
}
Note: If you prefer not to see HTTP child nodes under AWS SDK calls, set the SuppressDownstreamInstrumentation
option to true
.
In the following example, the DynamoDB operations DescribeTable
, UpdateItem
, and GetItem
are represented as separate span child nodes with a common parent outbound-aws-dotnet in eu-central-1
(Lambda function performing DynamoDB operations):