Integrate OpenTelemetry .NET on Azure Functions consumption plan
Installation
Add the following dependencies to your project:
- required
Dynatrace.OpenTelemetry
- Provides integration of Dynatrace-specific components (for activity export and propagation) into OpenTelemetry .NET. The minimum (referenced) OpenTelemetry version is currently 1.1.0. - optional
Dynatrace.OpenTelemetry.Instrumentation.AzureFunctions.Core
- Provides utilities for creating activities with the expected properties for Dynatrace. - optional
OpenTelemetry.Extensions.Hosting
- Uses aTracerProvider
with a dependency injection. Currently only available as a pre-release (release candidate).
Example commands to add dependencies:
dotnet add package Dynatrace.OpenTelemetry
dotnet add package Dynatrace.OpenTelemetry.Instrumentation.AzureFunctions.Core
dotnet add package --prerelease OpenTelemetry.Extensions.Hosting
Example using the dotnet (in-process) runtime
Your Startup.cs
could look as follows:
using Dynatrace.OpenTelemetry;
using Dynatrace.OpenTelemetry.Instrumentation.AzureFunctions;
using Microsoft.Azure.Functions.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection;
using OpenTelemetry.Trace;
[assembly: FunctionsStartup(typeof(Examples.AzureFunctionApp.Startup))]
namespace Examples.AzureFunctionApp
{
internal class Startup : FunctionsStartup
{
public override void Configure(IFunctionsHostBuilder builder)
{
builder.Services.AddOpenTelemetryTracing(sdk => sdk
.AddAzureFunctionsInstrumentation()
// ... any custom OTel setup ...
.AddDynatrace()
// ... if you need custom resources, set them after AddDynatrace (see below)
);
}
}
}
An instrumented in-process function could look like this:
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Threading.Tasks;
using Dynatrace.OpenTelemetry;
using Dynatrace.OpenTelemetry.Instrumentation.AzureFunctions;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.Extensions.Logging;
using OpenTelemetry.Context.Propagation;
using OpenTelemetry.Trace;
namespace Examples.AzureFunctionApp
{
public class Function
{
private readonly TracerProvider tracerProvider;
public Function(
TracerProvider tracerProvider,
ILoggerFactory loggerFactory)
{
this.tracerProvider = tracerProvider ?? throw new ArgumentNullException(nameof(tracerProvider));
// This is needed in every function in your app.
DynatraceSetup.InitializeLogging(loggerFactory);
}
[FunctionName("MyFunction")]
public Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Function, "get", Route = null)] HttpRequest request,
ExecutionContext ctx)
{
var parent = ExtractParentContext(request);
return AzureFunctionsCoreInstrumentation.TraceAsync(
this.tracerProvider, ctx.FunctionName, () => RunInternal(request), parent);
}
private async Task<IActionResult> RunInternal(HttpRequest request)
{
// Your actual handler code
}
private static ActivityContext ExtractParentContext(HttpRequest request)
{
var context = Propagators.DefaultTextMapPropagator.Extract(default, request, HeaderValuesGetter);
return context.ActivityContext;
}
private static IEnumerable<string> HeaderValuesGetter(HttpRequest request, string name) =>
request.Headers.TryGetValue(name, out var values) ? values : (IEnumerable<string>)null;
}
}
Additionally, you need to modify host.json
to allow logging for Dynatrace.OpenTelemetry
.
Note that this does not enable logging unless explicitly configured. See InitializeLogging.
{
"version": "2.0",
"logging": {
// ...
"logLevel": {
"Dynatrace.OpenTelemetry": "Debug"
}
}
}
Example using the dotnet-isolated runtime
Your Program.cs
could look as follows:
using Dynatrace.OpenTelemetry;
using Dynatrace.OpenTelemetry.Instrumentation.AzureFunctions;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
DynatraceSetup.InitializeLogging();
var host = new HostBuilder()
.ConfigureFunctionsWorkerDefaults()
.ConfigureServices(services => services
.AddOpenTelemetryTracing(tracing => tracing
.AddDynatrace()
.AddAzureFunctionsInstrumentation()))
.Build();
host.Run();
An instrumented worker function could look like this:
using System.Diagnostics;
using Dynatrace.OpenTelemetry.Instrumentation.AzureFunctions;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Http;
using OpenTelemetry.Context.Propagation;
using OpenTelemetry.Trace;
namespace Examples.AzureFunctionApp
{
public class Function
{
[Function("MyFunction")]
public Task<HttpResponseData> Run(
[HttpTrigger(AuthorizationLevel.Function, "get")] HttpRequestData request,
TracerProvider tracerProvider, FunctionContext context)
{
ActivityContext parent = ExtractParentContext(request, context);
return AzureFunctionsCoreInstrumentation.TraceAsync(
tracerProvider, context.FunctionDefinition.Name, () => RunInternal(request), parent);
}
public async Task<HttpResponseData> RunInternal(HttpRequestData request)
{
// Your actual handler code
}
private static ActivityContext ExtractParentContext(HttpRequestData request, FunctionContext context) {
ActivityContext parent = default;
PropagationContext ctx = Propagators.DefaultTextMapPropagator.Extract(
default,
request.Headers,
(c, k) => c.TryGetValues(k, out var value) ? value : null);
parent = ctx.ActivityContext;
if (parent == default)
{
PropagationContext ctx2 = Propagators.DefaultTextMapPropagator.Extract(
default,
context.TraceContext,
(c, k) =>
{
string? result =
k.Equals("traceparent", StringComparison.OrdinalIgnoreCase) ? c.TraceParent :
k.Equals("tracestate", StringComparison.OrdinalIgnoreCase) ? c.TraceState :
null;
return result == null ? null : new[] { result };
});
parent = ctx2.ActivityContext;
}
return parent;
}
}
}
Technical details
InitializeLogging
-
Calling
InitializeLogging
is required even if you don't plan to enable logging, and the actual log messages won't be logged even after calling this method, unless configured. -
If you use the
dotnet-isolated
runtime (out-of-process, worker functions), you need to callInitializeLogging
in yourMain
method before callingAddDynatrace
. You can passnull
asloggerFactory
, so that, if enabled, logging can useConsole.Out
/Console.Error
. This is automatically forwarded to AppInsights for thedotnet-isolated
runtime. -
If you have specific requirements, you can also pass any custom
LoggingFactory
. -
For the
dotnet
runtime (in-process, class-library), sending logs to AppInsights requires using anILogger
orILoggerFactory
injected into the function with a dependency injection. Thus, you shouldn't usenull
as an argument for theloggerFactory
parameter, but callInitializeLogging
the first time any function in your Function App is invoked. To get theILoggerFactory
, simply add a parameter of that type. -
If you use the
ILoggerFactory
provided by Azure Functions, you also need to modifyhost.json
to enable logging there. We recommend that you always use thedebug
log-level inhost.json
, as the actual log messages handed to the ILogger are separately configured in the Dynatrace configuration.
{
"version": "2.0",
"logging": {
// ...
"logLevel": {
"Dynatrace.OpenTelemetry": "Debug"
}
}
}
AddDynatrace
-
AddDynatrace
is an extension method to OpenTelemetry'sTracerProvider
. It requiresusing Dynatrace.OpenTelemetry
. Currently, there aren't any additional parameters for this function, as configuration is read from environment variables and adtconfig.json
file. For details, see Integrate OpenTelemetry on Azure Functions consumption plan. -
AddDynatrace
mainly adds anActivityProcessor
to theTracerProvider
that will send all activities to Dynatrace. This extension:- Sets the resources required by Dynatrace. Due to an issue with the OpenTelemetry .NET SDK, this will override any existing resources. If you need custom resources, you need to call
SetResourceBuilder
on theTracerProvider
afterAddDynatrace
. Be aware that this will override the resources configured byAddDynatrace
and you need to readd them as part of the sameSetResourceBuilder
call. You can do this by calling the OpenTelemetry SDK'sAddTelemetrySdk
extension method on theResourceBuilder
. - Exchanges the global
Propagators.DefaultTextMapPropagator
with a custom one that is based on the default W3C-format, but does additional processing oftracestate
and additional Dynatrace-specific HTTP headers. The baggage propagator is also enabled, as is default for OpenTelemetry .NET. There's currently no way to disable it. Using another propagator isn't supported and will lead to missing links in the distributed traces.
- Sets the resources required by Dynatrace. Due to an issue with the OpenTelemetry .NET SDK, this will override any existing resources. If you need custom resources, you need to call
The following minimal snippet might be used to initialize a TracerProvider
with AddDynatrace
:
using Dynatrace.OpenTelemetry;
using Dynatrace.OpenTelemetry.Instrumentation.AzureFunctions;
using OpenTelemetry.Trace;
// ...
// (call DynatraceSetup.InitializeLogging before or after AddDynatrace depending on runtime)
// ...
TracerProvider tracerProvider = Sdk.CreateTracerProviderBuilder().AddDynatrace().Build();
AzureFunctionsCoreInstrumentation.Trace/TraceAsync
This function creates and starts a System.Diagnostics.Activity
and runs the handler function argument, then stops Activity
and records any exception on it.
The parent ActivityContext
must be extracted from the HTTP headers using the Propagators.DefaultTextMapPropagator
, which AddDynatrace
initializes. If you don't pass any parent, a root span will be created (Activity.Current
won't be used).
Instrumenting HttpClient
calls (outgoing HTTP requests)
A very common need is to trace outgoing HTTP requests. This can be achieved by using the OpenTelemetry.Instrumentation.Http
NuGet package (currently only available as a pre-release).
The instrumentation then has to be added to your TracerProvider
setup by calling AddHttpClientInstrumentation
, for example, in Program.cs
:
// ...
using OpenTelemetry.Trace;
// ...
var host = new HostBuilder()
// ...
.ConfigureServices(services => services
.AddOpenTelemetryTracing(tracing => tracing
// ...
.AddHttpClientInstrumentation(op =>
{
// Exclude outgoing calls with no parent activity.
op.Filter = req => Activity.Current?.Parent != null;
})))
.Build();
// ...
Using a request filter as in the example above is highly recommended, as otherwise, depending on your Function Apps configuration, you might observe a large number of periodic requests to https://rt.services.visualstudio.com/QuickPulseService.svc/ping
or similar URLs.
Note: Because of an Azure Functions runtime issue, the HTTP instrumentation won't work on Azure Functions in-process version 3.
The underlying issue can also affect other instrumentations. Therefore, we do not recommend using in-process version 3 functions.