OpenTelemetry interoperability on AWS Lambda

OpenTelemetry is a collection of tools, APIs, and SDKs. You can use it to instrument, generate, collect, and export telemetry data (metrics, logs, and traces) for analysis to get insights into your software's performance and behavior.

Enable OpenTelemetry

OpenTelemetry interoperability is disabled by default in Dynatrace, but you can manually enable it. For instance, to enable OpenTelemetry when you configure OneAgent using the environment variables, add DT_OPEN_TELEMETRY_ENABLE_INTEGRATION=true.

Enabling OpenTelemetry interoperability connects OneAgent to the OpenTelemetry API. This allows OneAgent on AWS Lambda to make use of the instrumentation packages and extensions available for their respective OpenTelemetry implementation, which enables monitoring of technologies like databases or messaging frameworks that aren't supported by OneAgent out of the box.

Note: If an OpenTelemetry SDK already exists, it will be evicted by OneAgent if this option is enabled.

OpenTelemetry Python

Using an OpenTelemetry Python instrumentation

OpenTelemetry for Python provides a number of instrumentation packages in their OpenTelemetry Python contributions repository.

The code snippet below shows how to instrument PostgreSQL calls to your Python Lambda function by using the aiopg instrumentation package.

pythoncopydownload
import json
import aiopg

from opentelemetry.instrumentation.aiopg import AiopgInstrumentor

AiopgInstrumentor().instrument()


def lambda_handler(event, context):
    return {
        'statusCode': 200,
        'body': json.dumps(execute_query())
    }

def execute_query():
    result = []
    with aiopg.connect(database='my_db') as conn:
        with conn.cursor() as cur:
            cur.execute("SELECT 'hello db';")
            for row in cur:
                result.append(row)

    return result

To instrument boto3, the AWS SDK for Python, OpenTelemetry provides the botocore instrumentation package. The code snippet below shows how this instrumentation can be used to add observability for calls to a DynamoDB database.

pythoncopydownload
import boto3
import json

from opentelemetry.instrumentation.botocore import BotocoreInstrumentor

BotocoreInstrumentor().instrument()

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('MyTable')


def lambda_handler(event, handler):
    result = table.get_item(Key={'mykey': 42})

    return {
        "statusCode": 200,
        "answer": json.dumps(result.get("Item"))
    }

Starting with Dynatrace version 1.244+, Dynatrace provides support for DynamoDB database service.

Example DynamoDB service page after running the above code snippet:

Using the OpenTelemetry Python API

The code snippet below shows how OpenTelemetry Python can be used in an SDK-like approach to trace additional operations that aren't covered by an instrumentation package.

pythoncopydownload
import json
from opentelemetry import trace

def lambda_handler(event, context):
    tracer = trace.get_tracer(__name__)

    with tracer.start_as_current_span("do work"):
        # do work
        with tracer.start_as_current_span("do some more work") as span:
            span.set_attribute("foo", "bar")
            # do some more work

    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Hello world from OpenTelemetry Python!')
    }

These spans are displayed on the Code level tab.

OpenTelemetry JavaScript (Node.js)

Using an OpenTelemetry Node.js instrumentation

OpenTelemetry for JavaScript provides a number of instrumentation packages in their OpenTelemetry JavaScript contributions repository.

The code snippet below shows how to instrument PostgreSQL calls to your Node.js Lambda function by using the opentelemetry-instrumentation-pg instrumentation package.

javascriptcopydownload
const { registerInstrumentations } = require('@opentelemetry/instrumentation');
const { PgInstrumentation } = require('@opentelemetry/instrumentation-pg');

// You must create the PgInstrumentation (and other instrumentations)
// before loading any corresponding modules, such as `require('pg')`.
registerInstrumentations({
  instrumentations: [
    new PgInstrumentation(),
  ],
});

const { Client } = require('pg');

exports.handler = async function myHandler(event, context) {
  let client;
  try {
    client = new Client(/* DB connection information */);
    await client.connect();
    const result = await client.query('SELECT * FROM users;');
    return result.rows;
  } finally {
    client?.end();
  }
}

To instrument the AWS SDK for JavaScript, OpenTelemetry provides the opentelemetry/instrumentation-aws-sdk instrumentation package.

The code snippet below shows how this instrumentation can be used to add observability for calls to a DynamoDB database.

javascriptcopydownload
const AWS = require('aws-sdk');
const opentelemetry = require('@opentelemetry/api');
const { NodeTracerProvider } = require('@opentelemetry/sdk-trace-node');
const { registerInstrumentations } = require('@opentelemetry/instrumentation');
const { AwsInstrumentation } = require('@opentelemetry/instrumentation-aws-sdk');

const traceProvider = new NodeTracerProvider({
  // Make sure to disable the old plugin.
  plugins: {
    'aws-sdk': { enabled: false, path: 'opentelemetry-plugin-aws-sdk' }
  }
});

registerInstrumentations({
  traceProvider,
  instrumentations: [
    new AwsInstrumentation()
  ]
});

const lambda = new AWS.Lambda();
const tracer = opentelemetry.trace.getTracer('my-package-name');

exports.handler = function(event, context) {
  const ddb = new AWS.DynamoDB();

  const dbParamsGetDelete = {
    TableName: 'E2E_test_table',
    Key: {
      'svnr': { N: '1234'}
    }
  };
  ddb.getItem(dbParamsGetDelete, function(err, data) {
    if (err) {
      console.error('Error', err);
    } else {
      console.log('Success', data.Item);
    }
  });
};

Starting with Dynatrace version 1.244+, Dynatrace provides support for DynamoDB database service.

Example DynamoDB service page after running the above code snippet:

Using the OpenTelemetry Node.js API

The code snippet below shows how OpenTelemetry JavaScript can be used in an SDK-like approach to trace additional operations that aren't covered by an instrumentation package.

javascriptcopydownload
const opentelemetry = require('@opentelemetry/api');

const tracer = opentelemetry.trace.getTracer('my-package-name');

exports.handler = function(event, context) {
  // create a span using the OTel API
  const span = tracer.startSpan('do some work');
  span.setAttribute('foo', 'bar');
  span.end();

  // ...

  const response = {
    statusCode: 200,
    body: JSON.stringify('Hello from Node.js'),
  };
  return response;
};

OpenTelemetry Java

Using the OpenTelemetry Java API

The code snippet below shows how OpenTelemetry Java can be used in an SDK-like approach to trace additional operations that aren't covered by OneAgent out of the box.

javacopydownload
@Override
    public String handleRequest(Object input, Context context) {
        Tracer tracer = GlobalOpenTelemetry.getTracer("instrumentation-library-name", "1.0.0");

        Span span = tracer.spanBuilder("do some work").startSpan();
        try {
            span.setAttribute("foo", "bar");
            // ....
            return "Hello from OpenTelemetry Java!";
        } finally {
            span.end();
        }
    }

Note: OneAgent captures only spans created via tracers from GlobalOpenTelemetry and may not work if you try to manually (re)configure GlobalOpenTelemetry.