OpenTelemetry interoperability on AWS Lambda
OpenTelemetry is a collection of tools, APIs, and SDKs. You can use it to instrument, generate, collect, and export telemetry data (metrics, logs, and traces) for analysis to get insights into your software's performance and behavior.
Enable OpenTelemetry
OpenTelemetry interoperability is disabled by default in Dynatrace, but you can manually enable it. For instance, to enable OpenTelemetry when you configure OneAgent using the environment variables, add DT_OPEN_TELEMETRY_ENABLE_INTEGRATION=true
.
Enabling OpenTelemetry interoperability connects OneAgent to the OpenTelemetry API. This allows OneAgent on AWS Lambda to make use of the instrumentation packages and extensions available for their respective OpenTelemetry implementation, which enables monitoring of technologies like databases or messaging frameworks that aren't supported by OneAgent out of the box.
Note: If an OpenTelemetry SDK already exists, it will be evicted by OneAgent if this option is enabled.
OpenTelemetry Python
OneAgent version 1.219+Using an OpenTelemetry Python instrumentation
OpenTelemetry for Python provides a number of instrumentation packages in their OpenTelemetry Python contributions repository.
The code snippet below shows how to instrument PostgreSQL calls to your Python Lambda function by using the aiopg instrumentation package.
import json
import aiopg
from opentelemetry.instrumentation.aiopg import AiopgInstrumentor
AiopgInstrumentor().instrument()
def lambda_handler(event, context):
return {
'statusCode': 200,
'body': json.dumps(execute_query())
}
def execute_query():
result = []
with aiopg.connect(database='my_db') as conn:
with conn.cursor() as cur:
cur.execute("SELECT 'hello db';")
for row in cur:
result.append(row)
return result
To instrument boto3, the AWS SDK for Python, OpenTelemetry provides the botocore instrumentation package. The code snippet below shows how this instrumentation can be used to add observability for calls to a DynamoDB database.
import boto3
import json
from opentelemetry.instrumentation.botocore import BotocoreInstrumentor
BotocoreInstrumentor().instrument()
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('MyTable')
def lambda_handler(event, handler):
result = table.get_item(Key={'mykey': 42})
return {
"statusCode": 200,
"answer": json.dumps(result.get("Item"))
}
Starting with Dynatrace version 1.244+, Dynatrace provides support for DynamoDB database service.
Example DynamoDB service page after running the above code snippet:
Using the OpenTelemetry Python API
The code snippet below shows how OpenTelemetry Python can be used in an SDK-like approach to trace additional operations that aren't covered by an instrumentation package.
import json
from opentelemetry import trace
def lambda_handler(event, context):
tracer = trace.get_tracer(__name__)
with tracer.start_as_current_span("do work"):
# do work
with tracer.start_as_current_span("do some more work") as span:
span.set_attribute("foo", "bar")
# do some more work
return {
'statusCode': 200,
'body': json.dumps('Hello from Hello world from OpenTelemetry Python!')
}
These spans are displayed on the Code level tab.
OpenTelemetry JavaScript (Node.js)
OneAgent version 1.229+Using an OpenTelemetry Node.js instrumentation
OpenTelemetry for JavaScript provides a number of instrumentation packages in their OpenTelemetry JavaScript contributions repository.
The code snippet below shows how to instrument PostgreSQL calls to your Node.js Lambda function by using the opentelemetry-instrumentation-pg instrumentation package.
const { registerInstrumentations } = require('@opentelemetry/instrumentation');
const { PgInstrumentation } = require('@opentelemetry/instrumentation-pg');
// You must create the PgInstrumentation (and other instrumentations)
// before loading any corresponding modules, such as `require('pg')`.
registerInstrumentations({
instrumentations: [
new PgInstrumentation(),
],
});
const { Client } = require('pg');
exports.handler = async function myHandler(event, context) {
let client;
try {
client = new Client(/* DB connection information */);
await client.connect();
const result = await client.query('SELECT * FROM users;');
return result.rows;
} finally {
client?.end();
}
}
To instrument the AWS SDK for JavaScript, OpenTelemetry provides the opentelemetry/instrumentation-aws-sdk
instrumentation package.
The code snippet below shows how this instrumentation can be used to add observability for calls to a DynamoDB database.
const AWS = require('aws-sdk');
const opentelemetry = require('@opentelemetry/api');
const { NodeTracerProvider } = require('@opentelemetry/sdk-trace-node');
const { registerInstrumentations } = require('@opentelemetry/instrumentation');
const { AwsInstrumentation } = require('@opentelemetry/instrumentation-aws-sdk');
const traceProvider = new NodeTracerProvider({
// Make sure to disable the old plugin.
plugins: {
'aws-sdk': { enabled: false, path: 'opentelemetry-plugin-aws-sdk' }
}
});
registerInstrumentations({
traceProvider,
instrumentations: [
new AwsInstrumentation()
]
});
const lambda = new AWS.Lambda();
const tracer = opentelemetry.trace.getTracer('my-package-name');
exports.handler = function(event, context) {
const ddb = new AWS.DynamoDB();
const dbParamsGetDelete = {
TableName: 'E2E_test_table',
Key: {
'svnr': { N: '1234'}
}
};
ddb.getItem(dbParamsGetDelete, function(err, data) {
if (err) {
console.error('Error', err);
} else {
console.log('Success', data.Item);
}
});
};
Starting with Dynatrace version 1.244+, Dynatrace provides support for DynamoDB database service.
Example DynamoDB service page after running the above code snippet:
Using the OpenTelemetry Node.js API
The code snippet below shows how OpenTelemetry JavaScript can be used in an SDK-like approach to trace additional operations that aren't covered by an instrumentation package.
const opentelemetry = require('@opentelemetry/api');
const tracer = opentelemetry.trace.getTracer('my-package-name');
exports.handler = function(event, context) {
// create a span using the OTel API
const span = tracer.startSpan('do some work');
span.setAttribute('foo', 'bar');
span.end();
// ...
const response = {
statusCode: 200,
body: JSON.stringify('Hello from Node.js'),
};
return response;
};
OpenTelemetry Java
OneAgent version 1.225+Using the OpenTelemetry Java API
The code snippet below shows how OpenTelemetry Java can be used in an SDK-like approach to trace additional operations that aren't covered by OneAgent out of the box.
@Override
public String handleRequest(Object input, Context context) {
Tracer tracer = GlobalOpenTelemetry.getTracer("instrumentation-library-name", "1.0.0");
Span span = tracer.spanBuilder("do some work").startSpan();
try {
span.setAttribute("foo", "bar");
// ....
return "Hello from OpenTelemetry Java!";
} finally {
span.end();
}
}
Note: OneAgent captures only spans created via tracers from GlobalOpenTelemetry
and may not work if you try to manually (re)configure GlobalOpenTelemetry
.