AWS Lambda logs in context of traces

This page describes how to include TraceId and SpanId information in user-generated log messages. This way, you can associate log messages with the surrounding trace.

Prerequisites

Set up AWS Lambda log collection or AWS log forwarding.
Enable OpenTelemetry interoperability for the Lambda function.
The log output generated by the AWS Lambda function must be formatted so that the TraceId and SpanId information can be picked up. The expected format of the enriched fields in an unstructured log is as follows:
- Encapsulated in square brackets [] with a !dt prefix
- dt.trace_id and dt.span_id must be hex-encoded strings Example: [!dt dt.trace_id=0af7651916cd43dd8448eb211c80319c,dt.span_id=00f067aa0ba902b7]

Check AWS Lambda log collection to learn more about limitations since Lambda log enrichment is not automatic.

OpenTelemetry Python

In the example below, a dt_log function has been created to enrich a given log message with trace_id and span_id information. Printing this enriched message to the configured log sink associates the log message with the currently active span in the Dynatrace web UI.

import logging
from opentelemetry import trace


def dt_log(self, record):
    if (not self.disabled) and self.filter(record):
        ctx = trace.get_current_span().get_span_context()
        if ctx.is_valid:
            trace_id = "{0:032X}".format(ctx.trace_id)
            span_id = "{0:016X}".format(ctx.span_id)
            record.msg = f"[!dt dt.trace_id={trace_id},dt.span_id={span_id}] - {record.msg}"
    self.callHandlers(record)

logging.Logger.handle = dt_log


def lambda_handler(event, context):
    logger = logging.getLogger()
    logger.warning("Hello world")

    return {
        "statusCode": 200,
        "body": "Hello from lambda"
    }

OpenTelemetry JavaScript (Node.js)

In the example below, a dt_log function has been created to enrich a given log message with trace_id and span_id information. Printing this enriched message to stdout associates the log message with the currently active span in the Dynatrace web UI.

const opentelemetry = require('@opentelemetry/api');

function dtLog(msg) {
  const spanContext = opentelemetry.trace.getSpanContext(opentelemetry.context.active()) ?? opentelemetry.INVALID_SPAN_CONTEXT;
  console.log(`[!dt dt.trace_id=${spanContext.traceId},dt.span_id=${spanContext.spanId}] - ${msg}`);
}

exports.handler = function(event, context) {
  const msg = "Hello World"

  dtLog(msg);

  context.succeed({
    statusCode: 200,
    body: msg
  });
};

Logs created by some commonly used Node.js logging frameworks are automatically associated with traces by using the corresponding OpenTelemetry instrumentations.

Node.js logging framework	OpenTelemetry instrumentation
winston	@opentelemetry/instrumentation-winston
pino	@opentelemetry/instrumentation-pino
bunyan	@opentelemetry/instrumentation-bunyan

In the example below, the winston instrumentation is used to enrich a winston info log with TraceId and SpanId. The created log is associated with the currently active span in Dynatrace.

const otelApi = require('@opentelemetry/api');
const { registerInstrumentations } = require('@opentelemetry/instrumentation');
const { WinstonInstrumentation } = require('@opentelemetry/instrumentation-winston');

registerInstrumentations({
  instrumentations: [new WinstonInstrumentation()],
});

const winston = require('winston');

exports.handler = function (event, context, callback) {
  const logger = winston.createLogger({
    transports: [new winston.transports.Console()],
  });

  logger.info('winston info log');

  context.succeed({
    statusCode: 200,
    body: 'Hello from AWS Lambda Node.js',
  });
}

OpenTelemetry Java

In the example below, a dtLog method has been created to enrich a given log message with TraceId and SpanId information. Printing this enriched message via System.out associates the log message with the currently active span in the Dynatrace web UI.

package com.amazonaws.lambda.demo;

import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;

import io.opentelemetry.api.trace.Span;
import io.opentelemetry.api.trace.SpanContext;

public class HelloJava implements RequestHandler<Object, String> {

  private static void dtLog(final String msg) {
    SpanContext spanContext = Span.current().getSpanContext();
    System.out.printf(
      "[!dt dt.trace_id=%s,dt.span_id=%s] - %s%n",
      spanContext.getTraceId(),
      spanContext.getSpanId(),
      msg
    );
  }

  @Override
  public String handleRequest(Object input, Context context) {
    String msg = "Hello World";

    dtLog(msg);

    return msg;
  }
}

AWS Lambda logs in context of traces

Prerequisites

OpenTelemetry Python

OpenTelemetry JavaScript (Node.js)

OpenTelemetry Java

Related topics