AWS Lambda log collection
You can collect logs directly from your AWS Lambda functions and send them to Dynatrace for analysis. The solution is an alternative to the CloudWatch log forwarder with benefits in terms of cost and latency, and is also easier to set up, in particular if AWS Lambda tracing is already in place. As part of the OneAgent installation process, this feature provides a streamlined solution for collecting logs from your Lambda functions.
Prerequisites
- Dynatrace version 1.263
- Dynatrace Managed For layers with OneAgent version 1.263 and earlier, the Environment ActiveGate must have a trusted (not self-signed) certificate. For more details, see Custom SSL certificate for ActiveGate.
- An ActiveGate with the Log analytics collector module enabled. See Log Monitoring API - POST ingest logs.
Deploy
For Python, Node.js, and Java Lambda functions, Dynatrace provides a single Lambda layer that enables both trace and log collection. For .NET, Dynatrace provides a standalone layer that collects only logs.
If you are currently using it, you must disable CloudWatch log forwarding for functions on which you wish to use this log collection feature in order to avoid duplicate log exports. See CloudWatch Logs.
Usage
After deployment, collected logs for each future function invocation and initialization can be found in the Related logs card on the Lambda function's service page in Dynatrace, and in the Log viewer. You can inspect the log details to find the type of the log under the telemetryevent.type
attribute, among other metadata. Note that the content of platform
logs will be JSON data, while the content of function
logs will be plain text.
Configuration
Log event types
You can configure which log event types are collected. By default, two types of logs are collected: platform events and function logs. For more information about the log event types, see the AWS Lambda Telemetry API documentation.
To configure which types of log event types are collected, you have two options:
- Set the
DT_LOG_COLLECTION_EVENT_TYPES
environment variable. The value of this environment variable needs to be a colon-separated list of log event types. For example, to collect onlyfunction
logs, set the environment variable tofunction
. To collect bothplatform
andfunction
logs, set the environment variable toplatform:function
(this is the default value). - Set the
EventTypes
property of theLogCollection
block in thedtconfig.json
configuration file. The value of this property needs to be an array of strings. For example, to collect onlyfunction
logs, set the property to["function"]
. To collect bothplatform
andfunction
logs, set the property to["platform", "function"]
(this is the default value).
Endpoint
OneAgent version 1.275+
The endpoint that is used for exporting logs to is based on your configuration. This endpoint that is used by default is derived from the base URL of the default configuration from the deployment screen. You can override this value depending on your configuration method,
- If you are using environment variables for configuration, the default log collection endpoint is derived from
DT_CONNECTION_BASE_URL
. To override the default value, set theDT_LOG_COLLECTION_ENDPOINT
environment variable to a specific endpoint. - If you are using the
dtconfig.json
configuration file, the default log collection endpoint is derived fromConnection.BaseUrl
. To override the default value, add theEndpoint
property to theLogCollection
object and set it to a specific endpoint.
The value that specifies the endpoint must include the full endpoint URL, including the path.
Limitations
- The
extension
event type currently is not supported: if you try to configure theextension
event type, an error is shown and log collection does not start. - By default, logs collected by the log collector are not associated with traces and are not shown in the trace view in the Logs tab, and will only be shown on the Related logs card or in the Log viewer. However, you may use manual log enrichment in order to connect your function logs to your traces. See AWS Lambda logs in context of traces for language-specific examples of manual log enrichment.
- In the Dynatrace web UI, logs are marked as error logs only if the log message contains the string
[error]
or[ERROR]
. - We do not recommend combining the standalone collector layer with a OneAgent layer. If you want tracing and log collection for your Lambda functions, use the combined Traces and Logs layer instead.
- Dynatrace Managed The environment variable
DT_CONNECTION_BASE_URL
can't be copied from the deployment screen. Instead, the value ofDT_CONNECTION_BASE_URL
must be set to a URL in the form ofhttps://{activegate-host}:9999/e/{your-environment-id}
.
Troubleshooting
Dynatrace does not ingest logs (HTTP 429)
If the log exporter prints errors saying that log ingest is disabled or receives
an HTTP 429 status code, then you first need to make sure you are using Log
Monitoring
v2.
Once you are, you should then check that your log events ingest limit is
sufficient by navigating to Settings > Internal > Indexed Log Storage
Settings in your tenant and entering a reasonable value for Maximum ingest of
Log Events (100000
is the default).
Debug logging
If other methods of troubleshooting don't resolve your issue, you can enable debug logging of the log collector itself. To do so, set environment variable DT_LOG_COLLECTION_LOG_LEVEL
to debug
and set DT_LOGGING_DESTINATION
to stdout
in the Lambda function's configuration. This will cause the log collector to print debug logs to the Lambda function's log stream. You can then view these logs in the CloudWatch console. Note that the collector will not send its own logs to Dynatrace. Also be aware that enabling debug logging should only be used temporarily, as it will generate a large amount of logs in CloudWatch, which may impact the performance of the function.
Alternatively, if you are using dtconfig.json
configuration, you can enable debug logging by setting the LogCollection.LogLevel
and Logging.Destination
properties:
{
...other values...
"Logging": {
"Destination": "stdout"
},
"LogCollection": {
...other values...
"LogLevel": "debug"
}
}