Generic log ingestion

Log Monitoring Classic

Dynatrace automatically collects log and event data from a vast array of technologies. With generic log ingestion, you can stream log records to a system and have Dynatrace transform the stream into meaningful log messages.

Generic log ingestion allows you to stream log records to the system. It is available only via Log Monitoring API - POST ingest logs.

For Dynatrace SaaS, the generic logs ingest endpoint is available in your environment.
For Dynatrace Managed or if the Environment ActiveGate is your choice for an endpoint in your local environment, install an ActiveGate instance (In the Dynatrace menu, go to Deploy Dynatrace and select Install ActiveGate). The generic log ingest API v2 is automatically enabled on ActiveGate.
The endpoint is enabled by default on all of your ActiveGates.
ActiveGate is responsible for serving the endpoint, collecting the data, and forwarding it to Dynatrace in batches.
- SaaS https://{your-environment-id}.live.dynatrace.com/api/v2/logs/ingest
- Environment ActiveGate https://{your-activegate-domain}:9999/e/{your-environment-id}/api/v2/logs/ingest

ActiveGate will collect and attempt to automatically transform any log data containing the following elements:

Log content
Timestamp
Key-Values attributes

To list all predefined key-values attributes

In the user menu, go to Environment API v2.
Select Logs and expand the POST/logs/ingest endpoint.
In the Request body section, select the Schema tab and check the Supported semantic attribute keys section.

Log data queue

You can customize the log data queue properties by editing the custom.properties file (see Configuration properties and parameters of ActiveGate on your ActiveGate to set the following values:

plaintext
[generic_ingest]
#disk_queue_path=<custom_path> # defaults to temp folder
#disk_queue_max_size_mb=<limit> # defaults to 300 MB

503 Usable space limit reached

The log data ingestion API returns a 503 Usable space limit reached error when the ingested log data exceeds the configured queue size. Typically, this is a temporary situation that occurs only during spikes. If this error persists, increase the value of disk_queue_max_size_mb in custom.properties to allow log ingestion spikes to be queued.

Example

In this example, the API request ingests log data that will create a log event with defined log attributes content, status, service.name, and service.namespace.

The API token is passed in the Authorization header.

The response contains response code 204.

Curl

bash
curl -X POST \
   https://environment.activegate.domain.com:9999/e/abc123a/api/v2/logs/ingest \
  -H 'Content-Type: application/json; charset=utf-8' \
  -H 'Authorization: Api-Token dt0c01.abc123.abcdefjhij1234567890' \
  -d '[
    {
    "content": "Exception: Custom error log sent via Generic Log Ingest",
    "status": "error",
    "service.name": "log-monitoring-tenant",
    "service.namespace": "dev-stage-cluster"
    }
]'

Request URL

bash
https://environment.activegate.domain.com:9999/e/abc123a/api/v2/logs/ingest

Response content

plaintext
Success

Response code

204