Log Monitoring default limits

The following page lists default limits for the latest version of Dynatrace Log Monitoring.

The below limitations apply to both log file ingestion and generic log ingestion via API:

Log events per minute Managed

Log data ingestion is limited by default to 10,000 log events per minute per cluster.

• If your log data stream within your cluster exceeds the limit, all log events above the limit are ignored.
• If you increase resources (RAM) in your nodes, you can increase the limit based on the cluster resources size using an API call or Cluster Management Console (CMC).

This applies to all event sources (OneAgent and generic log ingestion). You can use self-monitoring metrics to get a count of incoming log events and log events rejected because this limit was exceeded. For details, see Self-monitoring metrics.

Refresh cluster limit using the API call
See Update log events per cluster for Log Monitoring.

Refresh cluster limit using Cluster Management Console (CMC)

1. In the CMC, select Environments and the environment for which you wish to update the total log events per cluster.
2. On the environment details page, in the Cluster overload prevention settings section, select the Refresh cluster limit.

Payload size Managed SaaS

The maximum payload size of a single request is 1 MB.

Log event minimum timestamp Managed SaaS

The earliest timestamp for a log event is the current time minus 24 hours. This applies to all event sources (OneAgent and generic log ingestion). If the log event contains timestamp earlier than current time minus 24 hours, the event is dropped and the generic log ingestion API returns a 400 response code.

Log event maximum timestamp Managed SaaS

The latest timestamp for a log event is the current time plus 12 hours. This applies to all event sources (OneAgent and generic log ingestion). If the log event contains timestamp later than current time plus 12 hours, the event is dropped and the generic log ingestion API returns a 400 response code.

Maximum attributes Managed SaaS

A log event can have up to 50 attributes. Additional attributes (as they appear in the JSON stream) are ignored.

High-cardinality attributes Managed SaaS

Unique log data attributes (high-cardinality attributes) such as span_id and trace_id generate unnecessarily excessive facet lists that may impact log viewer performance. Because of this, they aren't listed in log viewer facets. You can still use them in a log viewer advanced search query.

Log ingestion API request objects Managed SaaS

In addition to generic Dynatrace API limitations (Dynatrace API - Access limit) the following log ingestion API specific limits apply:

• LogMessagePlain plain text object.
  The length of the message is limited to 8,192 characters. Any content exceeding the limit is trimmed.

• LogMessageJson JSON object.
  The object might contain the following types of keys (the possible key values are listed below):

  • Timestamp. The following formats are supported: UTC milliseconds, RFC3339, and RFC3164. If not set, the current timestamp is used.
  • Severity. If not set, NONE is used.
  • Content. If the content key is not set, the whole JSON is parsed as the content.
  • Semantic attribute. Only values of the String type are supported. Semantic attributes are indexed and can be used in queries. These are also displayed in aggregations (facets). If an unsupported key occurs it is not indexed and cannot be used in indexing and aggregations.

  The length of the value is limited. Any content exceeding the limit is trimmed. Default limits:

  • Content: 8,192 characters.
  • Semantic attribute value: 250 characters.

OneAgent limits Managed SaaS

There are limits specific to OneAgent, including:

Limitations for automatically detected files

• Log files in OneAgent cannot be deleted earlier than a minute after creation.
• Log files in OneAgent must be appended (old content is not updated).
• Log files in OneAgent must have text content.
• Log files in OneAgent must be opened constantly (not just for short periods of adding log entry).
• Log files in OneAgent must be opened in write mode.

Unsupported rotated log monitoring scenarios

Scenarios that are not supported in the rotated log monitoring process in OneAgent include:

• Rotated log generation with a directory change. The potential consequence is the creation of numerous non-aggregated and/or incomplete logs.
• Rotated log generation with immediate compression, where the application logs a file with the same name. If a rotation criterion is met (for example, the required file size is reached), the file is moved to another location and immediately compressed. Example: /var/log/application.log -> /var/log/application.log.1.gz -> /var/log/application.log.2.gz -> /var/log/application.log.3.gz. This process might again lead to incomplete log ingest.