Log data storage

Log Monitoring enables you to store all logs centrally within external storage. This makes log data available independent of log files themselves. This can be beneficial in the following situations:

  • Short log retention periods
  • Volatile log storage
  • Legal requirements for keeping logs archived centrally for long time periods

In addition you can also:

  • Analyze multiple logs simultaneously
  • Parse log or JSON files
  • Generate metrics from log content

Log storage requirements and costs

Dynatrace SaaS

For Dynatrace SaaS customers, log files are stored in the same AWS availability zone where your Dynatrace environment resides. You don’t have to worry about storage performance, availability, or free space. Disk storage costs are included in your Log Monitoring subscription. Costs are based on the average size of your cloud-based log storage, including the amount of streamed log data and the defined retention period. For details, see How to calculate monitoring consumption and Data retention periods.

Dynatrace Managed

To store log files centrally on your Dynatrace Managed cluster, you must provide a common Network File System (NFS) mount point (path) that is identical throughout the cluster and available from all cluster nodes. With this approach, it's your responsibility to ensure appropriate levels of performance, availability, and free space on the mounted NFS volume. Costs are calculated based only on the amount of ingress log data (GB/day), not total storage size, so retention time doesn't influence storage costs. For deployments on AWS, we recommend that you use the Amazon Elastic File System for your log storage.

Before configuring the path for log storage, make sure that you have the appropriate license:

  1. From the navigation menu, select Licensing.
  2. In the Licensing section, look for the Avg. daily log volume license.
    • If the Avg. daily log volume is not listed, contact your sales representative for a Log Monitoring license.
    • If the Avg. daily log volume is listed, proceed to configuring the central storage location for your logs.

To set up your central location for log storage:

  1. From the navigation menu, select Settings > Log Monitoring.

  2. Set the Use network attached storage switch to the On position.

  3. Click the edit button and type the mount point (for example, /usr/local/path/to/storage) to the network resource.

    Make sure that all Dynatrace cluster nodes have write access to the mount point you indicated.

  4. Restart the Dynatrace cluster using the Restart button for each of the nodes in the cluster.

  5. For each monitored environment, in the Set total environment quotas section, set the Log monitoring storage to a non-zero value (default 0 MB).

Infrastructure impact

The log writing queue can take up to 1% of available memory on a cluster node. The CPU is typically not affected much. For metrics, a single node can process about two million log entries per second.

Because of the volume of logs, log files are compressed using Zstandard before they're sent for analysis. This task requires an average of 10% of compute power. Worst case, this task will require about 25% of compute power.

For the ActiveGate component, provide one physical core per 50 Mbps of traffic (compressed content).