DDUs for Log Management and Analytics

The DDU consumption model outlined here for Log Management and Analytics only affects Dynatrace SaaS environments that are activated for Dynatrace Grail for log data management. DDU consumption for all other DDU capability types, including Log Monitoring for Dynatrace SaaS & Managed, remains unchanged. For details about activating Log Management and Analytics for your Dynatrace environment, contact a Dynatrace ONE product specialist by selecting the Chat button in the upper-right corner of the Dynatrace web UI.

This page explains how DDUs are consumed by Dynatrace Log Management and Analytics and how you can estimate and track your environment’s DDU consumption.

DDU consumption model for Log Management and Analytics

The DDU consumption model for Log Management and Analytics is based on three dimensions of data usage (Ingest & Process, Retain, and Query). The unit of measure for consumed data volume is gigabytes (GB). Each of the three data usage dimensions consumes DDUs based on a different weight (see the DDU consumption row in the table below for details).

Total DDU consumption for Log Management and Analytics is calculated by multiplying the DDU weight of each of the three data-usage dimensions with the data volume in GB.

Unit of measure	Ingest & Process	Retain	Query
Definition	Ingested data is the amount of raw data in bytes (logs and events) sent to Dynatrace before enrichment and transformation.	Retained data is the amount of data saved to storage after data parsing, enrichment, transformation, and filtering but before compression.	Queried data is the data read during the execution of a DQL query, including sampled data.
DDU consumption	100.00 DDUs per GB	0.30 DDUs per GB retained per day	1.70 DDUs per GB read

Ingest & Process

What‘s included with the Ingest & Process data-usage dimension?

Concept	Explanation
Data delivery	Delivery of log data via OneAgent or Generic Log Ingestion API (via ActiveGate)
Topology enrichment	Enrichment of log events with data source and topology metadata
Data transformation	Add, edit, or drop any log attribute Perform mathematical transformations on numerical values (for example, creating new attributes based on calculations of existing attributes) Extract business, infrastructure, application, or other data from raw logs. This can be a single character, string, number, array of values, or other. Extracted data can be turned into a new attribute allowing additional querying, filtering, etc. Metrics can be created from newly extracted attributes (see Conversion to time series below), or extracted attributes can be queried for ad-hoc analysis Mask sensitive data by replacing either the whole log record, one specific log record attribute, or certain text with a masked string
Data-retention control	Filter incoming logs based on content, topology, or metadata to reduce noise. Log filtering consumes DDUs during Ingest & Process, but not for Retain. Manage data retention periods of incoming logs based on data-retention rules
Conversion to time series	Create metrics from log records or attributes (note that creating custom metrics consumes additional DDUs beyond those consumed for ingestion and processing.)

Apply the following calculation to determine your DDU consumption for the Ingest & Process data-usage dimension:
(number of GBs ingested) x (100.00 DDU weight) = DDUs consumed

Be aware that data enrichment and processing can increase your data volume significantly. Depending on the source of the data, the technology, the attributes, and metadata added during processing, the total data volume after processing can increase by a factor of 2 or more.

Retain

What’s included with the Retain data-usage dimension?

Concept	Explanation
Data availability	Retained data is accessible for analysis and querying until the end of the retention period.
Retention periods	Choose a desired retention period: 35 days (default) 1 year 3 years

Apply the following calculation to determine your DDU consumption for the Retain data-usage dimension:
(number of GB of processed data ingested) x (retention period in days) x (0.30 DDU weight) x (number of days that data is stored) = DDUs consumed

Query

Query data usage occurs when:

Accessing the Logs & Events viewer in simple mode with filters.
Submitting custom DQL queries in the Logs & Events viewer in advanced mode.
Unified analysis pages show the log data of a specific entity.
Dashboard tiles that are based on log data trigger the execution of DQL queries on refresh and include sampled data.
Executing DQL queries via API.

What’s included with the Query data-usage dimension?

Concept	Explanation
On-read parsing	Use DQL to query historical logs in storage and extract business, infrastructure, or other data across any timeframe, and use extracted data for follow-up analysis. No upfront indexes or schema required for on-read parsing
Aggregation	Perform aggregation, summarization, or statistical analysis of data in logs across specific timeframes or time patterns (for example, data occurrences in 30-second or 10-minute intervals), mathematical, or logical functions.
Reporting	Create reports or summaries with customized fields (columns) by adding, modifying, or dropping existing log attributes.
Context	Use DQL to analyze log data in context with relevant data on the Dynatrace platform, for example, user sessions or distributed traces.

Apply the following calculation to determine your DDU consumption for the Query data-usage dimension:
(number of GB of uncompressed data read during query execution) x (1.70 DDU weight) = DDUs consumed

Calculate DDU consumption across data-usage dimensions

Following are example DDU calculations which show how each data-usage dimension contributes to overall DDU consumption.

Step 1 – Ingest & Process

For example, say that you produce 500 GB of log data per day which you ingest into Log Management and Analytics for processing. The monthly DDU consumption for Ingest & Process in this case would be 1,500,000 DDUs:


Ingest volume per day	500 GB
Ingest volume per month	15,000 GB	500 (GB data per day) x 30 (days)
DDU consumption per month	1,500,000 DDUs	15,000 (GB per month) x 100.00 (DDUs per GB)

Step 2 - Retain

Following the Ingest & Process step, your data is retained and enriched on an on-going basis. If you ingested 500 GB of raw data in step 1, 900 GB of enriched data (500 GB x 1.8 for enrichment) is added to your storage daily. In this example, your enriched data is retained for 35 days. The monthly DDU consumption (after a ramp-up period of 35 days) for Retain in this case is 283,500 DDUs:


Retained volume for 1 day	900 GB	500 (GB data per day) x 1.8 (enrichment)
Retained volume for 35 days	31,500 GB	900 (GB data per day) x 35 (days)
DDU consumption per day	9,450 DDUs	31,500 (GB) x 0.3 (DDUs per GB per day)
DDU consumption per month	283,500 DDUs	9,450 (DDUs) x 30 (days)

If the same amount of processed data is to be retained for a year, the monthly DDU consumption (after a ramp-up of 365 days in this case) for Retain is 2,956,500 DDUs.


Retained volume for 1 day	900 GB	500 (GB data per day) x 1.8 (enrichment)
Retained volume for 365 days	328,500 GB	900 (GB data per day) x 365 (days)
DDU consumption per day	98,550 DDUs	328,500 (GB) x 0.3 (DDUs per GB per day)
DDU consumption per month	2,956,500 DDUs	98,550 (DDUs) x 30 (days)

Step 3 - Query

Let’s assume that to resolve incidents and analyze performance issues your team executes DQL queries with a total of 25 TB of data read per day. The monthly DDU consumption for Query in this case is 1,275,000 DDUs.


Data volume read per day	25,000 GB
Data volume read per month	750,000 GB	25,000 (GB data per day) x 30 (days)
DDU consumption per month	1,275,000 DDUs	750,000 (GB per year) x 1.70 (DDUs per GB)

Step 4 – Total DDU consumption

The total monthly DDU consumption for this example scenario of 35 days of data retention is 3,058,500 DDUs.


Ingest – DDU consumption per month	1,500,000 DDUs
Retain – DDU consumption per month	283,500 DDUs
Query – DDU consumption per month	1,275,000 DDUs
Total DDU consumption per month	3,058,500 DDUs

FAQs

Can I change the retention period for Log Management and Analytics?

Beginning with Dynatrace version 1.253, you can manually set the data retention period for your SaaS environment to 35 days (default), 1 year, or 3 years.

Where can I check my environment’s DDU consumption for logs and events?

In the Dynatrace menu, go to Manage > Consumption. You’ll need the change monitoring settings permission or an admin account to access this page.
On the Davis data units (DDU) page:
a. In the Consumption by DDU pool table, see the Log Monitoring row.
b. In the DDU consumption details section, go to the Log Monitoring tab.

What is the DDU weight of one log or event record?

For Log Management and Analytics, volume of data in gigabytes (GB) is the unit of measure. Consumption is based on three data-usage dimensions. The weight of DDU consumption per dimension is as follows:

Ingest & Process: 100.00 DDUs per GB of data ingested and processed
Retain: 0.30 DDUs per GB per day for data stored
Query: 1.70 DDUs per GB for data read from storage during query execution

What is the size of one log or event record?

The number and size of individual log records isn’t relevant for DDU consumption. For Log Management and Analytics, DDU consumption for a given period is calculated based on volume of data ingested and processed, volume of data stored per day, and volume of data read from storage during query execution.

Are there any host-included DDUs available for Log Management and Analytics?

No. Log Management and Analytics always consumes DDUs for Ingest & Process, Retain, and Query. Note that you can choose to only ingest and process data without storing it or querying it. Each Dynatrace environment includes a free tier of 200,000 DDUs per year, which can be used for Ingest & Process, Retain, and Query.

I’m currently using my DDUs for Log Monitoring for Dynatrace SaaS & Managed, custom metrics, custom traces, custom events, and serverless functions and I don’t plan to migrate. Will anything change for me?

No. The DDU consumption model outlined here for Log Management and Analytics only affects Dynatrace SaaS environments that are activated for and connected to a Dynatrace GrailTM cluster for log data management. DDU consumption for all other DDU capability types, including Log Monitoring for Dynatrace SaaS & Managed, remains unchanged.

I’m considering migrating to Log Management and Analytics. What will change for me?

If Log Management and Analytics is activated for your environment, the DDU consumption model that uses gigabytes as the unit of measure will replace the event-based consumption model that’s used for Log Monitoring for Dynatrace SaaS & Managed. Consumption of other DDU-based capability types, including Log Monitoring for Dynatrace SaaS & Managed (if you continue to use it in parallel) will remain unchanged.

Does the Dynatrace platform consume DDUs for fractional GBs? Would 0.5 GB of Ingest & Process consume only 50 DDUs?

Yes. Every ingested GB (or fraction thereof, before enrichment and processing) is added together and then multiplied by the DDU weight of 100.00 DDUs. For example, DDU consumption for Ingest & Process of 10.50 GB equates to 1,050 DDUs. 10.50 (GB log data) x 100.00 (DDU weight) = 1,050 DDUs

I want to convert my log messages into metrics. How will this affect my environment’s DDU consumption?

While writing ingested logs to time series is a cost-effective way of visualizing log-based metrics, this approach consumes additional DDUs for creating custom metrics in addition to the DDUs consumed for Ingest & Process.

Do failed query executions consume DDUs?

No, when internal failures occur while executing a query (for example, a time-out) no DDUs are consumed.

Do canceled query executions consume DDUs?

If you cancel a query execution, all data read before the cancellation will be factored into your DDU consumption.