Data storage penalty

Prototype feature

This prototype feature is disabled. The description that follows is shared for informational purposes only. Do not rely on this information in making purchasing decisions. The development, release, and timing of this feature or functionality remains at the sole discretion of Dynatrace, and it is subject to change.

A data storage penalty is a functionality in which Dynatrace periodically increases or decreases a penalty value that determines how much data from Transaction storage and session replay storage should be deleted if the tenant environment storage quota is exceeded or if disk space doesn't meet the needs of your Dynatrace Managed environment.

Each data type stored on a disk has a default retention time which describes how long the given data type can be stored on a disk.

The maximum amount of a given data type that can be stored on disk is determined by three values:

  • The default retention time defines how long that data type can be stored on the disk if there is no storage penalty.

  • The real retention time is the default retention time adjusted by the storage penalty.

  • The storage penalty is a percentage that is adjusted based on disk space (applies only to Dynatrace Managed) and whether the tenant has more data than its quota allows.

For example, if the default retention time is 10 days and the penalty is 10%, then the real retention time is 9 days:

Real retention time = 10 days * ((10010) / 100)
Real retention time = 10 * 0.9
Real retention time = 9 days 

The 9 days of real retention time means that all data that is older than 9 days will be deleted quickly, even though the default retention time is 10 days.

How the storage penalty is calculated

The penalty is increased if:

  • The disk is full (applies only to Dynatrace Managed)
  • The tenant uses more data than its quota allows

These conditions are continuously evaluated. If any of these conditions are violated, the penalty is increased and deletion begins.

After data is deleted according to the calculated penalty:

  • If enough data was deleted and none of the conditions are violated anymore, the penalty is decreased.
  • If one or more conditions are still violated, the penalty is increased and more data is deleted.

What to do if the penalty is applied

A storage penalty can be caused by an undersized disk or an inappropriate quota. If you would like to have your older data accessible in Dynatrace and prevent data deletion:

  • Increase your quota
  • Increase your disk size

Troubleshooting

The penalty based deletion changes the behavior under certain circumstances for example, not just one session chunk of session replay data is deleted. When the amount of data diverges heavily from the configured quota or available diskspace, penalty can reach a very high value which can cause too much deletion, and as a result, the amount of processed data and the available storage (limited by disk space or quota) do not align. To alleviate that, you can:

  • Increase available disk space or quota.
  • Decrease the retention time for that environment, so that an increase of the penalty by 10% has less impact.
    For example, because the penalty value is a percentage of your data retention time, a longer data retention time will proportionally cause more data (and therefore more hours) to be deleted.