Bucket assignment

powered by Grail

Logs powered by Grail can be stored in different buckets. Buckets can improve query performance by reducing query execution time and the scope of data read.

Default log buckets

Grail contains built-in Grail buckets with default retention periods. The default built-in bucket intended for log data is default_logs with a retention period of 35 days.

• Built-in Grail buckets

Create and manage custom log buckets

With the Dynatrace API, you can create a custom log bucket tailored to your needs. Using a custom log bucket, you can:

• Store log data with the same retention period.
• Store log data that needs to be queried/analyzed together.
• Store log data that needs to be deleted at the same time.

Defining log buckets can improve query performance by reducing query execution time and the scope of data read. Also, you can use buckets to set unique access rules to different logs or log areas.

• Custom Grail buckets

Permissions can be assigned on the bucket, table, and entity levels. You can set up permissions and choose from six predefined global policies. For details, see:

• Bucket and table permissions
• IAM storage service permissions.

Assign log data to a bucket

To choose a log bucket for specific log data, create a log bucket rule.

1. In the Dynatrace menu, go to Settings > Log monitoring > Bucket assignment.
2. Select Add rule.
3. Define the rule.
   • Set Rule name to the name you want to display for your rule.
   • Set Bucket to the bucket where you want to store the log data.
     • If you need buckets with specific retention times, you can create custom Grail buckets. For details, see Custom Grail buckets.
     • If you need to exclude certain logs from storage, select Don't store in a bucket. Logs excluded from storage are still processed by log events and metrics.
   • Set Matcher (DQL) to a Dynatrace Query Language (DQL) expression that describes the log data for this rule.
     • See DQL matcher in logs for details on writing a matcher that will match the ingested log data for log processing, log metrics, log events, and log buckets.
     • If you set Bucket to Don't store in a bucket, the expression needs to match the logs that you do not want to store.
4. Select Save changes.

Example 1: Store logs for 3 years

In this example, we need to have a 3-year retention period for the logs whose content contains the word error.

1. Create a custom log bucket with a 3-year retention period and name it Logs 3-year retention.
   For details, see Manage custom Grail buckets.

2. In the Dynatrace menu, go to Settings > Log monitoring > Bucket assignment and create a rule.

   • Set Rule name to Error log 3-year retention.
   • Set Bucket to Logs 3-year retention (the bucket created in the previous step).
   • Set Matcher (DQL) to the matcher for the ingested log data.

     dqlcopydownload
     matchesPhrase(content, "error")

3. Select Save changes.

This rule will place all ingested log data that matches the phrase error in the content in the log bucket with the 3-year retention period.

Example 2: Exclude logs

In this example, we need to exclude all logs that have the drop value in their log.source attribute.

1. In the Dynatrace menu, go to Settings > Log monitoring > Bucket assignment and create a rule.
   • Set Matcher (DQL) to Exclude drop logs.
   • Set Bucket to Don't store in a bucket.
   • Set Matcher (DQL) to the matcher for the ingested log data.

     dqlcopydownload
     matchesValue(log.source, "drop")

2. Select Save changes.

This rule will exclude log data that has the drop value in their log.source attribute.