Log storage configuration

Dynatrace 1.244+ OneAgent 1.243+ Preview

Dynatrace allows you to include and exclude specific log sources for analysis by Dynatrace Log Monitoring. Using Dynatrace identity and access management (IAM) framework, you can control which user can change configurations on which scope.

The configuration is based on rules that use matchers for hierarchy, log path, and process groups. These rules determine the upload of log files known to OneAgent, auto-detected log files, and custom log files defined per process group.

Supported scopes

Three hierarchy scopes are supported: host, host group, and tenant. The scope with the least possible set of rules has priority over larger sets.

1. Log storage rules configured for a host take precedence over log storage rules configured for a host group.
2. Log storage rules configured for a host group take precedence over log storage rules configured for a tenant.

Host scope

The host scope can be accessed through the Host settings for a specific host.

1. In the Dynatrace menu, go to Hosts and select your host.
2. Select More (…) > Settings to open the Host settings page (available only on hosts assigned to a host group).
3. On the Host settings page, select Log storage.
4. Configure storage upload by adding rules with a set of attributes that matches the log data to be stored by Dynatrace.

Host group scope

The host group scope can be accessed via the Host page.

1. In the Dynatrace menu, go to Hosts and select your host.
2. In the Properties and tags section, select the Host group (available only on Hosts assigned to a Host group).
3. On the Settings page, select Log storage.
4. Configure storage upload by adding rules with a set of attributes that matches the log data to be stored by Dynatrace.

Tenant scope

The tenant scope is available in the settings menu.

1. In the Dynatrace menu, go to Settings and select Log Monitoring > Log storage.
2. Configure storage upload by adding rules with a set of attributes that matches the log data to be stored by Dynatrace.

Matching rules to log data

Matching occurs in a predefined hierarchy and rules are executed from top to bottom. This means that if a rule above on the list matches certain log data, then the lower ones will be omitted. Items matched in the higher-level configurations are overwritten in the lower-level configurations if they match the same log data. The matching hierarchy is as follows:

1. Host configuration rules
2. Host group configuration rules
3. Tenant configuration rules

Configure log storage

1. In the Dynatrace menu, go to Settings and select Log Monitoring > Log storage.

2. Select Create new rule and provide the name for your configuration.
   By default, the Send to storage switch is turned on, indicating that items configured by this rule will be stored in Dynatrace. If you disable the switch, matching log data will be excluded from storage.

3. Expand Details of your new rule and select Add matcher to create a specific match for this rule.
   Multiple matchers can be included in one rule.

4. Select the matching attribute:

   Attribute	Description
   Container name	Matching is based on the name of the container.
   K8s container name	Matching is based on the name of the Kubernetes container.
   K8s deployment name	Matching is based on the name of the Kubernetes deployment.
   K8s namespace name	Matching is based on the name of the Kubernetes namespace.
   Log content	Matching is based on the content of the log; wildcards are supported in form of an asterisk.
   Log source	Matching is based on a log path; wildcards are supported in form of an asterisk.
   Process group	Matching is based on the process group ID.
   Process technology	Matching is based on the technology name.

   You can filter out log content using the Log content attribute in the Add matcher section. If no wildcard is used in the value, then this matcher will look for an exact match to the value. If a wildcard is used, the behavior will change to matches containing the value. For example, the value INFO will result in sending log data in which the entire content is INFO, but the value *INFO* (using the wildcards) will match log data that contains the INFO string anywhere in its content.

5. Select Add value and, from the Values, select the detected log data items (log files or process groups that contain log data). Multiple values can be added to the selected attribute. You can have one matcher that indicates log source and matches values /var/log/syslog and Windows Appication Log.

6. Save changes.

Defined rules can be reordered and are executed in the order in which they appear on the Log storage page.

Example upload

In this example, we configure the tenant storage upload for c:\inetpub\logs\LogFiles\ex_*.log files in two process groups: IIS (PROCESS_GROUP-3D9D854163F8F07A) and IIS (PROCESS_GROUP-4A7B47FDB53137AE). The log storage rule consists of two matchers: the first matcher finds the process groups and the second matcher matches only for the defined log source.

1. In the Dynatrace menu, go to Settings and select Log Monitoring > Log storage.
2. Select Create new rule and provide the title for your configuration.
3. Select Add matcher. This is the first matcher to match two specified process groups.
4. From the Attribute list, select Process group.
5. Select Add value and type IIS, and then, from the suggestion list, select IIS (PROCESS_GROUP-3D9D854163F8F07A).
6. Select Add value again, type IIS and select the second process group from the suggestion list: IIS (PROCESS_GROUP-4A7B47FDB53137AE).
7. Select Add matcher again. This is the second matcher to match the specified log data source.
8. From the Attribute list, select Log source.
9. Select Add value and enter c:\inetpub\logs\LogFiles\ex_*.log as the value.
10. Save changes.

Example exclude

In this example, we configure the tenant storage upload for all log sources except c:\inetpub\logs\LogFiles\ex_*.log files in a process group IIS (PROCESS_GROUP-4A7B47FDB53137AE).

1. In the Dynatrace menu, go to Settings and select Log Monitoring > Log storage.
2. Select Create new rule and provide the title for your configuration.
3. Turn off Send to storage.
4. Select Add matcher. This is the first matcher to match the specified process group.
5. From the Attribute list, select Process group.
6. Select Add value and type IIS, and then, from the suggestion list, select IIS (PROCESS_GROUP-3D9D854163F8F07A).
7. Select Add matcher again. This is the second matcher to exclude the specified log data source.
8. From the Attribute list select Log source.
9. Select Add value and enter c:\inetpub\logs\LogFiles\ex_*.log as a value.
10. Save changes.

Migration to the new storage configuration

The switch to the new storage configuration is done automatically after enabling the feature. The following changes will occur in your current configuration:

• Host perspective
  All items configured on the Hosts perspective are migrated as a set of matchers to the corresponding host scope.

• Process groups perspective
  All items configured from the Process groups perspective are migrated to the tenant scope.

After your configuration of log sources is successfully migrated, you can use new configuration items and add your matchers.

REST API

You can use the Settings API to manage your log storage configuration:

• View schema
• List stored configuration objects
• View single configuration object
• Create new, edit, or remove existing configuration object

To check the current schema version for log storage configuration, list all available schemas and look for the builtin:logmonitoring.log-storage-settings schema identifier.

Log storage configuration objects are available for configuration on the following scopes:

• tenant – configuration object affects all hosts on a given tenant.
• host_group – configuration object affects all hosts assigned to a given host group.
• host – configuration object affects only the given host.

To create a log storage configuration using the API:

1. Create an access token with the Write settings (settings.write) and Read settings (settings.read) permissions.

2. Use the GET a schema endpoint to learn the JSON format required to post your configuration. The log storage configuration schema identifier (schemaId) is builtin:logmonitoring.log-storage-settings. Here is an example JSON payload with the log storage configuration:

   jsoncopydownload
   [
     {
       "insertAfter":"uAAZ0ZW5hbnQABnRlbmFudAAkMGUzYmY2ZmYtMDc2ZC0zNzFmLhXaq0",
       "schemaId": "builtin:logmonitoring.log-storage-settings",
       "schemaVersion": "0.1.0",
       "scope": "tenant",
       "value": {
         "config-item-title": "Added from REST API",
         "send-to-storage": true,
         "matchers": [
           {
             "attribute": "dt.entity.process_group",
             "operator": "MATCHES",
             "values": [
               "PROCESS_GROUP-05F00CBACF39EBD1"
             ]
           },
           {
             "attribute": "log.source",
             "operator": "MATCHES",
             "values": [
               "Windows System Log",
               "Windows Security Log"
             ]
           }
         ]
       }
     }
   ]

3. Use the POST an object endpoint to send your configuration.

FAQ