Upgrading Log Monitoring

Log Management and Analytics is the latest Dynatrace log monitoring solution. In addition, Dynatrace contains legacy log solutions:

• Log Monitoring Classic (LMv2)
• Log Monitoring v1 (LMv1)

With the introduction of Dynatrace Platform and Grail, we encourage you to upgrade to the latest log monitoring offer:

Upgrade to Log Management and Analytics

powered by Grail

If you use Dynatrace SaaS on AWS, your environment will be enabled for Log Management and Analytics powered by Grail with a phased rollout.

For more information about the phased rollout, please reach out to one of your Dynatrace Account team members. You can also reach out directly to the Dynatrace ONE team by opening an in-product chat. It will get you in touch with your account team members and allow to get answers to any other questions you may have.

Once your environment is enabled for activation:

1. Go to Dynatrace menu > Observe and Explore > Logs.
2. In the banner message, select Go to activation page and select Activate Logs powered by Grail.

What changes after activation

After activating Log Management and Analytics, the following changes take place:

• Ingested log data

  Your existing log data will not be available.

  • Ingested log data is saved in the Grail database.

  • Ingested log data can be routed to buckets with different retention periods.

• Logs in the Dynatrace menu points to the new Logs and events, where two query modes are available:

  • Simple mode, where data can be searched and analyzed using attribute filters.

  • Advanced mode, where data can be searched and analyzed using DQL.

  • You start consuming DDUs in a new model with three dimensions: Ingest & Process, Retain, Query.

• Log API data

  • The Log GET search and Log GET aggregate APIs require OAuth2 token.
  • The log export API will not be available. Instead, log data in Grail can be queried with Dynatrace API Storage Query Service.

What does not change after activation

After activating Log Management and Analytics, the following will not change:

• Ingestion configuration, including OneAgent configuration and generic API ingest.
• Log processing, including processing rules with matchers based on the LQL syntax.
• Log metrics, including metric queries based on the LQL syntax.
• Log events, including event queries based on the LQL syntax.

User access

The user access granting process depends on whether you are a new or existing user.

• Assign policy to existing users
  After activating Log Management and Analytics, all users who already had access to log data are assigned a new policy to access the log data in Grail.

• Assign policy to new users

  There are two options for configuring access policies for Grail:

  Assign policy using Account Admin

  In Dynatrace SaaS, only admin users can manage policies (users with account permission Manage users).
  You need to have two policies, Storage Events Read and Storage Logs Read assigned, bound to a group. To check if your policies are assigned:

  1. In the user menu, select Account settings.
  2. Go to Identity management > Policy management.
  3. Check if Storage Events Read and Storage Logs Read are present on the policy list.

  If Storage Events Read and Storage Logs Read are not present on you policy list, you need to add them manually:

  • Storage Events Read:
    Policy name: Storage Events Read
    Policy description: Enables reading events from GRAIL
    Policy statements: ALLOW storage:events:read
  • Storage Logs Read:
    Policy name: Storage Logs Read
    Policy description: Enables reading logs from GRAIL
    Policy statements: ALLOW storage:logs:read
    For details, see Manage IAM policies.

  To make a policy effective, you need to bind it to a group.

  1. In the user menu, select Account settings.
  2. Go to Identity management > Group management.
     For details, see Manage group permissions with IAM policies.
  3. Edit the group to which you want to bind the policy (for example, Logs and events). Make sure the users who need to use the Logs and events have this group assigned to their names.
  4. Select the Policies tab.
  Assign policy via API

  1. Obtain an OAuth token Make a POST call with form parameters to SSO.

     • client_id = [client_id]
     • client_secret = [secret]
     • grant_type = client_credentials
     • scope = iam:policies:write iam:policies:read

     In response, you get an authorization token

     jsoncopydownload
     {
       "scope": "iam:policies:read iam:policies:write",
       "token_type": "Bearer",
       "expires_in": 300,
       "access_token": "123(...)ABC"
     }

  2. Create a storage events read policy Make a POST call to IAM

     Body payload for the policy is:

     jsoncopydownload
     {
       "name": "Storage Events Read",
       "description": "Storage Events Read",
       "tags": [
     
       ],
       "statementQuery": "ALLOW storage:events:read;"

  3. Create a storage logs read policy Make a POST call to IAM

     Body payload for the policy is:

     jsoncopydownload
     {
         "name": "Storage Logs Read",
         "description": "Storage Logs Read",
         "tags": [
     
       ]  ,
         "statementQuery": "ALLOW storage:logs:read;"
      }

  Your newly created policies will be visible on the account level. To check it, go to the user menu and select Account settings > Identity management > Policy management > Edit Storage Events Read.

Upgrade to Log Monitoring Classic

If you are using the legacy Log Monitoring v1, and you would like to use Log Monitoring Classic, you first need to enable it.

What changes after activation

After enabling Log Monitoring Classic, the following changes take place:

• No data will be copied from Log Monitoring v1 storage.
  Switching to the latest version of Dynatrace log monitoring will reset all of your Log Monitoring v1 data. Log Monitoring v1 data won't be accessible.
• You will enable new OneAgent log module.
  The latest version of Dynatrace log monitoring works properly with OneAgent 1.217 or later. Earlier OneAgent versions are not supported.
• You will enable a limit of:
  • 1,000,000 events per minute for SaaS
  • 10,000 events per minute for Managed deployments
  Contact Dynatrace ONE to change this limit.
• Your Log Monitoring v1 event definitions won't be accessible.
• Your Log Monitoring v1 metrics configuration won't be accessible.

What does not change after activation

After enabling Log Monitoring Classic, the following will not change:

• Log sources configuration persists and can be used in a newer log solution.
• Licensing for Log Monitoring Classic is similarly based on the DDU model counting all incoming log records (see more in DDUs for Log Monitoring Classic).

Dynatrace SaaS

To enable the latest version of Log Monitoring in Dynatrace SaaS

1. Make sure that you're running the latest version of Dynatrace.
2. In the Dynatrace menu, go to Logs.
3. Select Start using Log Monitoring v2 and confirm your choice.

Dynatrace Managed

To enable the latest Dynatrace Log Monitoring for a particular Dynatrace Managed environment, you can use an API call or the Cluster Management Console (CMC).

Using an API call

To enable the latest version of Log Monitoring in Dynatrace Managed using an API call

1. Make sure that you're running the latest version of Dynatrace and OneAgent version 1.217+.
2. Make sure that Managed cluster nodes follow the hardware recommendations for Log Monitoring.
3. Configure your Environment ActiveGate to enable Generic Ingestion: Generic log ingestion (Logs Classic)
4. Enable the latest version of Log Monitoring on a specific environment via API: Enable Log Monitoring

Using Cluster Management Console (CMC)

To enable the latest version of Log Monitoring in Dynatrace Managed using the Cluster Management Console (CMC)

1. In the CMC, select Environments and the environment for which you want to enable the latest version of Log Monitoring.
2. On the environment details page, in the Capability settings section, select Start using Log Monitoring v2.