Set up Dynatrace SaaS for AWS monitoring

You can integrate Dynatrace with Amazon Web Services (AWS) for intelligent monitoring of services running in the Amazon Cloud. AWS integration helps you stay on top of the dynamics of your data center in the cloud.

Overview

Follow these basic steps to integrate Dynatrace SaaS with Amazon Web Services (AWS):

1. Create AWS monitoring policy.

2. Choose an access method:

   • Key-based access.
   • Role-based access.
     If you choose role-based access, use the appropriate procedure for your deployment scenario:
     • Role-based access for SaaS deployments with Environment ActiveGate.
     • Role-based access for SaaS deployments with NO Environment ActiveGate.

   You must install and configure an Environment ActiveGate if you want to monitor either or both of the following:

   • More than 2,000 AWS resources (AWS service instances)
   • AWS Cloud Services

3. Select your AWS partition

4. Adjust monitoring to your needs

AWS costs

Dynatrace makes Amazon API requests every 5 minutes. In addition to CloudWatch API calls, Dynatrace makes API calls to the monitored AWS services in order to learn about their instances, tags, etc. The list of called services and actions is available below in the Create the monitoring policy section. Here's a rough estimate of AWS monitoring costs:

AWS monitoring policy

The AWS monitoring policy defines the minimum scope of permissions you need to give to Dynatrace to monitor the services running in your AWS account. Create it once and use anytime when enabling Dynatrace access to your AWS account.

1. Go to Identity and Access Management (IAM) in your Amazon Console.

2. Go to Policies and select Create policy.

3. Select the JSON tab, and paste the predefined policy from the box below.

   Predefined policy in JSON

   jsoncopydownload
   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Sid": "VisualEditor0",
         "Effect": "Allow",
         "Action": [
           "acm-pca:ListCertificateAuthorities",
           "apigateway:GET",
           "apprunner:ListServices",
           "appstream:DescribeFleets",
           "appsync:ListGraphqlApis",
           "athena:ListWorkGroups",
           "autoscaling:DescribeAutoScalingGroups",
           "cloudformation:ListStackResources",
           "cloudfront:ListDistributions",
           "cloudhsm:DescribeClusters",
           "cloudsearch:DescribeDomains",
           "cloudwatch:GetMetricData",
           "cloudwatch:GetMetricStatistics",
           "cloudwatch:ListMetrics",
           "codebuild:ListProjects",
           "datasync:ListTasks",
           "dax:DescribeClusters",
           "directconnect:DescribeConnections",
           "dms:DescribeReplicationInstances",
           "dynamodb:ListTables",
           "dynamodb:ListTagsOfResource",
           "ec2:DescribeAvailabilityZones",
           "ec2:DescribeInstances",
           "ec2:DescribeNatGateways",
           "ec2:DescribeSpotFleetRequests",
           "ec2:DescribeTransitGateways",
           "ec2:DescribeVolumes",
           "ec2:DescribeVpnConnections",
           "ecs:ListClusters",
           "eks:ListClusters",
           "elasticache:DescribeCacheClusters",
           "elasticbeanstalk:DescribeEnvironmentResources",
           "elasticbeanstalk:DescribeEnvironments",
           "elasticfilesystem:DescribeFileSystems",
           "elasticloadbalancing:DescribeInstanceHealth",
           "elasticloadbalancing:DescribeListeners",
           "elasticloadbalancing:DescribeLoadBalancers",
           "elasticloadbalancing:DescribeRules",
           "elasticloadbalancing:DescribeTags",
           "elasticloadbalancing:DescribeTargetHealth",
           "elasticmapreduce:ListClusters",
           "elastictranscoder:ListPipelines",
           "es:ListDomainNames",
           "events:ListEventBuses",
           "firehose:ListDeliveryStreams",
           "fsx:DescribeFileSystems",
           "gamelift:ListFleets",
           "glue:GetJobs",
           "inspector:ListAssessmentTemplates",
           "kafka:ListClusters",
           "kinesis:ListStreams",
           "kinesisanalytics:ListApplications",
           "kinesisvideo:ListStreams",
           "lambda:ListFunctions",
           "lambda:ListTags",
           "lex:GetBots",
           "logs:DescribeLogGroups",
           "mediaconnect:ListFlows",
           "mediaconvert:DescribeEndpoints",
           "mediapackage-vod:ListPackagingConfigurations",
           "mediapackage:ListChannels",
           "mediatailor:ListPlaybackConfigurations",
           "opsworks:DescribeStacks",
           "qldb:ListLedgers",
           "rds:DescribeDBClusters",
           "rds:DescribeDBInstances",
           "rds:DescribeEvents",
           "rds:ListTagsForResource",
           "redshift:DescribeClusters",
           "robomaker:ListSimulationJobs",
           "route53:ListHostedZones",
           "route53resolver:ListResolverEndpoints",
           "s3:ListAllMyBuckets",
           "sagemaker:ListEndpoints",
           "sns:ListTopics",
           "sqs:ListQueues",
           "storagegateway:ListGateways",
           "sts:GetCallerIdentity",
           "swf:ListDomains",
           "tag:GetResources",
           "tag:GetTagKeys",
           "transfer:ListServers",
           "workmail:ListOrganizations",
           "workspaces:DescribeWorkspaces"
         ],
         "Resource": "*"
       }
     ]
   }
   

4. Give the policy a name. For example Dynatrace_monitoring_policy. Type it in the Name field.

5. Select Create policy.

Access methods

To get the information required for comprehensive AWS cloud-computing monitoring, Dynatrace needs to identify all the virtualized infrastructure components in your AWS environment and collect performance metrics related to those components. We use this information to understand the context of your applications, services, and hosts. For this to happen, you need to authorize Dynatrace to access your Amazon metrics.

You can enable Dynatrace access to your AWS metrics by either using a private access key (key-based access) or defining a special role for Dynatrace (role-based access). In all the cases, make sure that your Environment ActiveGate has a working connection to AWS. Configure your proxy for ActiveGate, or allow access to *.amazonaws.com in your firewall settings.

As a best practice, use temporary security credentials (IAM roles) instead of access keys, and disable any AWS account root user access keys.

Key-based access

If you decide to use the key-based authentication, remember to rotate the keys periodically. Keep in mind that you need to perform this procedure each time you change the key.

Prerequisites for key-based access

• Rights to create a new AWS user
• Your AWS account ID
• Your Amazon Access key ID and Secret access key

Enable access to your Amazon account using key-based access

Dynatrace can use access keys to make secure REST or Query protocol requests to the AWS service API. You'll need to generate an Access key ID and a Secret access key that Dynatrace can use to get metrics from Amazon Web Services.

1. In your Amazon Console, go to Users and select Add User.

2. Enter a name for the key you want to create (for example, Dynatrace_monitoring_user). In Select AWS access type, select Programmatic access, and then select Next:Permissions.

3. Select Attach existing policies directly and choose the monitoring policy you defined (for example, Dynatrace_monitoring_policy), and then select Next: Review.

4. Review the user details and select Create user.

5. Store the Access Key ID name (AKID) and Secret access key values.
   You can either download the user credentials or copy the credentials displayed online (select Show).

Connect your Amazon account to Dynatrace using key-based access

Once you've granted AWS access to Dynatrace, it's time to connect Dynatrace to your Amazon AWS account.

1. In the Dynatrace menu, go to Settings > Cloud and virtualization > AWS and select Connect new instance.

2. Select Key-based authentication method.

   • Create a name for this connection. This is mandatory. Dynatrace needs this name to identify and display the connection.
   • In the Access key ID field, paste the identifier of the key you created in Amazon for Dynatrace access.
   • In the Secret access key field, paste the value of the key you created in Amazon for Dynatrace access.
   • Select Connect to verify and save the connection.

3. Once the connection is successfully verified and saved, your AWS account will be listed in the Cloud and virtualization settings page. You should soon begin to see AWS cloud monitoring data.

Role-based access

Dynatrace SaaS deployments might vary. Integrating Dynatrace SaaS with AWS is different when deployment includes an Environment ActiveGate. Select the appropriate set up procedure for your Dynatrace SaaS deployment scenario.

Deployments with Environment ActiveGateDeployments with NO Environment ActiveGate

Role-based access for SaaS deployments with Environment ActiveGate

The instructions below apply whether or not the account hosting your ActiveGate is the same as your monitored account.

In a typical setup, you need to create two CloudFormation stacks using CloudFormation templates:

• A CloudFormation stack from the account hosting your ActiveGate, containing the following resources:
  • A role for your Environment ActiveGate hosted in your AWS infrastructure, on an AWS EC2 host.
  • Its attached policy, defining the monitored account permissions.
• A CloudFormation stack from the monitored account, containing the following resources:
  • A dedicated monitoring role for Dynatrace in your AWS account.
  • Its attached policy, defining the Dynatrace authentication permissions to your AWS environment.

Note: To monitor multiple accounts, add all resources to the Resource array in the template in Step 1 and repeat Step 2 to create a stack for each monitored account.

Prerequisites for role-based access, with Environment ActiveGate

• Environment ActiveGate installed on an AWS EC2 host. It must be able to assume a role within your AWS account that allows it to read the Dynatrace monitoring data.

• The ID of the AWS account that hosts the ActiveGate (for example, the account that hosts your Dynatrace components, which in this case is the one hosting Environment ActiveGate).

• The Amazon Web Services monitored account ID (the account you want to monitor).

• The name of the role with which your Environment ActiveGate was started.

• The External ID.

  How to get the External ID

  1. In the Dynatrace menu, go to Settings.
  2. Go to Cloud and virtualization > AWS and select Connect new instance.
  3. Under Authentication method, select Role-based authentication.
  4. Under Your Amazon account ID, select Copy to copy the token (the External ID).

To enable access to your Amazon account using role-based access, follow the steps below.

Create a role for ActiveGate on the account that hosts ActiveGate

Create a monitoring role for Dynatrace on your monitored account

Modify ActiveGate configuration

Create a role for ActiveGate on the account that hosts ActiveGate

1. Create a YAML file and paste the content from github role_based_access_AG_account_template.yml.

   For multiple monitored accounts

   For each account you want to monitor, add a new item to the !Sub array in the Resource section of the template above, in the following format: 'arn:aws:iam::<new_monitored_account_id>:role/<new_monitoring_role_name>'.

   Note: Be sure to replace the placeholders (<new_monitored_account_id>, <new_monitoring_role_name>) with your own values.

2. Create the stack in your Amazon Console or using the CLI.

   In the Amazon Console

   1. In your Amazon Console, go to CloudFormation.
   2. Go to Stacks and create a new stack with new resources.
   3. Select Template is ready, upload the template you created above, and select Next.
   4. In Parameters, for MonitoredAccountID, enter the ID of the account Dynatrace will monitor. Optionally, adapt other parameters as needed.
   5. Enter a name for your stack, and then select Next twice.
   6. Review your configuration, select I acknowledge that AWS CloudFormation might create IAM resources with custom names, and select Create stack.
   Using the CLI

   Run the command below, making sure to replace the parameter values with your actual values.

   Note: You need to remove the angle brackets (< and >).

   bashcopydownload
   aws cloudformation create-stack \
   --capabilities CAPABILITY_NAMED_IAM \
   --stack-name <stack_name> \
   --template-body <file:///home/user/template_file.yaml> \
   --parameters ParameterKey=ActiveGateRoleName,ParameterValue=<role_name> ParameterKey=AssumePolicyName,ParameterValue=<policy_name> ParameterKey=MonitoringRoleName,ParameterValue=<monitoring_role_name> ParameterKey=MonitoredAccountID,ParameterValue=<monitored_account_id>

3. Go to the Amazon EC2 console, right-click an instance hosting your Environment ActiveGate, and select Security > Modify IAM role.

4. Select the role you created at step 1 (for example, Dynatrace_ActiveGate_role), and select Apply.

Create a monitoring role for Dynatrace on your monitored account

After the Dynatrace_ActiveGate_role is created on the account hosting the ActiveGate, create a role for the account to be monitored.

1. Create a YAML file and paste the content from the github role_based_access_monitored_account_template.yml.

2. Create the stack in your Amazon Console or using the CLI.

   In the Amazon Console

   1. In your Amazon Console, go to CloudFormation.

   2. Go to Stacks and create a new stack with new resources.

   3. Select Template is ready, upload the template you created above, and select Next.

   4. In Parameters, enter:

   • The External ID. For details, see Prerequisites
   • The ActiveGateRoleName and the ActiveGateAccountID from the stack created in Step 1.

   Optionally, adapt other parameters as needed.

   5. Enter a name for your stack, and then select Next twice.
   6. Review your configuration, enable I acknowledge that AWS CloudFormation might create IAM resources with custom names, and select Create stack.
   Using the CLI

   Run the command below, making sure to replace the parameter values with your actual values.

   Note: You need to remove the angle brackets (< and >).

   bashcopydownload
   aws cloudformation create-stack \
   --capabilities CAPABILITY_NAMED_IAM \
   --stack-name <stack_name> \
   --template-body <file:///home/user/template_file.yaml> \
   --parameters ParameterKey=ExternalID,ParameterValue=<external_id> ParameterKey=ActiveGateRoleName,ParameterValue=<activegate_role_name> ParameterKey=ActiveGateAccountID,ParameterValue=<activegate_account_id>
   ParameterKey=RoleName,ParameterValue=<role_name> ParameterKey=PolicyName,ParameterValue=<policy_name>

Modify ActiveGate configuration

Starting with ActiveGate version 1.217, AWS monitoring is enabled by default. For configuration details, see Customize ActiveGate properties. The following configuration settings refer to earlier ActiveGate versions.

1. Edit the custom.properties file of your Environment ActiveGate.

2. Set the following properties as below:

   plaintextcopydownload
   [aws_monitoring]
   use_aws_proxy_role = false
   aws_monitoring_enabled = true

   ActiveGate version 1.183 or earlier

   plaintextcopydownload
   [vertical.topology]
   use_aws_proxy_role = false

   plaintextcopydownload
   [aws_monitoring]
   aws_monitoring_enabled = true

   If the ActiveGate is dedicated to AWS monitoring, you must also set the MSGrouter property to false:

   plaintextcopydownload
   [collector]
   MSGrouter = false

   Remove aws_proxy_account and aws_proxy_role properties.

3. Save the file and restart the ActiveGate main service.

Role-based access for SaaS deployments without Environment ActiveGate

To give Dynatrace SaaS role-based monitoring access to your AWS account, you need to create a CloudFormation stack using a CloudFormation template containing the following resources:

• A dedicated monitoring role for Dynatrace in your AWS account.
• Its attached policy, defining the Dynatrace authentication permissions to your AWS environment. For multiple accounts, you need to repeat the following steps for every account that you want to monitor.

Note: You won't be able to monitor AWS Cloud Services without an AWS-hosted Environment ActiveGate.

Prerequisites for role-based access, without Environment ActiveGate

• Your Amazon Web Services account ID

• Rights to assign role-based access to your AWS account

• The External ID.

  How to get the External ID

  1. In the Dynatrace menu, go to Settings.
  2. Go to Cloud and virtualization > AWS and select Connect new instance.
  3. Under Authentication method, select Role-based authentication.
  4. Under Your Amazon account ID, select Copy to copy the token (the External ID).

Create a monitoring role for Dynatrace in your AWS account

1. Create a YAML file and paste the content from the github role_based_access_no_AG_template.yml.

2. Create the stack in your Amazon Console or using the CLI.

   In the Amazon Console

   1. In your Amazon Console, go to CloudFormation.
   2. Go to Stacks and create a new stack with new resources.
   3. Select Template is ready, upload the template you created above, and then select Next.
   4. In Parameters, enter the External ID. For details, see Prerequisites. Optionally, adapt other parameters as needed.
   5. Enter a name for your stack, and then select Next twice.
   6. Review your configuration, enable I acknowledge that AWS CloudFormation might create IAM resources with custom names, and then select Create stack.
   Using the CLI

   Run the command below, making sure to replace the parameter values with your actual values.

   Note: You need to remove the angle brackets (< and >).

   bashcopydownload
   aws cloudformation create-stack \
   --capabilities CAPABILITY_NAMED_IAM \
   --stack-name <stack_name> \
   --template-body <file:///home/user/template_file.yaml> \
   --parameters ParameterKey=ExternalID,ParameterValue=<external_id> ParameterKey=RoleName,ParameterValue=<role_name> ParameterKey=PolicyName,ParameterValue=<policy_name>

Connect your Amazon account to Dynatrace using role-based access

After you've granted AWS access to Dynatrace, it's time to connect Dynatrace to your Amazon AWS account.

1. In Dynatrace, go to Settings > Cloud and virtualization > AWS and select Connect new instance.

2. Enter a name for this connection. If you leave it empty, the name Role will be used on Dynatrace pages to define this connection.

3. Select Role based authentication and enter configuration details:

   • Enter the name of the role you created in Amazon for Dynatrace (for example, Dynatrace_monitoring_role).
   • Enter your Amazon account ID (the account you want Dynatrace to pull metrics from).
   • Select Connect to verify and save the connection.

4. After the connection is verified and saved, your AWS account is listed on the Cloud and virtualization settings page. You should soon begin to see AWS cloud monitoring data.

Select your AWS partition

If your AWS account is on a different partition than the default aws partition, you can select it and Dynatrace will connect to it instead.

To change your AWS partition

1. In the Dynatrace menu, go to Settings and select Cloud and virtualization > AWS.
2. Find the instance where you want to change the partition and select to edit the instance.
3. In the AWS partition list, select your partition.
4. Select Save.

Adjust monitoring to your needs

You can alter the scope and content of your monitoring depending on your preferences by using tags and listing services needed.

Limit monitored resources using tags

We recommend that you limit the scope of your AWS monitoring and reduce the number API calls to Amazon. You can use tagging to limit the AWS resources (AWS service instances) that are monitored by Dynatrace.

Set up metric events for alerting

To configure metric events for alerting

1. In the Dynatrace menu, go to Settings > Cloud and virtualization > AWS.
2. Under Metric events for alerting, select Manage alerting rules.
3. On the Metric events for alerting page, you can create, enable/disable, and configure recommended alerting rules.

For an overview of all recommended alerting rules for all supporting services, see the list below.

1. Create and enable alerting rules.

   To enable recommended alerting rules, you first need to create them. You can create alerting rules and automatically enable them, or (if you clear Automatically enable created rules) create them and manually enable them after possible configuration changes.

   

   For example, you can create and automatically enable a first batch of alerts. When you start monitoring new services, you can create alerts for these new services without automatically enabling them (because you want to configure them first).

2. Configure alerting rules. How you edit rules depends on whether you chose to automatically enable alerts.

   • If you chose to automatically enable alerts when creating them, go to Adjust recommended alerting rules, expand Enabled recommended alerting rules, and select any rule. This takes you to Edit custom event for alerting, where you can change the configuration rules for that specific service.

     

   • If you didn't choose to automatically enable alerts when creating them, go to Enable recommended alerting rules, expand Disabled recommended alerting rules, and select any of the disabled rules. This takes you to the same Edit custom event for alerting page.

     

3. Disable alerting rules.

4. You can disable all alerting rules, or disable or delete them selectively.

   

   • To disable all alerting rules, go to Adjust recommended alerting rules and select Disable all enabled recommended alerting rules.
   • To disable or delete alerting rules selectively, go to Adjust recommended alerting rules and select Custom events for alerting. On the Custom events for alerting page, you can disable an alert by turning it off in the On/Off column, or you can delete it by selecting x in the Delete column.

   

   Note: If you disable any or all of the alerting rules, you can always re-enable them.

   

Choose Cloud services to be monitored

ActiveGate version 1.245+

Once your credentials are saved, you can decide which services will be monitored. To select your preferred services

1. In the Dynatrace menu, go to Settings and select Cloud and virtualization > AWS.
2. Find the instance where you want to perform your monitoring and select to edit the instance.
3. In the Services section, select Manage services.
4. The following services are added by default: Amazon EC2, Amazon Lambda, Amazon RDS, Amazon DynamoDB, Amazon ALB, Amazon ELB, Amazon S3, and Amazon EBS. You can extend this list by selecting Add service and choosing services from the dropdown menu. The full list of services is also available at AWS cloud services.
5. Select Add service and Save changes.