• Home
  • Deploy Dynatrace
  • Set up Dynatrace on cloud platforms
  • Amazon Web Services
  • Integrations
  • Amazon Web Services
  • Amazon MSK (Kafka) monitoring

Amazon MSK (Kafka) monitoring

Dynatrace ingests metrics for multiple preselected namespaces, including Amazon MSK (Kafka). You can view metrics for each service instance, split metrics into multiple dimensions, and create custom charts that you can pin to your dashboards.

Prerequisites

To enable monitoring for this service, you need

  • ActiveGate version 1.197+, as follows:
    • For Dynatrace SaaS deployments, you need an Environment ActiveGate or a Multi-environment ActiveGate.
    • For Dynatrace Managed deployments, you can use any kind of ActiveGate.
      Note: For role-based access (whether in a SaaS or Managed deployment), you need an Environment ActiveGate installed on an Amazon EC2 host.
  • Dynatrace version 1.203+
  • An updated AWS monitoring policy to include the additional AWS services.
    To update the AWS IAM policy, use the JSON below, containing the monitoring policy (permissions) for all supporting services.
JSON predefined policy for all supporting services
json
{ "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": [ "acm-pca:ListCertificateAuthorities", "apigateway:GET", "apprunner:ListServices", "appstream:DescribeFleets", "appsync:ListGraphqlApis", "athena:ListWorkGroups", "autoscaling:DescribeAutoScalingGroups", "cloudformation:ListStackResources", "cloudfront:ListDistributions", "cloudhsm:DescribeClusters", "cloudsearch:DescribeDomains", "cloudwatch:GetMetricData", "cloudwatch:GetMetricStatistics", "cloudwatch:ListMetrics", "codebuild:ListProjects", "datasync:ListTasks", "dax:DescribeClusters", "directconnect:DescribeConnections", "dms:DescribeReplicationInstances", "dynamodb:ListTables", "dynamodb:ListTagsOfResource", "ec2:DescribeAvailabilityZones", "ec2:DescribeInstances", "ec2:DescribeNatGateways", "ec2:DescribeSpotFleetRequests", "ec2:DescribeTransitGateways", "ec2:DescribeVolumes", "ec2:DescribeVpnConnections", "ecs:ListClusters", "eks:ListClusters", "elasticache:DescribeCacheClusters", "elasticbeanstalk:DescribeEnvironmentResources", "elasticbeanstalk:DescribeEnvironments", "elasticfilesystem:DescribeFileSystems", "elasticloadbalancing:DescribeInstanceHealth", "elasticloadbalancing:DescribeListeners", "elasticloadbalancing:DescribeLoadBalancers", "elasticloadbalancing:DescribeRules", "elasticloadbalancing:DescribeTags", "elasticloadbalancing:DescribeTargetHealth", "elasticmapreduce:ListClusters", "elastictranscoder:ListPipelines", "es:ListDomainNames", "events:ListEventBuses", "firehose:ListDeliveryStreams", "fsx:DescribeFileSystems", "gamelift:ListFleets", "glue:GetJobs", "inspector:ListAssessmentTemplates", "kafka:ListClusters", "kinesis:ListStreams", "kinesisanalytics:ListApplications", "kinesisvideo:ListStreams", "lambda:ListFunctions", "lambda:ListTags", "lex:GetBots", "logs:DescribeLogGroups", "mediaconnect:ListFlows", "mediaconvert:DescribeEndpoints", "mediapackage-vod:ListPackagingConfigurations", "mediapackage:ListChannels", "mediatailor:ListPlaybackConfigurations", "opsworks:DescribeStacks", "qldb:ListLedgers", "rds:DescribeDBClusters", "rds:DescribeDBInstances", "rds:DescribeEvents", "rds:ListTagsForResource", "redshift:DescribeClusters", "robomaker:ListSimulationJobs", "route53:ListHostedZones", "route53resolver:ListResolverEndpoints", "s3:ListAllMyBuckets", "sagemaker:ListEndpoints", "sns:ListTopics", "sqs:ListQueues", "storagegateway:ListGateways", "sts:GetCallerIdentity", "swf:ListDomains", "tag:GetResources", "tag:GetTagKeys", "transfer:ListServers", "workmail:ListOrganizations", "workspaces:DescribeWorkspaces" ], "Resource": "*" } ] }

If you don't want to add permissions to all services, and just select permissions for certain services, consult the table below. The table contains a set of permissions that are required for all services (All monitored Amazon services) and, for each supporting service, a list of optional permissions specific to that service.

Complete list of permissions for cloud services
NameAdditional permissions
AWS Certificate Manager Private Certificate Authority"acm-pca:ListCertificateAuthorities"
All monitored Amazon services"cloudwatch:GetMetricData",
"cloudwatch:GetMetricStatistics",
"cloudwatch:ListMetrics",
"sts:GetCallerIdentity",
"tag:GetResources",
"tag:GetTagKeys",
"ec2:DescribeAvailabilityZones"
Amazon MQ
Amazon API Gateway"apigateway:GET"
AWS App Runner"apprunner:ListServices"
Amazon AppStream"appstream:DescribeFleets"
AWS AppSync"appsync:ListGraphqlApis"
Amazon Athena"athena:ListWorkGroups"
Amazon Aurora"rds:DescribeDBClusters"
Amazon EC2 Auto Scaling"autoscaling:DescribeAutoScalingGroups"
Amazon EC2 Auto Scaling (built-in)"autoscaling:DescribeAutoScalingGroups"
AWS Billing
Amazon Keyspaces
AWS Chatbot
Amazon CloudFront"cloudfront:ListDistributions"
AWS CloudHSM"cloudhsm:DescribeClusters"
Amazon CloudSearch"cloudsearch:DescribeDomains"
AWS CodeBuild"codebuild:ListProjects"
Amazon Cognito
Amazon Connect
Amazon Elastic Kubernetes Service (EKS)"eks:ListClusters"
AWS DataSync"datasync:ListTasks"
Amazon DynamoDB Accelerator (DAX)"dax:DescribeClusters"
Amazon Database Migration Service"dms:DescribeReplicationInstances"
Amazon DocumentDB"rds:DescribeDBClusters"
AWS Direct Connect"directconnect:DescribeConnections"
Amazon DynamoDB (built-in)"dynamodb:ListTables",
"dynamodb:ListTagsOfResource"
Amazon EBS (built-in)"ec2:DescribeVolumes"
Amazon EC2 API
Amazon EC2 (built-in)"ec2:DescribeInstances"
Amazon EC2 Spot Fleet"ec2:DescribeSpotFleetRequests"
Amazon Elastic Container Service (ECS)"ecs:ListClusters"
Amazon ECS ContainerInsights"ecs:ListClusters"
Amazon ElastiCache (EC)"elasticache:DescribeCacheClusters"
AWS Elastic Beanstalk"elasticbeanstalk:DescribeEnvironments"
Amazon Elastic File System (EFS)"elasticfilesystem:DescribeFileSystems"
Amazon Elastic Inference
Amazon Elastic Map Reduce (EMR)"elasticmapreduce:ListClusters"
Amazon Elasticsearch Service (ES)"es:ListDomainNames"
Amazon Elastic Transcoder"elastictranscoder:ListPipelines"
AWS Elastic Load Balancing (ELB) (built-in)"elasticloadbalancing:DescribeInstanceHealth",
"elasticloadbalancing:DescribeListeners",
"elasticloadbalancing:DescribeLoadBalancers",
"elasticloadbalancing:DescribeRules",
"elasticloadbalancing:DescribeTags",
"elasticloadbalancing:DescribeTargetHealth"
Amazon EventBridge"events:ListEventBuses"
Amazon FSx"fsx:DescribeFileSystems"
Amazon GameLift"gamelift:ListFleets"
AWS Glue"glue:GetJobs"
Amazon Inspector"inspector:ListAssessmentTemplates"
AWS Internet of Things (IoT)
AWS IoT Analytics
Amazon Managed Streaming for Kafka"kafka:ListClusters"
Amazon Kinesis Data Analytics"kinesisanalytics:ListApplications"
Amazon Kinesis Data Firehose"firehose:ListDeliveryStreams"
Amazon Kinesis Data Streams"kinesis:ListStreams"
Amazon Kinesis Video Streams"kinesisvideo:ListStreams"
AWS Lambda (built-in)"lambda:ListFunctions",
"lambda:ListTags"
Amazon Lex"lex:GetBots"
AWS Application and Network Load Balancer (built-in)"elasticloadbalancing:DescribeInstanceHealth",
"elasticloadbalancing:DescribeListeners",
"elasticloadbalancing:DescribeLoadBalancers",
"elasticloadbalancing:DescribeRules",
"elasticloadbalancing:DescribeTags",
"elasticloadbalancing:DescribeTargetHealth"
Amazon CloudWatch Logs"logs:DescribeLogGroups"
AWS Elemental MediaConnect"mediaconnect:ListFlows"
Amazon MediaConvert"mediaconvert:DescribeEndpoints"
Amazon MediaPackage Live"mediapackage:ListChannels"
Amazon MediaPackage Video on Demand"mediapackage-vod:ListPackagingConfigurations"
Amazon MediaTailor"mediatailor:ListPlaybackConfigurations"
Amazon VPC NAT Gateways"ec2:DescribeNatGateways"
Amazon Neptune"rds:DescribeDBClusters"
AWS OpsWorks"opsworks:DescribeStacks"
Amazon Polly
Amazon QLDB"qldb:ListLedgers"
Amazon RDS (built-in)"rds:DescribeDBInstances",
"rds:DescribeEvents",
"rds:ListTagsForResource"
Amazon Redshift"redshift:DescribeClusters"
Amazon Rekognition
AWS RoboMaker"robomaker:ListSimulationJobs"
Amazon Route 53"route53:ListHostedZones"
Amazon Route 53 Resolver"route53resolver:ListResolverEndpoints"
Amazon S3"s3:ListAllMyBuckets"
Amazon S3 (built-in)"s3:ListAllMyBuckets"
Amazon SageMaker Batch Transform Jobs
Amazon SageMaker Endpoint Instances"sagemaker:ListEndpoints"
Amazon SageMaker Endpoints"sagemaker:ListEndpoints"
Amazon SageMaker Ground Truth
Amazon SageMaker Processing Jobs
Amazon SageMaker Training Jobs
AWS Service Catalog
Amazon Simple Email Service (SES)
Amazon Simple Notification Service (SNS)"sns:ListTopics"
Amazon Simple Queue Service (SQS)"sqs:ListQueues"
AWS Systems Manager - Run Command
AWS Step Functions
AWS Storage Gateway"storagegateway:ListGateways"
Amazon SWF"swf:ListDomains"
Amazon Textract
AWS IoT Things Graph
Amazon Transfer Family"transfer:ListServers"
AWS Transit Gateway"ec2:DescribeTransitGateways"
Amazon Translate
AWS Trusted Advisor
AWS API Usage
AWS Site-to-Site VPN"ec2:DescribeVpnConnections"
Amazon WAF Classic
Amazon WAF
Amazon WorkMail"workmail:ListOrganizations"
Amazon WorkSpaces"workspaces:DescribeWorkspaces"

Example of JSON policy for one single service.

JSON policy for Amazon API Gateway
json
{ "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": [ "apigateway:GET", "cloudwatch:GetMetricData", "cloudwatch:GetMetricStatistics", "cloudwatch:ListMetrics", "sts:GetCallerIdentity", "tag:GetResources", "tag:GetTagKeys", "ec2:DescribeAvailabilityZones" ], "Resource": "*" } ] }

In this example, from the complete list of permissions you need to select

  • "apigateway:GET" for Amazon API Gateway
  • "cloudwatch:GetMetricData", "cloudwatch:GetMetricStatistics", "cloudwatch:ListMetrics", "sts:GetCallerIdentity", "tag:GetResources", "tag:GetTagKeys", and "ec2:DescribeAvailabilityZones" for All monitored Amazon services.
AWS endpoints that need to be reachable from ActiveGate with corresponding AWS services
EndpointService

acm-pca.<REGION>.amazonaws.com

AWS Certificate Manager Private Certificate Authority

apigateway.<REGION>.amazonaws.com

Amazon API Gateway

apprunner.<REGION>.amazonaws.com

AWS App Runner

appstream2.<REGION>.amazonaws.com

Amazon AppStream

appsync.<REGION>.amazonaws.com

AWS AppSync

athena.<REGION>.amazonaws.com

Amazon Athena

rds.<REGION>.amazonaws.com

Amazon Aurora, Amazon DocumentDB, Amazon Neptune, Amazon RDS (built-in)

autoscaling.<REGION>.amazonaws.com

Amazon EC2 Auto Scaling, Amazon EC2 Auto Scaling (built-in)

cloudfront.amazonaws.com

Amazon CloudFront

cloudhsmv2.<REGION>.amazonaws.com

AWS CloudHSM

cloudsearch.<REGION>.amazonaws.com

Amazon CloudSearch

cloudsearch.<REGION>.amazonaws.com

Amazon CloudSearch

codebuild.<REGION>.amazonaws.com

AWS CodeBuild

eks.<REGION>.amazonaws.com

Amazon Elastic Kubernetes Service (EKS)

datasync.<REGION>.amazonaws.com

AWS DataSync

dax.<REGION>.amazonaws.com

Amazon DynamoDB Accelerator (DAX)

dms.<REGION>.amazonaws.com

Amazon Database Migration Service

directconnect.<REGION>.amazonaws.com

AWS Direct Connect

dynamodb.<REGION>.amazonaws.com

Amazon DynamoDB (built-in)

ec2.<REGION>.amazonaws.com

Amazon EBS (built-in), Amazon EC2 (built-in), Amazon EC2 Spot Fleet, Amazon VPC NAT Gateways, AWS Transit Gateway, AWS Site-to-Site VPN

ecs.<REGION>.amazonaws.com

Amazon Elastic Container Service (ECS), Amazon ECS ContainerInsights

elasticache.<REGION>.amazonaws.com

Amazon ElastiCache (EC)

elasticbeanstalk.<REGION>.amazonaws.com

AWS Elastic Beanstalk, AWS Elastic Beanstalk (built-in)

elasticfilesystem.<REGION>.amazonaws.com

Amazon Elastic File System (EFS)

es.<REGION>.amazonaws.com

Amazon Elasticsearch Service (ES)

elastictranscoder.<REGION>.amazonaws.com

Amazon Elastic Transcoder

elasticloadbalancing.<REGION>.amazonaws.com

AWS Elastic Load Balancing (ELB) (built-in), AWS Application and Network Load Balancer (built-in)

events.<REGION>.amazonaws.com

Amazon EventBridge

elastictranscoder.<REGION>.amazonaws.com

Amazon Elastic Transcoder

elasticloadbalancing.<REGION>.amazonaws.com

AWS Elastic Load Balancing (ELB) (built-in), AWS Application and Network Load Balancer (built-in)

events.<REGION>.amazonaws.com

AWS Elastic Load Balancing (ELB) (built-in), AWS Application and Network Load Balancer (built-in)

fsx.<REGION>.amazonaws.com

Amazon FSx

gamelift.<REGION>.amazonaws.com

Amazon GameLift

glue.<REGION>.amazonaws.com

AWS Glue

inspector.<REGION>.amazonaws.com

Amazon Inspector

kafka.<REGION>.amazonaws.com

Amazon Managed Streaming for Kafka

lambda.<REGION>.amazonaws.com

AWS Lambda (built-in)

models.lex.<REGION>.amazonaws.com

Amazon Lex

logs.<REGION>.amazonaws.com

Amazon CloudWatch Logs

mediaconnect.<REGION>.amazonaws.com

AWS Elemental MediaConnect

mediapackage.<REGION>.amazonaws.com

Amazon MediaPackage Live

mediapackage-vod.<REGION>.amazonaws.com

Amazon MediaPackage Video on Demand

api.mediatailor.<REGION>.amazonaws.com

Amazon MediaTailor

opsworks.<REGION>.amazonaws.com

AWS OpsWorks

qldb.<REGION>.amazonaws.com

Amazon QLDB

redshift.<REGION>.amazonaws.com

Amazon Redshift

robomaker.<REGION>.amazonaws.com

Amazon Redshift

route53.amazonaws.com

Amazon Route 53

route53resolver.<REGION>.amazonaws.com

Amazon Route 53 Resolver

s3.<REGION>.amazonaws.com

Amazon S3 (built-in)

api.sagemaker.<REGION>.amazonaws.com

Amazon SageMaker Endpoint Instances, Amazon SageMaker Endpoints

sns.<REGION>.amazonaws.com

Amazon S3 (built-in)

sqs.<REGION>.amazonaws.com

Amazon Simple Queue Service (SQS)

storagegateway.<REGION>.amazonaws.com

AWS Storage Gateway

swf.<REGION>.amazonaws.com

Amazon SWF

transfer.<REGION>.amazonaws.com

Amazon Transfer Family

workmail.<REGION>.amazonaws.com

Amazon WorkMail

workspaces.<REGION>.amazonaws.com

Amazon WorkSpaces

Enable monitoring

To enable monitoring for this service, you first need to integrate Dynatrace with Amazon Web Services:

  • Set up Dynatrace SaaS integration
  • Set up Dynatrace Managed integration

Add the service to monitoring

In order to view the service metrics, you must add the service to monitoring in your Dynatrace environment.

To add a service to monitoring
  1. In the Dynatrace menu, go to Settings > Cloud and virtualization and select AWS.
  2. On the AWS overview page, scroll down and select the desired AWS instance. Select the Edit button.
  3. Scroll down and select Add service. Choose the service name from the drop-down and select Add service.
  4. Select Save changes.
Cloud-service monitoring consumption

All cloud services consume Davis data units (DDUs). The amount of DDU consumption per service instance depends on the number of monitored metrics and their dimensions (each metric dimension results in the ingestion of 1 data point; 1 data point consumes 0.001 DDUs).

Monitor resources based on tags

You can choose to monitor resources based on existing AWS tags, as Dynatrace automatically imports them from service instances. Nevertheless, the transition from AWS to Dynatrace tagging isn't supported for all AWS services. Expand the table below to see which cloud services are filtered by tagging.

Tag filtering per service
NameTags monitoring & filtering
AWS Certificate Manager Private Certificate Authorityyes
Amazon MQ-
Amazon API Gatewayyes
AWS App Runneryes
Amazon AppStreamyes
AWS AppSyncyes
Amazon Athenayes
Amazon Aurorayes
Amazon EC2 Auto Scaling-
Amazon EC2 Auto Scaling (built-in)yes
AWS Billing-
Amazon Keyspacesyes
AWS Chatbot-
Amazon CloudFrontyes
AWS CloudHSMyes
Amazon CloudSearch-
AWS CodeBuildyes
Amazon Cognito-
Amazon Connect-
Amazon Elastic Kubernetes Service (EKS)yes
AWS DataSyncyes
Amazon DynamoDB Accelerator (DAX)yes
Amazon Database Migration Serviceyes
Amazon DocumentDByes
AWS Direct Connectyes
Amazon DynamoDB (built-in)yes
Amazon EBS (built-in)yes
Amazon EC2 API-
Amazon EC2 (built-in)yes
Amazon EC2 Spot Fleet-
Amazon Elastic Container Service (ECS)yes
Amazon ECS ContainerInsightsyes
Amazon ElastiCache (EC)yes
AWS Elastic Beanstalkyes
Amazon Elastic File System (EFS)yes
Amazon Elastic Inferenceyes
Amazon Elastic Map Reduce (EMR)yes
Amazon Elasticsearch Service (ES)yes
Amazon Elastic Transcoder-
AWS Elastic Load Balancing (ELB) (built-in)yes
Amazon EventBridgeyes
Amazon FSxyes
Amazon GameLift-
AWS Glueyes
Amazon Inspectoryes
AWS Internet of Things (IoT)-
AWS IoT Analytics-
Amazon Managed Streaming for Kafkayes
Amazon Kinesis Data Analyticsyes
Amazon Kinesis Data Firehoseyes
Amazon Kinesis Data Streamsyes
Amazon Kinesis Video Streamsyes
AWS Lambda (built-in)yes
Amazon Lexyes
AWS Application and Network Load Balancer (built-in)yes
Amazon CloudWatch Logsyes
AWS Elemental MediaConnect-
Amazon MediaConvertyes
Amazon MediaPackage Liveyes
Amazon MediaPackage Video on Demandyes
Amazon MediaTailoryes
Amazon VPC NAT Gatewaysyes
Amazon Neptuneyes
AWS OpsWorksyes
Amazon Polly-
Amazon QLDByes
Amazon RDS (built-in)yes
Amazon Redshiftyes
Amazon Rekognition-
AWS RoboMakeryes
Amazon Route 53-
Amazon Route 53 Resolveryes
Amazon S3yes
Amazon S3 (built-in)yes
Amazon SageMaker Batch Transform Jobs-
Amazon SageMaker Endpoint Instancesyes
Amazon SageMaker Endpointsyes
Amazon SageMaker Ground Truth-
Amazon SageMaker Processing Jobs-
Amazon SageMaker Training Jobs-
AWS Service Catalog-
Amazon Simple Email Service (SES)-
Amazon Simple Notification Service (SNS)yes
Amazon Simple Queue Service (SQS)yes
AWS Systems Manager - Run Command-
AWS Step Functions-
AWS Storage Gatewayyes
Amazon SWF-
Amazon Textract-
AWS IoT Things Graph-
Amazon Transfer Familyyes
AWS Transit Gatewayyes
Amazon Translate-
AWS Trusted Advisor-
AWS API Usage-
AWS Site-to-Site VPNyes
Amazon WAF Classic-
Amazon WAF-
Amazon WorkMailyes
Amazon WorkSpacesyes

To monitor resources based on tags

  1. In the Dynatrace menu, go to Settings > Cloud and virtualization > AWS and select Edit for the desired AWS instance.
  2. For Resources to be monitored, select Monitor resources selected by tags.
  3. Enter the Key and Value.
  4. Select Save.

Configure service metrics

Once you add a service, Dynatrace starts automatically collecting a suite of metrics for this particular service. These are recommended metrics.

Recommended metrics:

  • Are enabled by default
  • Can't be disabled
  • Can have recommended dimensions (enabled by default, can't be disabled)
  • Can have optional dimensions (disabled by default, can be enabled)

Apart from the recommended metrics, most services have the possibility of enabling optional metrics.

Optional metrics:

  • Can be added and configured manually
To add and configure metrics
  1. In the Dynatrace menu, go to Settings > Cloud and virtualization and select AWS.
  2. On the AWS overview page, scroll down and select Edit for the desired AWS instance.
  3. Scroll down to the Services section and select Manage services.
  4. To add a metric, select the service for which you want to add metrics.
  5. Select Add new metric.
  6. From the menu, select the metric you want.
  7. Select Add metric to add the metric to monitoring.
  8. To configure a metric, select Edit.
  9. Select Apply to save your configuration.

View service metrics

You can view the service metrics in your Dynatrace environment either on the custom device overview page or on your Dashboards page.

View metrics on the custom device overview page

To access the custom device overview page

  1. In the Dynatrace menu, go to Technologies and processes.
  2. Filter by service name and select the relevant custom device group.
  3. Once you select the custom device group, you're on the custom device group overview page.
  4. The custom device group overview page lists all instances (custom devices) belonging to the group. Select an instance to view the custom device overview page.

View metrics on your dashboard

After you add the service to monitoring, a preset dashboard containing all recommended metrics is automatically listed on your Dashboards page. To look for specific dashboards, filter by Preset and then by Name.

AWS presets

Note: For existing monitored services, you might need to resave your credentials for the preset dashboard to appear on the Dashboards page. To resave your credentials, go to Settings > Cloud and virtualization > AWS, select the desired AWS instance, and then select Save.

You can't make changes on a preset dashboard directly, but you can clone and edit it. To clone a dashboard, open the browse menu (…) and select Clone.

To remove a dashboard from the dashboards page, you can hide it. To hide a dashboard, open the browse menu (…) and select Hide.

Note: Hiding a dashboard doesn't affect other users.

Clone hide AWS

To check the availability of preset dashboards for each AWS service, see the list below.

Preset dashboard availability list
AWS servicePreset dashboard
AWS Certificate Manager Private Certificate Authorityno
Amazon MQyes
Amazon API Gatewayno
AWS App Runnerno
Amazon AppStreamyes
AWS AppSyncyes
Amazon Athenayes
Amazon Aurorano
Amazon EC2 Auto Scalingyes
Amazon EC2 Auto Scaling (built-in)no
AWS Billingyes
Amazon Keyspacesyes
AWS Chatbotyes
Amazon CloudFrontno
AWS CloudHSMyes
Amazon CloudSearchyes
AWS CodeBuildyes
Amazon Cognitono
Amazon Connectyes
Amazon Elastic Kubernetes Service (EKS)yes
AWS DataSyncyes
Amazon DynamoDB Accelerator (DAX)yes
Amazon Database Migration Serviceyes
Amazon DocumentDByes
AWS Direct Connectyes
Amazon DynamoDB (built-in)no
Amazon EBS (built-in)no
Amazon EC2 APIyes
Amazon EC2 (built-in)no
Amazon EC2 Spot Fleetno
Amazon Elastic Container Service (ECS)no
Amazon ECS ContainerInsightsyes
Amazon ElastiCache (EC)no
AWS Elastic Beanstalkyes
Amazon Elastic File System (EFS)no
Amazon Elastic Inferenceyes
Amazon Elastic Map Reduce (EMR)no
Amazon Elasticsearch Service (ES)no
Amazon Elastic Transcoderyes
AWS Elastic Load Balancing (ELB) (built-in)no
Amazon EventBridgeyes
Amazon FSxyes
Amazon GameLiftyes
AWS Glueno
Amazon Inspectoryes
AWS Internet of Things (IoT)no
AWS IoT Analyticsyes
Amazon Managed Streaming for Kafkayes
Amazon Kinesis Data Analyticsno
Amazon Kinesis Data Firehoseno
Amazon Kinesis Data Streamsno
Amazon Kinesis Video Streamsno
AWS Lambda (built-in)no
Amazon Lexyes
AWS Application and Network Load Balancer (built-in)no
Amazon CloudWatch Logsyes
AWS Elemental MediaConnectyes
Amazon MediaConvertyes
Amazon MediaPackage Liveyes
Amazon MediaPackage Video on Demandyes
Amazon MediaTailoryes
Amazon VPC NAT Gatewaysno
Amazon Neptuneyes
AWS OpsWorksyes
Amazon Pollyyes
Amazon QLDByes
Amazon RDS (built-in)no
Amazon Redshiftno
Amazon Rekognitionyes
AWS RoboMakeryes
Amazon Route 53yes
Amazon Route 53 Resolveryes
Amazon S3no
Amazon S3 (built-in)no
Amazon SageMaker Batch Transform Jobsno
Amazon SageMaker Endpoint Instancesno
Amazon SageMaker Endpointsno
Amazon SageMaker Ground Truthno
Amazon SageMaker Processing Jobsno
Amazon SageMaker Training Jobsno
AWS Service Catalogyes
Amazon Simple Email Service (SES)no
Amazon Simple Notification Service (SNS)no
Amazon Simple Queue Service (SQS)no
AWS Systems Manager - Run Commandyes
AWS Step Functionsyes
AWS Storage Gatewayyes
Amazon SWFyes
Amazon Textractyes
AWS IoT Things Graphyes
Amazon Transfer Familyyes
AWS Transit Gatewayyes
Amazon Translateyes
AWS Trusted Advisoryes
AWS API Usageyes
AWS Site-to-Site VPNyes
Amazon WAF Classicyes
Amazon WAFyes
Amazon WorkMailyes
Amazon WorkSpacesyes

Msk

Available metrics

NameDescriptionUnitStatisticsDimensionsRecommended
ActiveControllerCountOnly one controller per cluster should be active at any given time.CountMultiCluster Name✔️
ActiveControllerCountCountSumCluster Name✔️
BytesInPerSecThe number of bytes per second received from clientsBytes/SecondMultiCluster Name, Broker ID
BytesInPerSecBytes/SecondMultiCluster Name, Broker ID, Topic
BytesInPerSecBytes/SecondSumCluster Name, Broker ID
BytesInPerSecBytes/SecondSumCluster Name, Broker ID, Topic
BytesOutPerSecThe number of bytes per second sent to clientsBytes/SecondMultiCluster Name, Broker ID
BytesOutPerSecBytes/SecondMultiCluster Name, Broker ID, Topic
BytesOutPerSecBytes/SecondSumCluster Name, Broker ID
BytesOutPerSecBytes/SecondSumCluster Name, Broker ID, Topic
CPUCreditBalanceThe number of earned creditsCountMultiCluster Name, Broker ID
CPUCreditBalanceCountSumCluster Name, Broker ID
CPUCreditUsageThe number of used creditsCountMultiCluster Name, Broker ID
CPUCreditUsageCountSumCluster Name, Broker ID
CpuIdleThe percentage of CPU idle timePercentMultiCluster Name, Broker ID✔️
CpuIdlePercentSumCluster Name, Broker ID✔️
CpuSystemThe percentage of CPU in kernel spacePercentMultiCluster Name, Broker ID✔️
CpuSystemPercentSumCluster Name, Broker ID✔️
CpuUserThe percentage of CPU in user spacePercentMultiCluster Name, Broker ID✔️
CpuUserPercentSumCluster Name, Broker ID✔️
FetchConsumerLocalTimeMsMeanThe mean time in milliseconds that the consumer request is processed at the leaderMillisecondsMultiCluster Name, Broker ID
FetchConsumerLocalTimeMsMeanMillisecondsSumCluster Name, Broker ID
FetchConsumerRequestQueueTimeMsMeanThe mean time in milliseconds that the consumer request waits in the request queueMillisecondsMultiCluster Name, Broker ID
FetchConsumerRequestQueueTimeMsMeanMillisecondsSumCluster Name, Broker ID
FetchConsumerResponseQueueTimeMsMeanThe mean time in milliseconds that the consumer request waits in the response queueMillisecondsMultiCluster Name, Broker ID
FetchConsumerResponseQueueTimeMsMeanMillisecondsSumCluster Name, Broker ID
FetchConsumerResponseSendTimeMsMeanMillisecondsMultiCluster Name, Broker ID
FetchConsumerResponseSendTimeMsMeanMillisecondsSumCluster Name, Broker ID
FetchConsumerTotalTimeMsMeanThe mean total time in milliseconds that consumers spend on fetching data from the brokerMillisecondsMultiCluster Name, Broker ID
FetchConsumerTotalTimeMsMeanMillisecondsSumCluster Name, Broker ID
FetchFollowerLocalTimeMsMeanThe mean time in milliseconds that the follower request is processed at the leaderMillisecondsMultiCluster Name, Broker ID
FetchFollowerLocalTimeMsMeanMillisecondsSumCluster Name, Broker ID
FetchFollowerRequestQueueTimeMsMeanThe mean time in milliseconds that the follower request waits in the request queueMillisecondsMultiCluster Name, Broker ID
FetchFollowerRequestQueueTimeMsMeanMillisecondsSumCluster Name, Broker ID
FetchFollowerResponseQueueTimeMsMeanThe mean time in milliseconds that the follower request waits in the response queueMillisecondsMultiCluster Name, Broker ID
FetchFollowerResponseQueueTimeMsMeanMillisecondsSumCluster Name, Broker ID
FetchFollowerResponseSendTimeMsMeanThe mean time in milliseconds for the follower to send a responseMillisecondsMultiCluster Name, Broker ID
FetchFollowerResponseSendTimeMsMeanMillisecondsSumCluster Name, Broker ID
FetchFollowerTotalTimeMsMeanThe mean total time in milliseconds that followers spend on fetching data from the brokerMillisecondsMultiCluster Name, Broker ID
FetchFollowerTotalTimeMsMeanMillisecondsSumCluster Name, Broker ID
FetchMessageConversionsPerSecThe number of fetch message conversions per second for the brokerCount/SecondMultiCluster Name, Broker ID
FetchMessageConversionsPerSecCount/SecondMultiCluster Name, Broker ID, Topic
FetchMessageConversionsPerSecCount/SecondSumCluster Name, Broker ID
FetchMessageConversionsPerSecCount/SecondSumCluster Name, Broker ID, Topic
FetchMessageConversionsTimeMsMeanThe mean total time in milliseconds that messages being fetched spend convertingMillisecondsMultiCluster Name, Broker ID
FetchMessageConversionsTimeMsMeanMillisecondsSumCluster Name, Broker ID
FetchThrottleByteRateThe number of throttled bytes per secondBytes/SecondMultiCluster Name, Broker ID
FetchThrottleByteRateBytes/SecondSumCluster Name, Broker ID
FetchThrottleQueueSizeThe number of messages in the throttle queueCountMultiCluster Name, Broker ID
FetchThrottleQueueSizeCountSumCluster Name, Broker ID
FetchThrottleTimeThe average fetch throttle time in millisecondsMillisecondsMultiCluster Name, Broker ID
FetchThrottleTimeMillisecondsSumCluster Name, Broker ID
GlobalPartitionCountTotal number of partitions across all brokers in the clusterCountMultiCluster Name✔️
GlobalPartitionCountCountSumCluster Name✔️
GlobalTopicCountTotal number of topics across all brokers in the clusterCountMultiCluster Name✔️
GlobalTopicCountCountSumCluster Name✔️
KafkaAppLogsDiskUsedThe percentage of disk space used for application logsPercentMultiCluster Name, Broker ID✔️
KafkaAppLogsDiskUsedPercentSumCluster Name, Broker ID✔️
KafkaDataLogsDiskUsedThe percentage of disk space used for data logsPercentMultiCluster Name, Broker ID✔️
KafkaDataLogsDiskUsedPercentSumCluster Name, Broker ID✔️
LeaderCountThe number of leader replicasCountMultiCluster Name, Broker ID
LeaderCountCountSumCluster Name, Broker ID
MemoryBufferedThe size in bytes of buffered memory for the brokerBytesMultiCluster Name, Broker ID✔️
MemoryBufferedBytesSumCluster Name, Broker ID✔️
MemoryCachedThe size in bytes of cached memory for the brokerBytesMultiCluster Name, Broker ID✔️
MemoryCachedBytesSumCluster Name, Broker ID✔️
MemoryFreeThe size in bytes of memory that is free and available for the brokerBytesMultiCluster Name, Broker ID✔️
MemoryFreeBytesSumCluster Name, Broker ID✔️
MemoryUsedThe size in bytes of memory that is in use for the brokerBytesMultiCluster Name, Broker ID✔️
MemoryUsedBytesSumCluster Name, Broker ID✔️
MessagesInPerSecThe number of incoming messages per second for the brokerCount/SecondMultiCluster Name, Broker ID
MessagesInPerSecCount/SecondMultiCluster Name, Broker ID, Topic
MessagesInPerSecCount/SecondSumCluster Name, Broker ID
MessagesInPerSecCount/SecondSumCluster Name, Broker ID, Topic
NetworkProcessorAvgIdlePercentThe average percentage of the time the network processors are idlePercentMultiCluster Name, Broker ID
NetworkProcessorAvgIdlePercentPercentSumCluster Name, Broker ID
NetworkRxDroppedThe number of dropped receive packagesCountMultiCluster Name, Broker ID✔️
NetworkRxDroppedCountSumCluster Name, Broker ID✔️
NetworkRxErrorsThe number of network receive errors for the brokerCountMultiCluster Name, Broker ID✔️
NetworkRxErrorsCountSumCluster Name, Broker ID✔️
NetworkRxPacketsThe number of packets received by the brokerCountMultiCluster Name, Broker ID✔️
NetworkRxPacketsCountSumCluster Name, Broker ID✔️
NetworkTxDroppedThe number of dropped transmit packagesCountMultiCluster Name, Broker ID✔️
NetworkTxDroppedCountSumCluster Name, Broker ID✔️
NetworkTxErrorsThe number of network transmit errors for the brokerCountMultiCluster Name, Broker ID✔️
NetworkTxErrorsCountSumCluster Name, Broker ID✔️
NetworkTxPacketsThe number of packets transmitted by the brokerCountMultiCluster Name, Broker ID✔️
NetworkTxPacketsCountSumCluster Name, Broker ID✔️
OfflinePartitionsCountTotal number of partitions that are offline in the clusterCountMultiCluster Name✔️
OfflinePartitionsCountCountSumCluster Name✔️
PartitionCountThe number of partitions for the brokerCountMultiCluster Name, Broker ID
PartitionCountCountSumCluster Name, Broker ID
ProduceLocalTimeMsMeanThe mean time in milliseconds for the follower to send a responseMillisecondsMultiCluster Name, Broker ID
ProduceLocalTimeMsMeanMillisecondsSumCluster Name, Broker ID
ProduceMessageConversionsPerSecThe number of produce message conversions per second for the brokerCount/SecondMultiCluster Name, Broker ID
ProduceMessageConversionsPerSecCount/SecondMultiCluster Name, Broker ID, Topic
ProduceMessageConversionsPerSecCount/SecondSumCluster Name, Broker ID
ProduceMessageConversionsPerSecCount/SecondSumCluster Name, Broker ID, Topic
ProduceMessageConversionsTimeMsMeanThe mean time in milliseconds spent on message format conversionsMillisecondsMultiCluster Name, Broker ID
ProduceMessageConversionsTimeMsMeanMillisecondsSumCluster Name, Broker ID
ProduceRequestQueueTimeMsMeanThe mean time in milliseconds that request messages spend in the queueMillisecondsMultiCluster Name, Broker ID
ProduceRequestQueueTimeMsMeanMillisecondsSumCluster Name, Broker ID
ProduceResponseQueueTimeMsMeanThe mean time in milliseconds that response messages spend in the queueMillisecondsMultiCluster Name, Broker ID
ProduceResponseQueueTimeMsMeanMillisecondsSumCluster Name, Broker ID
ProduceResponseSendTimeMsMeanThe mean time in milliseconds spent on sending response messagesMillisecondsMultiCluster Name, Broker ID
ProduceResponseSendTimeMsMeanMillisecondsSumCluster Name, Broker ID
ProduceThrottleByteRateThe number of throttled bytes per secondBytes/SecondMultiCluster Name, Broker ID
ProduceThrottleByteRateBytes/SecondSumCluster Name, Broker ID
ProduceThrottleQueueSizeThe number of messages in the throttle queueCountMultiCluster Name, Broker ID
ProduceThrottleQueueSizeCountSumCluster Name, Broker ID
ProduceThrottleTimeThe average produce throttle time in millisecondsMillisecondsMultiCluster Name, Broker ID
ProduceThrottleTimeMillisecondsSumCluster Name, Broker ID
ProduceTotalTimeMsMeanThe mean produce time in millisecondsMillisecondsMultiCluster Name, Broker ID
ProduceTotalTimeMsMeanMillisecondsSumCluster Name, Broker ID
RequestBytesMeanThe mean number of request bytes for the brokerBytesMultiCluster Name, Broker ID
RequestBytesMeanBytesSumCluster Name, Broker ID
RequestExemptFromThrottleTimeThe average time in milliseconds spent in broker network and I/O threads to process requests that are exempt from throttlingMillisecondsMultiCluster Name, Broker ID
RequestExemptFromThrottleTimeMillisecondsSumCluster Name, Broker ID
RequestHandlerAvgIdlePercentThe average percentage of the time the request handler threads are idlePercentMultiCluster Name, Broker ID
RequestHandlerAvgIdlePercentPercentSumCluster Name, Broker ID
RequestThrottleQueueSizeThe number of messages in the throttle queueCountMultiCluster Name, Broker ID
RequestThrottleQueueSizeCountSumCluster Name, Broker ID
RequestThrottleTimeThe average request throttle time in millisecondsMillisecondsMultiCluster Name, Broker ID
RequestThrottleTimeMillisecondsSumCluster Name, Broker ID
RequestTimeThe average time in milliseconds spent in broker network and I/O threads to process requestsMillisecondsMultiCluster Name, Broker ID
RequestTimeMillisecondsSumCluster Name, Broker ID
RootDiskUsedThe percentage of the root disk used by the brokerPercentMultiCluster Name, Broker ID✔️
RootDiskUsedPercentSumCluster Name, Broker ID✔️
SwapFreeThe size in bytes of swap memory that is available for the brokerBytesMultiCluster Name, Broker ID✔️
SwapFreeBytesSumCluster Name, Broker ID✔️
SwapUsedThe size in bytes of swap memory that is in use for the brokerBytesMultiCluster Name, Broker ID✔️
SwapUsedBytesSumCluster Name, Broker ID✔️
UnderMinIsrPartitionCountThe number of under minIsr partitions for the brokerCountMultiCluster Name, Broker ID
UnderMinIsrPartitionCountCountSumCluster Name, Broker ID
UnderReplicatedPartitionsThe number of under-replicated partitions for the brokerCountMultiCluster Name, Broker ID
UnderReplicatedPartitionsCountSumCluster Name, Broker ID
ZooKeeperRequestLatencyMsMeanMean latency in milliseconds for ZooKeeper requests from brokerMillisecondsMultiCluster Name, Broker ID✔️
ZooKeeperRequestLatencyMsMeanMillisecondsMultiCluster Name✔️
ZooKeeperRequestLatencyMsMeanMillisecondsSumCluster Name, Broker ID✔️
ZooKeeperRequestLatencyMsMeanMillisecondsSumCluster Name✔️
ZooKeeperSessionStateConnection status of broker's ZooKeeper session which may be one of the following: NOT_CONNECTED: 0.0, ASSOCIATING: 0.1, CONNECTING: 0.5, CONNECTEDREADONLY: 0.8, CONNECTED: 1.0, CLOSED: 5.0, AUTH_FAILED: 10.0.CountMultiCluster Name, Broker ID✔️
ZooKeeperSessionStateCountMultiCluster Name✔️
ZooKeeperSessionStateCountSumCluster Name, Broker ID
ZooKeeperSessionStateCountSumCluster Name