Apache KafkaApache Kafka
Automatic and intelligent observability with trace and metric insights.
Extension
- Product information
- Release notes
Overview
With Dynatrace, you can get observability for Kafka without touching any code, thanks to automatic monitoring. Seamless end-to-end traces for connected producer and consumer clients allow you to diagnose anomalies and pinpoint the root cause of the broken code before your customers are affected. Comprehensive metrics give you insight into your Kafka servers with health and performance metrics for brokers, topics, producers, and consumers. Events point you to critical anomalies, reducing the mean repair time.
Use cases
- Capture every message across tiers without blind spots.
- Improve the performance of your producer and consumer services end-to-end.
- Troubleshoot asynchronous service problems across your stack proactively.
- Prevent message processing anomalies to reduce the mean time to repair.
- Monitor the health and performance of all your brokers and topics.
- Understand your consumer lag.
Get started
Messaging clients (applications)
To get trace insight:
- Install OneAgent on the virtual machine or server of your messaging clients (applications).
- Set up Dynatrace on Kubernetes or OpenShift for your messaging client (application) workloads.
- Activate the following OneAgent features:
- Java Kafka
- Java Kafka Streams
- Spring for Apache Kafka
- Node.js KafkaJs
- .NET Messaging Apache Kafka
To get log insight:
Messaging servers (brokers)
Prerequisites:
- Kafka broker, producer and consumer running on a supported Linux or Windows operating system.
- OneAgent version 1.270+
- Dynatrace version 1.270+
To get metric insight:
- Install OneAgent on the virtual machine or server of your Kafka broker process.
- Ensure your Kafka broker, producer and consumer processes are monitored.
- Activate the OneAgent feature
Java Metric Extensions 2.0 (JMX). - Select in Dynatrace Add to environment to configure the extension.
- Open the Apache Kafka Overview dashboard.
Extension
With the Kafka extension, you can get additional insight into your Kafka server with metrics for brokers, topics, producers, consumers, and more. The extension also provides alerts for the most critical metrics. It creates a custom topology and entities for brokers, topics, producers, and consumers. It provides a dashboard for easier access and configuration of the extension and its entities.
The extension will gather different metrics depending on if it's monitoring a Kafka broker, producer or consumer. Make sure to activate the extension on all of them to get all metrics.
Requirements
- OneAgent version 1.270+
- Dynatrace version 1.270+
- OneAgent feature
Java Metric Extensions 2.0 (JMX)activated
Select Add to environment to get started.
Extension content
Content typeNumber of items included
alerts
4
screen chart groups
12
list screen layout
4
screen message cards
4
screen layout
4
metric metadata
66
screen injections
6
screen actions
8
screen entities lists
11
generic relationship
8
dashboards
1
generic type
4
screen properties
3
Feature sets
Below is a complete list of the feature sets provided in this version. To ensure a good fit for your needs, individual feature sets can be activated and deactivated by your administrator during configuration.
Feature setsNumber of metrics included
| Metric name | Metric key | Description | Unit |
|---|---|---|---|
| Kafka Server - Handler Pool Idle Percent Rate | kafka.server.handler.average-idle-percent.rate | - | PerSecond |
| Metric name | Metric key | Description | Unit |
|---|---|---|---|
| Kafka Controller - Offline partitions | kafka.controller.KafkaController.OfflinePartitionsCount | - | Count |
| Kafka Controller - Active cluster controllers | kafka.controller.KafkaController.ActiveControllerCount.Value | - | Count |
| Metric name | Metric key | Description | Unit |
|---|---|---|---|
| Kafka Connector - Status | kafka.connector.status | Equals 1 if the status is running, 0 otherwise. | Count |
| Kafka Connector - Task status | kafka.connector.task.status | Equals 1 if the status is running, 0 otherwise. | Count |
| Metric name | Metric key | Description | Unit |
|---|---|---|---|
| Kafka Producer - Incoming byte rate | kafka.producer.producer-metrics.incoming-byte-rate | - | BytePerSecond |
| Kafka Producer - Outgoing byte rate | kafka.producer.producer-metrics.outgoing-byte-rate | - | BytePerSecond |
| Kafka Producer - I/O Wait time | kafka.producer.producer-metrics.io-wait-time-ns-avg | - | NanoSecond |
| Kafka Producer - Response rate | kafka.producer.producer-metrics.response-rate | - | PerSecond |
| Kafka Producer - Request latency | kafka.producer.producer-metrics.request-latency-avg | - | MilliSecond |
| Kafka Producer - Compression rate | kafka.producer.producer-metrics.compression-rate-avg | - | PerSecond |
| Kafka Producer - Request size | kafka.producer.producer-metrics.request-size-avg | - | Byte |
| Kafka Producer - Requests | kafka.producer.producer-metrics.request-rate | - | PerSecond |
| Kafka Producer - Byte rate | kafka.producer.producer-topic-metrics.byte-rate | - | BytePerSecond |
| Kafka Producer - Compression rate | kafka.producer.producer-topic-metrics.compression-rate | - | BytePerSecond |
| Kafka Producer - Failed Requests Rate | kafka.producer.producer-topic-metrics.record-error-rate | - | PerSecond |
| Kafka Producer - Requests Sent Rate | kafka.producer.producer-topic-metrics.record-send-rate | - | PerSecond |
| Metric name | Metric key | Description | Unit |
|---|---|---|---|
| Kafka Server - Purgatory Produce Size | kafka.server.purgatory.produce-delay-size | - | Count |
| Kafka Server - Purgatory Fetch Size | kafka.server.purgatory.fetch-delay-size | - | Count |
| Metric name | Metric key | Description | Unit |
|---|---|---|---|
| Kafka Consumer - Requests | kafka.consumer.consumer-metrics.request-rate | - | PerSecond |
| Kafka Consumer - Request size | kafka.consumer.consumer-metrics.request-size-avg | - | Byte |
| Kafka Consumer - Incoming byte rate | kafka.consumer.consumer-metrics.incoming-byte-rate | - | BytePerSecond |
| Kafka Consumer - Outgoing byte rate | kafka.consumer.consumer-metrics.outgoing-byte-rate | - | BytePerSecond |
| Kafka Consumer - Request latency | kafka.consumer.consumer-metrics.request-latency-avg | - | MilliSecond |
| Kafka Consumer - Messages consumed rate | kafka.consumer.consumer-metrics.records-consumed-rate | - | PerSecond |
| Kafka Consumer - Bytes consumed rate | kafka.consumer.consumer-metrics.bytes-consumed-rate | - | PerSecond |
| Kafka Consumer - Fetch latency | kafka.consumer.consumer-metrics.fetch-latency-avg | - | MilliSecond |
| Kafka Consumer - Consumer lag | kafka.consumer.consumer-metrics.records-lag | - | Count |
| Kafka Consumer - Consumer lag average | kafka.consumer.consumer-metrics.records-lag-avg | - | Count |
| Kafka Consumer - Consumer lag maximum | kafka.consumer.consumer-metrics.records-lag-max | - | Count |
| Metric name | Metric key | Description | Unit |
|---|---|---|---|
| Kafka Log - Log flush 95th percentile | kafka.log.LogFlushStats.LogFlushRateAndTimeMs.Percentile95th | - | MilliSecond |
| Kafka Log - Log flush mean time | kafka.log.LogFlushStats.LogFlushRateAndTimeMs.Mean | - | MilliSecond |
| Metric name | Metric key | Description | Unit |
|---|---|---|---|
| Kafka Connect - Requests | kafka.connect.connect-metrics.request-rate | - | PerSecond |
| Kafka Connect - Outgoing byte rate | kafka.connect.connect-metrics.outgoing-byte-rate | - | BytePerSecond |
| Kafka Connect - Request size | kafka.connect.connect-metrics.request-size-avg | - | Byte |
| Kafka Connect - Incoming byte rate | kafka.connect.connect-metrics.incoming-byte-rate | - | BytePerSecond |
| Metric name | Metric key | Description | Unit |
|---|---|---|---|
| Kafka Server - ZooKeeper disconnects | kafka.server.SessionExpireListener.ZooKeeperDisconnectsPerSec.OneMinuteRate | - | PerSecond |
| Kafka Server - ZooKeeper expires | kafka.server.SessionExpireListener.ZooKeeperExpiresPerSec.OneMinuteRate | - | PerSecond |
| Kafka Server - Zookeeper Active Connections | kafka.server.active-connections | - | Count |
| Metric name | Metric key | Description | Unit |
|---|---|---|---|
| Kafka Network - Produce requests per second | kafka.network.RequestMetrics.RequestsPerSec.Produce.OneMinuteRate | - | PerSecond |
| Kafka Network - FetchConsumer requests per second | kafka.network.RequestMetrics.RequestsPerSec.FetchConsumer.OneMinuteRate | - | PerSecond |
| Kafka Network - FetchFollower requests per second | kafka.network.RequestMetrics.RequestsPerSec.FetchFollower.OneMinuteRate | - | PerSecond |
| Kafka Network - Total time per Produce request | kafka.network.RequestMetrics.TotalTimeMs.Produce.Count | - | MilliSecond |
| Kafka Network - Total time per FetchConsumer request | kafka.network.RequestMetrics.TotalTimeMs.FetchConsumer.Count | - | MilliSecond |
| Kafka Network - Total time per FetchFollower request | kafka.network.RequestMetrics.TotalTimeMs.FetchFollower.Count | - | MilliSecond |
| Kafka Network - Request queue size | kafka.network.RequestChannel.RequestQueueSize.Value | - | Count |
| Metric name | Metric key | Description | Unit |
|---|---|---|---|
| Kafka Controller - Leader election rate | kafka.controller.ControllerStats.LeaderElectionRateAndTimeMs.OneMinuteRate | - | MilliSecond |
| Kafka Controller - Unclean election rate | kafka.controller.ControllerStats.UncleanLeaderElectionsPerSec.OneMinuteRate | - | PerSecond |
| Kafka Server - Leader count | kafka.server.ReplicaManager.LeaderCount.Value | - | Count |
| Metric name | Metric key | Description | Unit |
|---|---|---|---|
| Kafka Broker - Incoming byte rate | kafka.server.BrokerTopicMetrics.BytesInPerSec.OneMinuteRate | - | BytePerSecond |
| Kafka Broker - Outgoing byte rate | kafka.server.BrokerTopicMetrics.BytesOutPerSec.OneMinuteRate | - | BytePerSecond |
| Kafka Broker - Messages in rate | kafka.server.BrokerTopicMetrics.MessagesInPerSec.OneMinuteRate | - | PerSecond |
| Kafka Broker - Follower fetch requests rate | kafka.server.BrokerTopicMetrics.TotalFollowerFetchRequestsPerSec.OneMinuteRate | - | PerSecond |
| Kafka Broker - Produce message conversions rate | kafka.server.BrokerTopicMetrics.ProduceMessageConversionsPerSec.OneMinuteRate | - | PerSecond |
| Kafka Broker - Partitions | kafka.server.ReplicaManager.PartitionCount | - | Count |
| Kafka Broker - Under replicated partitions | kafka.server.ReplicaManager.UnderReplicatedPartitions | - | Count |
| Kafka Broker - Produce request rate | kafka.server.BrokerTopicMetrics.TotalProduceRequestsPerSec.OneMinuteRate | - | PerSecond |
| Kafka Broker - Fetch request rate | kafka.server.BrokerTopicMetrics.TotalFetchRequestsPerSec.OneMinuteRate | - | PerSecond |
| Kafka Broker - Failed produce requests | kafka.server.BrokerTopicMetrics.FailedProduceRequestsPerSec.OneMinuteRate | - | PerSecond |
| Kafka Broker - Failed fetch requests | kafka.server.BrokerTopicMetrics.FailedFetchRequestsPerSec.OneMinuteRate | - | PerSecond |
| Kafka Server - Max follower lag | kafka.server.ReplicaFetcherManager.MaxLag.Replica.Value | - | Count |
| Kafka Server - Current follower lag | kafka.server.FetcherLagMetrics.ConsumerLag.Value | - | Count |
| Kafka Server - Fetch Conversions Rate | kafka.server.FetchConversionsRate.OneMinuteRate | - | Count |
| Kafka Server - Produce Conversions Rate | kafka.server.ProduceConversionsRate.OneMinuteRate | - | Count |
| Metric name | Metric key | Description | Unit |
|---|---|---|---|
| Kafka Server - Disk Read Rate | kafka.server.disk.read-bytes | - | PerSecond |
| Kafka Server - Disk Write Rate | kafka.server.disk.write-bytes | - | PerSecond |
Full version history
To have more information on how to install the downloaded package, please follow the instructions on this page.
ReleaseDate
Full version history
- Broker now uses the hostname instead of the host ID, when available.
- Entity lists for broker, producer and consumer now show the host and the process they are related to, as well as allowing you to filter by those entities.
- Removed any reference to zookeeper metrics or entities. To get insight into Zookeeper, activate the recently released standalone extension.
Full version history
- Changed the name of the out-of-the-box alerts to include the word Kafka in the title for increased accessibility.
- Added the following metrics under the zookeeper-metrics feature set:
- kafka.zookeeper.server.maxRequestLatency
- kafka.zookeeper.server.minRequestLatency
- kafka.zookeeper.server.avgRequestLatency
- kafka.zookeeper.server.aliveConnections
- kafka.zookeeper.server.packetsReceived.count
- kafka.zookeeper.server.packetsSent.count
- kafka.zookeeper.server.outstandingRequests
- Added custom topology for the zookeeper
- Added UA screens for the zookeeper
- Modified the out-of-the-box dashboard to include data about the zookeeper
Full version history
- Added a new metric kafka.consumer.consumer-metrics.records-lag to show latest consumer lag instead of the already available consumer lag average and maximum.
- Removed the status dimension from the kafka.connector.status and kafka.connector.task.status as the framework is not ready to report such dimensions. The metrics now report 1 when the status is running or 0 otherwise.
Full version history
- Fixed a bug where consumer-fetch-manager-metrics where not being captured correctly.
- Added connector task status to metrics.
- Replace entity lists for Host and Process Group Instance with properties in broker, consumer and producer screens.
Full version history
- Fixed a bug where metrics for kafka.consumer:type=consumer-fetch-manager-metrics where not being populated:
- kafka.consumer.consumer-metrics.records-lag-avg
- kafka.consumer.consumer-metrics.records-lag-max
- kafka.consumer.consumer-metrics.records-consumed-rate
- kafka.consumer.consumer-metrics.bytes-consumed-rate
- kafka.consumer.consumer-metrics.fetch-latency-avg
Full version history
- Add topic dimension to existing metrics:
- kafka.server.BrokerTopicMetrics.BytesInPerSec.OneMinuteRate
- kafka.server.BrokerTopicMetrics.BytesOutPerSec.OneMinuteRate
- kafka.server.BrokerTopicMetrics.MessagesInPerSec.OneMinuteRate
- kafka.server.BrokerTopicMetrics.TotalProduceRequestsPerSec.OneMinuteRate
- Add new metrics:
- kafka.server.BrokerTopicMetrics.TotalFollowerFetchRequestsPerSec.OneMinuteRate
- kafka.server.BrokerTopicMetrics.ProduceMessageConversionsPerSec.OneMinuteRate
- Modified UA screens to include the metrics and dimensions listed above:
- Kafka Broker screen
- Kafka Topic screen
- Host injected tile
- Process Group Instance injected tile
Full version history
- Kafka Server - Current follower lag: Fixed typo in metric key, changed from kafka.server.FetcherLagMetrics.CosumerLag.Value to kafka.server.FetcherLagMetrics.ConsumerLag.Value
- This is a breaking change if you're using this metric in dashboards, metric events or other configurations. You will need to reference the new key.
- Added connector status metric
- Added primary entity for most metrics
- Added navigation card to all screens
Full version history
No release notes