All
0 Results filtered by:
We couldn't find any results
You can search all listings, or try a different spelling or keyword. Still nothing? Dynatrace makes it easy to create custom apps.
Confluent Cloud (Kafka)Confluent Cloud (Kafka)
Remotely monitor your Confluent Cloud Kafka Clusters and other resources!
Extension- Product information
- Release notes
Overview
This extension provides the ability to remotely monitor your Confluent Cloud Kafka Clusters, Connectors, Schema Registries, and KSQL DB Applications. Every minute using the API provided by Confluent it ingests data about how your different Confluent Resources are performing.
This is intended for users, who:
- Would like to monitor the health state and performance of their Confluent Cloud Resources.
- Look for analysis support for Ops, IT and Network Admins.
This enables you to:
- Monitor infrastructure with a comprehensive dashboard
- Detect usage anomalies and alert on them
Compatibility Requirements Confluent Cloud Resource(s) and API User/Token
Note: The metrics in the Kafka Lag Partition Metrics and Kafka Lag Consumer Group Metrics feature sets are not provided by the Confluent API. To obtain these metrics the Kafka Lag Exporter is needed. See the Use Cases section for additional information.
Use cases
-
This extension provides monitoring of Confluent Cloud Resources via their public API (see details tab).
-
Also supported, via the
Kafka Lag Partition MetricsandKafka Lag Consumer Group Metricsfeature sets, are metrics provided by the Kafka Lag Exporter.- NOTE: This exporter is not supported by Dynatrace and needs to be set up and run independently from this extension.
- Currently the extension only supports ingesting metrics from this exporter.
Extension content
Content typeNumber of items included
alerts
2
list screen layout
8
document dashboard
1
screen dql table
8
screen entities lists
13
screen metric tables
3
screen chart groups
28
metric metadata
52
generic type
8
screen properties
5
generic relationship
5
screen layout
16
dashboards
2
metric query
3
Feature sets
Below is a complete list of the feature sets provided in this version. To ensure a good fit for your needs, individual feature sets can be activated and deactivated by your administrator during configuration.
Feature setsNumber of metrics included
| Metric name | Metric key | Description | Unit |
|---|---|---|---|
| Confluent Flink Num Records In | confluent_flink_num_records_in | Total number of records this statement has received. | Count |
| Confluent Flink Num Records Out | confluent_flink_num_records_out | Total number of records this task statement emitted. | Count |
| Confluent Flink Pending Records | confluent_flink_pending_records | Total amount of available records after the consumer offset in a Kafka partition across all operators | Count |
| Confluent Flink Current Input Watermark Milliseconds | confluent_flink_current_input_watermark_milliseconds | The last watermark this statement has received (in milliseconds) for the given table. | MilliSecond |
| Confluent Flink Current Output Watermark Milliseconds | confluent_flink_current_output_watermark_milliseconds | The last watermark this statement has produced (in milliseconds) to the given table. | MilliSecond |
| Metric name | Metric key | Description | Unit |
|---|---|---|---|
| Kafka Cluster Received Bytes | confluent_kafka_server_received_bytes | The number of bytes of the customer's data received from the network. | Byte |
| Kafka Cluster Sent Bytes | confluent_kafka_server_sent_bytes | The number of bytes of the customer's data sent over the network. | Byte |
| Kafka Cluster Received Records | confluent_kafka_server_received_records | The number of records received. | Count |
| Kafka Cluster Sent Records | confluent_kafka_server_sent_records | The number of records sent. | Count |
| Kafka Cluster Retained Bytes | confluent_kafka_server_retained_bytes | The current number of bytes retained by the cluster. | Byte |
| Kafka Cluster Partition Count | confluent_kafka_server_partition_count.gauge | The number of partitions. | Count |
| Kafka Cluster Load Raw | confluent_kafka_server_cluster_load_percent | A measure of the utilization of the cluster. The value is between 0.0 and 1.0. | Count |
| Metric name | Metric key | Description | Unit |
|---|---|---|---|
| Kafka Cluster Request Bytes | confluent_kafka_server_request_bytes | The delta count of total request bytes from the specified request types sent over the network. Each sample is the number of bytes sent since the previous data point. The count is sampled every 60 seconds. | Byte |
| Kafka Cluster Response Bytes | confluent_kafka_server_response_bytes | The delta count of total response bytes from the specified response types sent over the network. Each sample is the number of bytes sent since the previous data point. The count is sampled every 60 seconds. | Byte |
| Kafka Cluster Active Connection Count | confluent_kafka_server_active_connection_count.gauge | The count of active authenticated connections. | Count |
| Kafka Cluster Request Count | confluent_kafka_server_request_count.gauge | The number of requests received over the network. | Count |
| Kafka Cluster Successful Authentication Count | confluent_kafka_server_successful_authentication_count.gauge | The number of successful authentications. | Count |
| Metric name | Metric key | Description | Unit |
|---|---|---|---|
| Kafka Server Cluster Link Destination Response Bytes | confluent_kafka_server_cluster_link_destination_response_bytes | The delta count of cluster linking response bytes from all request types. Each sample is the number of bytes sent since the previous data point. The count is sampled every 60 seconds. | Byte |
| Kafka Server Cluster Link Source Response Bytes | confluent_kafka_server_cluster_link_source_response_bytes | The delta count of cluster linking source response bytes from all request types. Each sample is the number of bytes sent since the previous data point. The count is sampled every 60 seconds. | Byte |
| Kafka Server Cluster Link Count | confluent_kafka_server_cluster_link_count.gauge | The current count of cluster links. The count is sampled every 60 seconds. The implicit time aggregation for this metric is MAX. | Count |
| Kafka Server Cluster Link Mirror Topic Count | confluent_kafka_server_cluster_link_mirror_topic_count.gauge | The cluster linking mirror topic count for a link. The count is sampled every 60 seconds. | Count |
| Kafka Server Cluster Link Mirror Topic Offset Lag | confluent_kafka_server_cluster_link_mirror_topic_offset_lag | The cluster linking mirror topic offset lag maximum across all partitions. The lag is sampled every 60 seconds. | Count |
| Kafka Server Cluster Link Mirror Topic Bytes | confluent_kafka_server_cluster_link_mirror_topic_bytes | The delta count of cluster linking mirror topic bytes. The count is sampled every 60 seconds. | Byte |
| Metric name | Metric key | Description | Unit |
|---|---|---|---|
| Kafka Consumer Group Group Topic Sum Lag | kafka_consumergroup_group_topic_sum_lag | Sum of group offset lag across topic partitions | Count |
| Kafka Consumer Group Poll Time (ms) | kafka_consumergroup_poll_time_ms | Group poll time | MilliSecond |
| Kafka Consumer Group Group Offset | kafka_consumergroup_group_offset | Last group consumed offset of a partition | Count |
| Kafka Consumer Group Group Sum Lag | kafka_consumergroup_group_sum_lag | Sum of group offset lag | Count |
| Kafka Consumer Group Group Lag | kafka_consumergroup_group_lag | Group offset lag of a partition | Count |
| Kafka Consumer Group Group Lag Seconds | kafka_consumergroup_group_lag_seconds | Group time lag of a partition | Second |
| Kafka Consumer Group Group Max Lag | kafka_consumergroup_group_max_lag | Max group offset lag | Count |
| Kafka Consumer Group Group Max Lag Seconds | kafka_consumergroup_group_max_lag_seconds | Max group time lag | Second |
| Metric name | Metric key | Description | Unit |
|---|---|---|---|
| Kafka Connect Sent Records | confluent_kafka_connect_sent_records | The delta count of total number of records sent from the transformations and written to Kafka for the source connector. Each sample is the number of records sent since the previous data point. | Count |
| Kafka Connect Received Records | confluent_kafka_connect_received_records | The delta count of total number of records received by the sink connector. Each sample is the number of records received since the previous data point. | Count |
| Kafka Connect Sent Bytes | confluent_kafka_connect_sent_bytes | The delta count of total bytes sent from the transformations and written to Kafka for the source connector. Each sample is the number of bytes sent since the previous data point. | Byte |
| Kafka Connect Received Bytes | confluent_kafka_connect_received_bytes | The delta count of total bytes received by the sink connector. Each sample is the number of bytes received since the previous data point. | Byte |
| Kafka Connect Dead Letter Queue Records | confluent_kafka_connect_dead_letter_queue_records | The delta count of dead letter queue records written to Kafka for the sink connector. | Count |
| Metric name | Metric key | Description | Unit |
|---|---|---|---|
| Kafka Schema Registry Schema Count | confluent_kafka_schema_registry_schema_count.gauge | The number of registered schemas. | Count |
| Kafka Schema Registry Request Count | confluent_kafka_schema_registry_request_count.gauge | The delta count of requests received by the schema registry server. Each sample is the number of requests received since the previous data point. The count sampled every 60 seconds. | Count |
| Metric name | Metric key | Description | Unit |
|---|---|---|---|
| Kafka Ksql Streaming Unit Count | confluent_kafka_ksql_streaming_unit_count.gauge | The count of Confluent Streaming Units (CSUs) for this KSQL instance. The implicit time aggregation for this metric is MAX. | Count |
| Kafka Ksql Query Saturation | confluent_kafka_ksql_query_saturation | The maximum saturation for a given ksqlDB query across all nodes. Returns a value between 0 and 1, a value close to 1 indicates that ksqlDB query processing is bottlenecked on available resources. | Count |
| Kafka Ksql Task Stored Bytes | confluent_kafka_ksql_task_stored_bytes | The size of a given task's state stores in bytes. | Byte |
| Kafka Ksql Storage Utilization | confluent_kafka_ksql_storage_utilization | The total storage utilization for a given ksqlDB application. | Percent |
| Metric name | Metric key | Description | Unit |
|---|---|---|---|
| Kafka Partition Earliest Offset | kafka_partition_earliest_offset | Earliest offset of a partition | Count |
| Kafka Partition Latest Offset | kafka_partition_latest_offset | Latest offset of a partition | Count |
| Metric name | Metric key | Description | Unit |
|---|---|---|---|
| Confluent Flink Statement Status | confluent_flink_statement_status | This metric monitors the status of a statement within the system. Its value is always set to 1, signifying the statement's presence. The statement's current operational state is identified through the metric.status tag. | Unspecified |
| Metric name | Metric key | Description | Unit |
|---|---|---|---|
| Kafka Server Consumer Lag Offsets | confluent_kafka_server_consumer_lag_offsets | The lag between a group member's committed offset and the partition's high watermark | Count |
| Metric name | Metric key | Description | Unit |
|---|---|---|---|
| Confluent Flink Compute Pool Utilization Current CFUs | confluent_flink_compute_pool_utilization_current_cfus | The absolute number of CFUs at a given moment | Count |
| Confluent Flink Compute Pool Utilization CFU Minutes Consumed | confluent_flink_compute_pool_utilization_cfu_minutes_consumed | The number of how many CFUs consumed since the last measurement | Count |
| Confluent Flink Compute Pool Utilization CFU Limit | confluent_flink_compute_pool_utilization_cfu_limit | The possible max number of CFUs for the pool | Count |
Full version history
To have more information on how to install the downloaded package, please follow the instructions on this page.
ReleaseDate
Full version history
2.3.0
- DXS-3317
- Added Platform Dashboard
- Added
dt.security_contextattribute to entities
Full version history
V2.2.0
- DXS-3211
- Added support for Compute Pool and Flink Statement Metrics
- Added
Compute Pool Metrics,Flink Statement Metrics, andFlink Statement StatusFeature Sets - Added filters for
Table Name,Flink Statement Name, andFlink Statement Status - Added new Compute Pool entity type
- Added
- Added support for Compute Pool and Flink Statement Metrics
Full version history
- Added 3rd gen/platform charts and screens for upcoming compatibility with I&O app
Full version history
DXS-2054
- Add new calculated metrics to correct aggregation issue
func:confluent_kafka_server_received_bytes_per_secfunc:confluent_kafka_server_sent_bytes_per_sec
Full version history
v2.1.2
- Added display names to metrics calculated on the different entity screens
Full version history
v2.1.1
- Update screens section to help with validation errors seen activation of v2.1.0
Full version history
v2.0.0
- IMPORTANT : Updated Dynatrace metric keys to match metric keys from Prometheus.
- This will cause existing Dashboards & Alerts (or anything that relies on the old metric keys) to stop working! Please update them accordingly.
- Please immediately update your Monitoring Configurations once this new version is activated
- You can use still view the old metrics either via the
Confluent Kafka Overview (Deprecated Dashboard)or the Data Explorer
- Added new Cluster, Schema Registry and ksqlDB metrics.
Full version history
v1.2.1
- Updated to use Schema v1.256
- Added Entity Type to metrics
- Updated Cluster Count Dashboard Tile
Full version history
v1.1.1
- Added support for
confluent_kafka_server_cluster_load_percentmetric
Full version history
v1.1.0
- Updates to metric metadata to correct units for Lag Offsets
v1.0.0
- Initial Version to collect metrics from Confluent Cloud's API & the Kafka Lag Exporter
Full version history
No release notes