Remotely monitor your Confluent Cloud Kafka Cluster and other resources with this Prometheus based Extension 2.0.
This extension provides the ability to remotely monitor your Confluent Cloud Kafka Clusters, Connectors, Schema Registries, and KSQL DB Applications. Every minute using the API provided by Confluent it ingests data about how your different Confluent Resources are performing.
This is intended for users, who:
This enables you to:
Compatibility Requirements Confluent Cloud Resource(s) and API User/Token
This extension provides monitoring of Confluent Cloud Resources via their public API (see details tab).
Also supported, via the Kafka Lag Partition Metrics
and Kafka Lag Consumer Group Metrics
feature sets, are metrics provided by the Kafka Lag Exporter.
This extension uses the Confluent Metric Export API to gather metrics.
First you will need to create either a Cloud/Cluster API Key and Secret. This can be done via the Confluent UI or via their CLI. The MetricsViewer role is required to access the Confluent API. It is suggested to use the Organization scope for this role so it can be used as clusters are created or destroyed.
In Dynatrace, then create a new Monitoring Configuration and select "Monitor Remotely without OneAgent" near the bottom on the Monitoring Source Screen.
In the Dynatrace Monitoring Configuration the Confluent Cloud API Key and API Secret are used as the Basic Auth User (API Key) /Password (API secret) combination.
Next you'll create a URL with your resource types and IDs at the end, similar to what is shown below. This URL supports multiple resources but it is recommended to not have more than 5 to 10 per URL.
https://api.telemetry.confluent.cloud/v2/metrics/cloud/export?resource.kafka.id=lkc-XXXXX&resource.connector.id=lcc-XXXX1&resource.connector.id=lcc-XXXX2
Base URL
https://api.telemetry.confluent.cloud/v2/metrics/cloud/export?
Confluent Kafka Cluster
resource.kafka.id=lkc-XXXXX
Confluent Kafka Schema Registry
resource.schema_registry.id=lsrc=XXXXX
Confluent Kafka Connector
resource.connector.id=lcc-XXXXX
Confluent Kafka KSQL DB Application
resource.ksql.id=lksqlc-XXXXX
Simply activate the extension in your environment using the in-product Hub, provide the necessary device configuration and you’re all set up.
Read more in the Prometheus Extension Documentation