Scrape data from Prometheus

The following configuration example shows how you configure a Collector instance to scrape data from an existing Prometheus setup and import it as OTLP request into Dynatrace.

Demo configuration

yaml
receivers:
  prometheus:
     config:
       scrape_configs:
       - job_name: 'node-exporter'
         scrape_interval: 60s
         static_configs:
         - targets: ['prometheus-prometheus-node-exporter:9100']
       - job_name: 'kubestatemetric'
         scrape_interval: 60s
         static_configs:
         - targets: ['prometheus-kube-state-metrics:8080']
       - job_name: 'cadvisor'
         scrape_interval: 60s
         static_configs:
         - targets: ['cadvisor.cadvisor.svc.cluster.local:8080']
       - job_name: opentelemetry-collector
         scrape_interval: 60s
         static_configs:
         - targets:
            - 127.0.0.1:8888
       - job_name: 'istiod'
         kubernetes_sd_configs:
         - role: endpoints
           namespaces:
              names:
              - istio-system
           relabel_configs:
           -  source_labels: [__meta_kubernetes_service_name, __meta_kubernetes_endpoint_port_name]
              action: keep
              regex: istiod;http-monitoring
       - job_name: 'envoy-stats'
         metrics_path: /stats/prometheus
         kubernetes_sd_configs:
          - role: pod
         relabel_configs:
         - source_labels: [__meta_kubernetes_pod_container_port_name]
           action: keep
           regex: '.*-envoy-prom'

exporters:
  otlphttp:
    endpoint: $DT_ENDPOINT/api/v2/otlp
    headers:
      Authorization: "Api-Token $DT_API_TOKEN"

service:
  pipelines:
    metrics:
      receivers: [prometheus]
      processors: []
      exporters: [otlphttp]

Prerequisites

A Prometheus instance running on port 8888
Contrib distribution or a custom Builder version with the Prometheus receiver
The API URL of your Dynatrace environment
An API token with the relevant access scope

Components

For our configuration, we configure the following components.

Receiver

Under receivers, we specify the prometheus receiver as active receiver component for our Collector instance.

We configure the receiver with the following six jobs under scrape_configs:

node-exporter, kubestatemetric, cadvisor, and opentelemetry-collector are scrape jobs that fetch data every five seconds from the specified hosts
istiod and envoy-stats are two Kubernetes related jobs

For a full list of configuration parameters, see the Prometheus documentation.

Exporter

Under exporters, we specify the default otlphttp exporter and configure it with our Dynatrace API URL and the required authentication token.

For this purpose, we set the following two environment variables and reference them in the configuration values for endpoint and Authorization.

DT_ENDPOINT contains the base URL of your ActiveGate (for example, https://{your-environment-id}.live.dynatrace.com)
DT_API_TOKEN contains the API token

Service pipeline

Under service, we assemble our receiver and exporter objects into a metrics pipeline, which will execute the Prometheus jobs and ingest the data into Dynatrace.