All
0 Results filtered by:
We couldn't find any results
You can search all listings, or try a different spelling or keyword. Still nothing? Dynatrace makes it easy to create custom apps.
Prometheus Alertmanager (v2)Prometheus Alertmanager (v2)
Receive Prometheus Alertmanager events and open problems in Dynatrace
Extension- Product information
- Release notes
Overview
Receive Alertmanager events and turn them into Dynatrace Problems
Get started
Configuration
In Dynatrace, create a monitoring configuration for this extension, this extension should run on a single Activegate, later Prometheus Alertmanager will need to be configured to send events to this specific Activegate.
The parameters are:
- Group name - A user friendly group name for the Custom Devices to be created by this endpoint
- API Token - A token to create custom devices, read and write problems (DataExport, problems.read, problems.write, events.ingest)
- Listening Port - The port where the Activegate will listen to, Alertmanager needs to reach this port.
- Problem opening severities - The list of "severity" labels that should send problem opening events. For instance critical. All other severity labels will send Custom Info Events instead.
Alertmanager
In Alertmanager, create a webhook config where the url field points to the Dynatrace Activegate, example: http://<my.ag.ip.address>:9393/webhook.
Example receiver configuration:
receivers:
- name: 'dynatrace'
webhook_configs:
- url: 'http://100.94.253.97:9393/webhook'
The endpoint always ends in /webhook. Note that problems will only be created for alerts where label called severity matches the ones configured in Dynatrace.
All other alerting options and routes are to be configured in Alertmanager.
FAQ
Troubleshooting
You can troubleshoot the integration by sending a POST request directly to the Activegate:
curl -H 'Content-Type: application/json' -d '{"receiver": "dynatrace", "status": "firing", "alerts": [{"status": "firing", "labels": {"alertname": "TargetDown", "namespace": "kube-system", "service": "kubelet", "severity": "warning"}, "annotations": {"message": "11.11% of the kubelet/kubelet targets in kube-system"}, "startsAt": "2021-03-19T01:35:45.72Z", "endsAt": "0001-01-01T00:00:00Z", "generatorURL": "http://openshift.com", "fingerprint": "e425bb91067b6c9e"}], "groupKey": "{}:{alertname=\"Test Alert\", cluster=\"Cluster 02\", service=\"kubelet", "groupLabels": {"alertname": "Test Alert", "cluster": "Cluster 02", "service": "Service 02"}, "commonLabels": {"alertname": "Test Alert", "cluster": "Cluster 02", "service": "Service 02"}, "commonAnnotations": {"annotation_01": "annotation 01", "annotation_02": "annotation 03"}, "externalURL": "http://8598cebf58a1:9093"}' http://<ACTIVEGATE_IP_ADDRESS>:9393/webhook
Related to Prometheus Alertmanager (v2)
Full version history
To have more information on how to install the downloaded package, please follow the instructions on this page.
ReleaseDate
Full version history
Version 2.1.11
- Add a 30 seconds timeout on the dynatrace api client
Full version history
Version 2.1.10
- Fix an issue with jobs that stopped the extension from starting.
Full version history
Bugfixes
- Add exception handling to the cache in case of failures.
Full version history
Version 2.1.7
- Use the label device if available, and host if device is not available, for the custom device name
Full version history
Version 2.1.5
- Fix an issue where the AG location could not be determined in custom installs
Full version history
Version 2.1.4
- Fix an issue where a 500 error was returned when attempting to delete a problem that was not present on the cache
Full version history
Version 2.1.3
- Fix an issue for AG installations that don't have a custom.properties file
Full version history
Version 2.1.2
- Fix an issue where the extension could not find the correct directory for custom AG installations
Full version history
Version 2.1.1
Breaking change
Version 2.1.1 uses the Event v2 API, which allow us to close problems gracefully instead of manually.
This means that the token now must have the Ingest events scope.
Problems closed this way can be reopened by Alertmanager, which solves an issue where a problem that is resent with the same body was supressed, even though the alert was active in Alertmanager.
Full version history
Version 2.0.1
- First EF2 version release