Prometheus Alertmanager (v2)

Prometheus Alertmanager (v2)

Receive Prometheus Alertmanager events and open problems in Dynatrace

Extension

Product information
Release notes

Overview

Receive Alertmanager events and turn them into Dynatrace Problems

Get started

Configuration

In Dynatrace, create a monitoring configuration for this extension, this extension should run on a single Activegate, later Prometheus Alertmanager will need to be configured to send events to this specific Activegate.

The parameters are:

Group name - A user friendly group name for the Custom Devices to be created by this endpoint
API Token - A token to create custom devices, read and write problems (DataExport, problems.read, problems.write, events.ingest)
Listening Port - The port where the Activegate will listen to, Alertmanager needs to reach this port.
Problem opening severities - The list of "severity" labels that should send problem opening events. For instance critical. All other severity labels will send Custom Info Events instead.

Alertmanager

In Alertmanager, create a webhook config where the url field points to the Dynatrace Activegate, example: http://<my.ag.ip.address>:9393/webhook.

Example receiver configuration:

receivers:
- name: 'dynatrace'
  webhook_configs:
  - url: 'http://100.94.253.97:9393/webhook'

The endpoint always ends in /webhook. Note that problems will only be created for alerts where label called severity matches the ones configured in Dynatrace.

All other alerting options and routes are to be configured in Alertmanager.

FAQ

Troubleshooting

You can troubleshoot the integration by sending a POST request directly to the Activegate:

curl -H 'Content-Type: application/json' -d '{"receiver": "dynatrace", "status": "firing", "alerts": [{"status": "firing", "labels": {"alertname": "TargetDown", "namespace": "kube-system", "service": "kubelet", "severity": "warning"}, "annotations": {"message": "11.11% of the kubelet/kubelet targets in kube-system"}, "startsAt": "2021-03-19T01:35:45.72Z", "endsAt": "0001-01-01T00:00:00Z", "generatorURL": "http://openshift.com", "fingerprint": "e425bb91067b6c9e"}], "groupKey": "{}:{alertname=\"Test Alert\", cluster=\"Cluster 02\", service=\"kubelet", "groupLabels": {"alertname": "Test Alert", "cluster": "Cluster 02", "service": "Service 02"}, "commonLabels": {"alertname": "Test Alert", "cluster": "Cluster 02", "service": "Service 02"}, "commonAnnotations": {"annotation_01": "annotation 01", "annotation_02": "annotation 03"}, "externalURL": "http://8598cebf58a1:9093"}' http://<ACTIVEGATE_IP_ADDRESS>:9393/webhook

Subscribe to new releases

Copy to clipboard

Related to Prometheus Alertmanager (v2)

Prometheus

Unlock Prometheus power with intelligent analytics and automated insights.

Full version history

To have more information on how to install the downloaded package, please follow the instructions on this page.

ReleaseDate

Full version history

Version 2.1.7

Use the label device if available, and host if device is not available, for the custom device name

Full version history

Version 2.1.5

Fix an issue where the AG location could not be determined in custom installs

Full version history

Version 2.1.4

Fix an issue where a 500 error was returned when attempting to delete a problem that was not present on the cache

Full version history

Version 2.1.3

Fix an issue for AG installations that don't have a custom.properties file

Full version history

Version 2.1.2

Fix an issue where the extension could not find the correct directory for custom AG installations

Full version history

Version 2.1.1

Breaking change

Version 2.1.1 uses the Event v2 API, which allow us to close problems gracefully instead of manually.

This means that the token now must have the Ingest events scope.

Problems closed this way can be reopened by Alertmanager, which solves an issue where a problem that is resent with the same body was supressed, even though the alert was active in Alertmanager.

Full version history

Version 2.0.1

First EF2 version release

Discover recent additions to Dynatrace

Problems

Analyze abnormal system behavior and performance problems detected by Davis AI.

Logs

Explore all your logs without writing a single query.

Security Investigator

Fast and precise forensics for security and logs on Grail data with DQL queries.

Business Flow

Track, analyze, and optimize your critical business processes.

Cost & Carbon Optimization

Track, analyze, and optimize your IT carbon footprint and public cloud costs.

Davis Anomaly Detection

Detect anomalies in timeseries using the Davis AI

Analyze your data

Understand your data better with deep insights and clear visualizations.

Notebooks

Create powerful, data-driven documents for custom analytics and collaboration.

Dashboards

Transform complex data into clear visualizations with custom dashboards.

Automate your processes

Turn data and answers into actions, securely, and at scale.

Workflows

Automate tasks in your IT landscape, remediate problems, and visualize processes

Jira

Create, query, comment, transition, and resolve Jira tickets within workflows.

Slack

Automate Slack messaging for security incidents, attacks, remediation, and more.

Secure your cloud application

See vulnerabilities and attacks in your environment.

Security Overview

Get a comprehensive overview of the security of your applications.

Code-Level Vulnerabilities

Detect vulnerabilities in your code in real time.

Threats & Exploits

Understand, triage, and investigate application security findings and alerts.

Are you looking for something different?

We have hundreds of apps, extensions, and other technologies to customize your environment

Leverage our newest innovations of Dynatrace Saas

Kick-start your app creation

Kick-start your app creation

Whether you’re a beginner or a pro, Dynatrace Developer has the tools and support you need to create incredible apps with minimal effort.

Go to Dynatrace Developer

Upgrading from Dynatrace Managed to SaaS

Upgrading from Dynatrace Managed to SaaS

Drive innovation, speed, and agility in your organization by seamlessly and securely upgrading.

Log Management and Analytics

Log Management and Analytics

Innovate faster and more efficiently with unified log management and log analytics for actionable insights and automation.