Metric selector in custom metric events

The metric selector is a powerful tool for querying your data. It provides you two major possibilities:

In this example, we want to detect anomalies on the combined incoming and outgoing network traffic by calculating the sum of all bytes read (builtin:host.net.bytesRx) and written (builtin:host.net.bytesTx). The metric expression for that is:

((builtin:host.net."bytesTx":splitBy())+(builtin:host.net."bytesRx":splitBy()))

This expression evaluates to a single metric result that Davis will use to learn a baseline and to detect and alert on anomalies.

metric expression anomaly detection

A metric selector can consist of thousands of individual metric measurements. It is important to understand the implications when configuring a selector that consists of measurements coming from thousands of individual sources. Dynatrace applies safety limits to anomaly detection in terms of the number of metric dimensions that can be observed within one monitoring environment to avoid any operational issues.

Combining metrics for anomaly detection

With the power of a metric expression, you can implement alerting with a top-down view of a situation rather than alerting on each individual component.

For example, you can observe log patterns across multiple hosts. By calculating the total count of observed log patterns across all relevant log files, Dynatrace can detect pattern anomalies on the accumulated log stream rather than on the individual counts per log file.

In case of sparse counts across many entities (for example, an error count across multiple processes of the same type), aggregated top-down anomaly detection is much more resilient against false-positive alerts compared to detection on an individual error count per process.

Topology mapping

Metric events based on a metric selector support topology awareness. The resulting mapping depends on the data granularity of the result.

Metric selectors that are split by an entity persist that mapping and are topology-aware. The events raised on such metrics are mapped to the original source.

anomaly detection - host level

When metric selectors result in a single aggregated series, with no clear entity and topology reference, the events raised on such metrics are mapped to the global monitoring environment.

anomaly detection - global level

Override topology mapping

You can override automatic selection of the entity type the events are mapped to. Be aware that you should select only entity types that are really referenced in the incoming metric measurements. If an entity type is selected where the metric does not show the necessary dimension, the entity override is ignored.

To override the automatic entity type, in the metric event configuration, expand Advanced entity settings and select the required entity type.