• Home
  • API
  • Environment
  • Settings
  • Schemas
  • Kubernetes workload anomaly detection

Settings API - Kubernetes workload anomaly detection schema table

Kubernetes workload anomaly detection (builtin:anomaly-detection.kubernetes.workload)

Dynatrace automatically detects a wide range of common Kubernetes-related issues. Use these settings to configure alerts relevant to your Kubernetes workload. Changing thresholds resets the observation period.

Schema IDSchema groupsScope
builtin:anomaly-detection.kubernetes.workload
  • group:anomaly-detection.kubernetes
  • group:anomaly-detection
CLOUD_APPLICATION_NAMESPACE - Kubernetes namespace
KUBERNETES_CLUSTER - Kubernetes cluster
environment
Retrieve schema via Settings API
GETManagedhttps://{your-domain}/e/{your-environment-id}/api/v2/settings/schemas/builtin:anomaly-detection.kubernetes.workload
SaaShttps://{your-environment-id}.live.dynatrace.com/api/v2/settings/schemas/builtin:anomaly-detection.kubernetes.workload
Environment ActiveGatehttps://{your-activegate-domain}/e/{your-environment-id}/api/v2/settings/schemas/builtin:anomaly-detection.kubernetes.workload

Authentication

To execute this request, you need an access token with Read settings (settings.read) scope. To learn how to obtain and use it, see Tokens and authentication.

Parameters

PropertyLabelTypeDescriptionRequired
containerRestarts-ContainerRestarts-required
deploymentStuck-DeploymentStuck-required
pendingPods-PendingPods-required
podStuckInTerminating-PodStuckInTerminating-required
workloadWithoutReadyPods-WorkloadWithoutReadyPods-required
notAllPodsReady-NotAllPodsReady-required
highMemoryUsage-HighMemoryUsage-required
highCpuUsage-HighCpuUsage-required
highCpuThrottling-HighCpuThrottling-required
oomKills-OOMKills-required
jobFailureEvents-JobFailureEvents-required
podBackoffEvents-PodBackoffEvents-required
podEvictionEvents-PodEvictionEvents-required
podPreemptionEvents-PodPreemptionEvents-required

Request body objects

The ContainerRestartsConfig object
PropertyLabelTypeDescriptionRequired
thresholdthere is at leastinteger-required
samplePeriodInMinutesper minute, for anyinteger-required
observationPeriodInMinuteswithin the lastinteger-required
The ContainerRestarts object
PropertyLabelTypeDescriptionRequired
enabledDetect container restartsboolean-required
configuration-ContainerRestartsConfig

Alert if

required
The WorkloadWithoutReadyPodsConfig object
PropertyLabelTypeDescriptionRequired
samplePeriodInMinutesworkload has no ready pods for at leastinteger-required
observationPeriodInMinuteswithin the lastinteger-required
The WorkloadWithoutReadyPods object
PropertyLabelTypeDescriptionRequired
enabledDetect workloads without ready podsboolean

As of specific pod life cycles of different workload types, cronjobs and jobs are excluded.

required
configuration-WorkloadWithoutReadyPodsConfig

Alert if

required
The NotAllPodsReadyConfig object
PropertyLabelTypeDescriptionRequired
samplePeriodInMinutessome workload pods are not ready for at leastinteger-required
observationPeriodInMinuteswithin the lastinteger-required
The NotAllPodsReady object
PropertyLabelTypeDescriptionRequired
enabledDetect workloads with non-ready podsboolean

As of specific pod life cycles of different workload types, cronjobs and jobs are excluded.

required
configuration-NotAllPodsReadyConfig

Alert if

required
The PendingPodsConfig object
PropertyLabelTypeDescriptionRequired
thresholdthere is at leastinteger-required
samplePeriodInMinutesstuck in pending state for at leastinteger-required
observationPeriodInMinuteswithin the lastinteger-required
The PendingPods object
PropertyLabelTypeDescriptionRequired
enabledDetect pods stuck in pendingboolean

Number of pods in Pending phase

required
configuration-PendingPodsConfig

Alert if

required
The DeploymentStuckConfig object
PropertyLabelTypeDescriptionRequired
samplePeriodInMinutesworkload stops progressing for at leastinteger-required
observationPeriodInMinuteswithin the lastinteger-required
The DeploymentStuck object
PropertyLabelTypeDescriptionRequired
enabledDetect stuck deploymentsboolean

Evaluates workload condition 'Progressing'

required
configuration-DeploymentStuckConfig

Alert if

required
The PodStuckInTerminatingConfig object
PropertyLabelTypeDescriptionRequired
samplePeriodInMinutespod termination stops progressing for at leastinteger-required
observationPeriodInMinuteswithin the lastinteger-required
The PodStuckInTerminating object
PropertyLabelTypeDescriptionRequired
enabledDetect pods stuck in terminatingboolean

Deleted pods in 'Running' phase

required
configuration-PodStuckInTerminatingConfig

Alert if

required
The HighMemoryUsageConfig object
PropertyLabelTypeDescriptionRequired
thresholdamount of utilized workload memory is aboveinteger-required
samplePeriodInMinutesof defined memory limits for at leastinteger-required
observationPeriodInMinuteswithin the lastinteger-required
The HighMemoryUsage object
PropertyLabelTypeDescriptionRequired
enabledDetect memory usage saturationboolean

Memory usage (working set memory) is close to limits.

required
configuration-HighMemoryUsageConfig

Alert if

required
The HighCpuUsageConfig object
PropertyLabelTypeDescriptionRequired
thresholdamount of utilized workload CPU is aboveinteger-required
samplePeriodInMinutesof defined CPU limits for at leastinteger-required
observationPeriodInMinuteswithin the lastinteger-required
The HighCpuUsage object
PropertyLabelTypeDescriptionRequired
enabledDetect CPU usage saturationboolean

CPU usage is close to limits.

required
configuration-HighCpuUsageConfig

Alert if

required
The HighCpuThrottlingConfig object
PropertyLabelTypeDescriptionRequired
thresholdamount of CPU throttling is aboveinteger-required
samplePeriodInMinutesof CPU usage for at leastinteger-required
observationPeriodInMinuteswithin the lastinteger-required
The HighCpuThrottling object
PropertyLabelTypeDescriptionRequired
enabledDetect high CPU throttlingboolean

The CPU throttling to usage ratio exceeds the specified threshold. Important: This alert uses throttled seconds / used seconds (in millicores) in contrast to Prometheus and Grafana, which use throttled periods / total periods for the throttling ratio.

required
configuration-HighCpuThrottlingConfig

Alert if

required
The OOMKills object
PropertyLabelTypeDescriptionRequired
enabledDetect out-of-memory killsboolean-required
The JobFailureEvents object
PropertyLabelTypeDescriptionRequired
enabledDetect job failure eventsboolean

Alerts on any occurrence of Kubernetes events with reason 'BackoffLimitExceeded', 'DeadlineExceeded', or 'PodFailurePolicy'.

If 'Filter events' is enabled, make certain that you ingest events with the aforementioned reasons in order to receive alerts.

required
The PodBackoffEvents object
PropertyLabelTypeDescriptionRequired
enabledDetect pod backoff eventsboolean

Alerts on any occurrence of Kubernetes events with reason 'BackOff', as observed on pod statuses 'ImagePullBackOff', and 'CrashLoopBackOff'.

If 'Filter events' is enabled, make certain that you ingest events with the aforementioned reasons in order to receive alerts.

required
The PodEvictionEvents object
PropertyLabelTypeDescriptionRequired
enabledDetect pod eviction eventsboolean

Eviction is the process of terminating one or more pods on a node to free up resources.

Alerts on any occurrence of Kubernetes events with reason 'Evicted'.

If 'Filter events' is enabled, make certain that you ingest events with the aforementioned reasons in order to receive alerts.

required
The PodPreemptionEvents object
PropertyLabelTypeDescriptionRequired
enabledDetect pod preemption eventsboolean

Preemption is the process of terminating pods with lower priority so that pods with higher priority can be scheduled on a node.

Alerts on any occurrence of Kubernetes events with reason 'Preempted', or 'Preempting'.

If 'Filter events' is enabled, make certain that you ingest events with the aforementioned reasons in order to receive alerts.

required