Log Monitoring in Kubernetes

Dynatrace Log Monitoring supports collecting logs from Kubernetes container orchestration systems via OneAgent.

How autodiscovery works

For Kubernetes version 1.14+, OneAgent autodiscovers log messages written to the containerized application's stdout/stderr streams. Logs written directly to pods are not discovered by OneAgent. Kubernetes Engine saves these log streams to a file in the /var/log/pods directory on the Kubernetes node. OneAgent autodiscovers these log files from that path.

Autodiscovery requirements

Requirements for autodiscovery of Kubernetes logs:

• Docker, containerd, or CRI-O container runtime is used.
• The process running in the container is an important process.
• Logs are written to the container's stdout/stderr streams.
• The log file in /var/log/pods exists for a minimum of one minute after container execution is finished.

Connecting logs with Kubernetes properties and annotations

Before you start, you need to deploy Dynatrace Operator and enable Kubernetes API monitoring.

Note: Only the classic full-stack and cloud-native full-stack injections are supported. For details, see Deployment options on Kubernetes/OpenShift.

As OneAgent discovers logs from /var/log/pods path on Kubernetes node, it connects specific logs to corresponding pods based on the pod uid from the file path.

In addition, OneAgent enchances these logs with the following Kubernetes metadata:

• dt.kubernetes.cluster.name
• dt.kubernetes.node.system_uuid
• k8s.pod.name
• k8s.pod.uid
• k8s.container.name
• k8s.namespace.name
• k8s.deployment.name

This metadata is used to map the logs to the entity model of Kubernetes clusters, namespaces, workloads, and pods. As a result, the logs are present in the Kubernetes entity model on the Dynatrace platform.