Monitor your Kubernetes clusters with Dynatrace

Prerequisites

Connecting your Kubernetes clusters to Dynatrace to take advantage of the dedicated Kubernetes overview page requires that you install an ActiveGate in your environment (version 1.163+).

Set up Kubernetes integration

To connect your Kubernetes clusters to Dynatrace, follow the instructions below.

1. Create a dedicated namespace

$ kubectl create namespace dynatrace

2. Create a service account and cluster role

Create a service account and cluster role for accessing the Kubernetes API. This creates the bearer token necessary to authenticate in the Kubernetes API. Use the following snippet.

$ kubectl apply -f https://www.dynatrace.com/support/help/codefiles/kubernetes/kubernetes-monitoring-service-account.yaml

3. Get the Kubernetes API URL for later use

$ kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}'

4. Get the bearer token for later use

$ kubectl get secret $(kubectl get sa dynatrace-monitoring -o jsonpath='{.secrets[0].name}' -n dynatrace) -o jsonpath='{.data.token}' -n dynatrace | base64 --decode

Special instructions for Rancher distributions

5. Connect your Kubernetes cluster to Dynatrace

You'll need the bearer token and the Kubernetes API URL mentioned above to set up the connection to the Kubernetes API.

  1. Go to Settings > Cloud and virtualization > Kubernetes.
  2. Select Connect new cluster.
  3. Provide a Name, Kubernetes API URL, and the Bearer token for the Kubernetes cluster.

Note: For Rancher distributions, you need the bearer token that was created in Rancher web UI, as described in Special instructions for Rancher distributions above.

6. Configure ActiveGate

If your environment uses proxies or self-signed certificates, you need to adapt your ActiveGate configuration.

Set up Kubernetes workloads

  1. Ensure that the Show workloads and cloud applications toggle is turned on to enable Kubernetes workload ingestion.
  2. In Dynatrace environments earlier than version 1.190, you need to enable Cloud application and workload detection in Process group detection settings. This way, cloud applications and workloads will be detected properly and process groups won't be spread across different cloud applications and workloads.

Integrate Kubernetes events

Events field selectors

Ensure that the Events integration toggle is turned on to enable Kubernetes events ingestion. Also, be sure to specify at least one events field selector. The field selector syntax is the same as the one used in Kubernetes. An event field selector expression can have up to 10 selectors concatenated with a comma. Events matching all comma-separated selectors will be ingested. The logical operator is AND.

events-field-selector

The expression shown in the above example will store all the events related to the namespace hipster-shop that are of type Warning. This is the equivalent of the following command:

kubectl get events --all-namespaces --field-selector involvedObject.namespace=hipster-shop,type=Warning

If you separate the expression into two independent field selectors, you'll get all events for namespace hipster-shop and all events of type Warning. The logical operator is OR.

Events requiring permission

To use this feature, you need the Events watch permission on your service account.

To check if your role has the necessary permission, run the following command:

kubectl auth can-i watch events --as=system:serviceaccount:dynatrace:dynatrace-monitoring

If the output of this command is no, update your service account according to the YAML file provided in step 2.

Troubleshoot

The connection between Dynatrace and your Kubernetes API might fail due to various connectivity issues. Review the following guidelines for errors that might occur.

Monitor large Kubernetes environments

Contact Dynatrace ONE if you want to monitor environments that are larger than:

  • 50 Kubernetes clusters per Dynatrace environment
  • 500 nodes per Kubernetes cluster
  • 50,000 pods per Kubernetes cluster