• Home
  • Deploy Dynatrace
  • Set up Dynatrace on container platforms
  • Kubernetes
  • Get started with Kubernetes/OpenShift monitoring
  • Set up application-only monitoring on Kubernetes/OpenShift

Set up application-only monitoring on Kubernetes/OpenShift

Dynatrace supports Full-Stack Monitoring for Kubernetes, from the application down to the infrastructure layer. However, if you don't have access to the infrastructure layer, Dynatrace also provides the option of application-only monitoring. See below for instructions on how to set up Dynatrace to monitor your applications running on Kubernetes.

Note: When deployed in application-only mode, OneAgent monitors the memory, disk, CPU, and networking of processes within the container only. Host metrics aren't monitored.

To integrate OneAgent with Kubernetes applications you have the following options:

  • Automatic injection

  • Pod runtime injection

  • Container build-time injection

    Note: For PPCLE and S390x architectures, pod runtime and container build-time injections are supported.

Automatic application-only injection

This section describes how to deploy OneAgent for application-only monitoring by automatic injection using Dynatrace Operator.

Dynatrace offers the option to inject OneAgent into Kubernetes pods using Kubernetes-native admission controllers. Dynatrace Operator runs a mutating webhook that can modify pods to inject OneAgent by adding an init container. This init container will download the OneAgent package and configure the other containers to be monitored.

Prerequisites

  • Kubernetes version 1.20+ or OpenShift version 4.7+
  • Review the list of supported applications and versions.
  • Review the storage requirements.
  • Review the tokens and permissions required.

Deploy

Automatic application-only monitoring is activated using the Dynatrace Operator applicationMonitoring mode. Instructions for this mode, together with all other modes, are part of Set up Kubernetes monitoring.

Notes:

  • At step 5 of the instructions, make sure you download and configure the applicationMonitoring Dynakube custom resource.
  • To start monitoring, you need to configure monitoring for namespaces and pods.

Limitations

Dynatrace Operator doesn't support OpenShift version 3.11 or earlier.

Update

Each time you want to leverage a new OneAgent version, you only need to redeploy your pods. With automated application monitoring, OneAgent is downloaded and injected within an init container. By default, the latest version of OneAgent is downloaded, but you can control which OneAgent version is downloaded by specifying it with the version parameter of the custom resource.

If you're using CSI drivers, the new OneAgent is downloaded automatically, but you need to recycle the pods to use the latest version.

Uninstall

To uninstall OneAgent from application-only monitoring, simply remove references from your application or Docker image and redeploy the application.

  • To uninstall Dynatrace Operator in deployments where a CSI driver isn't used, run one of the following commands:

    bash
    kubectl delete -f https://github.com/Dynatrace/dynatrace-operator/releases/download/v0.10.3/kubernetes.yaml
    bash
    oc delete -f https://github.com/Dynatrace/dynatrace-operator/releases/download/v0.10.3/openshift.yaml
  • To uninstall Dynatrace Operator in deployments where a CSI driver is used

    1. Remove DynaKube custom resources and clean up all remaining Dynatrace Operator–specific objects.

      bash
      kubectl delete dynakube --all -n dynatrace
      bash
      oc delete dynakube --all -n dynatrace
    2. Restart your monitored applications.

    3. Uninstall Dynatrace Operator and delete the Dynatrace namespace/project.

      bash
      kubectl delete -f https://github.com/Dynatrace/dynatrace-operator/releases/download/v0.10.3/kubernetes-csi.yaml kubectl delete -f https://github.com/Dynatrace/dynatrace-operator/releases/download/v0.10.3/kubernetes.yaml kubectl delete namespace dynatrace
      bash
      oc delete -f https://github.com/Dynatrace/dynatrace-operator/releases/download/v0.10.3/openshift-csi.yaml oc delete -f https://github.com/Dynatrace/dynatrace-operator/releases/download/v0.10.3/openshift.yaml oc delete project dynatrace

Pod runtime injection

Runtime injection puts OneAgent inside a container as it is deployed.

Note: This method of instrumenting applications has limitations in linking Kubernetes workloads with monitored containers/processes. To gain proper relationships and linking, use the automatic application-only injection.

Prerequisites

  • Review the list of supported applications and versions.

  • Create a PaaS token.

  • Storage requirements:

    • ~325 MB for glibc
    • ~290 MB for musl
    • ~650 MB for glibc and musl combined

Deploy

To integrate OneAgent into your application at runtime, select one of the options below, depending on your platform.

OneAgent is made available to the application container via an initContainer—your application image remains unaffected.

To integrate OneAgent into your application at runtime, extend your deployment template as follows.

yaml
# your application containers containers: - name: customer-app image: tomcat env: - name: LD_PRELOAD value: /opt/dynatrace/oneagent/agent/lib64/liboneagentproc.so - name: DT_NETWORK_ZONE value: <your_network_zone> volumeMounts: - mountPath: /opt/dynatrace/oneagent name: oneagent # initcontainer to download OneAgent initContainers: - name: install-oneagent image: alpine:latest command: - /bin/sh args: - -c - ARCHIVE=$(mktemp) && wget -O $ARCHIVE "$DT_API_URL/v1/deployment/installer/agent/unix/paas/latest?Api-Token=$DT_PAAS_TOKEN&$DT_ONEAGENT_OPTIONS" && unzip -o -d /opt/dynatrace/oneagent $ARCHIVE && rm -f $ARCHIVE env: - name: DT_API_URL value: https://<Your-environment-ID>.live.dynatrace.com/api - name: DT_PAAS_TOKEN value: <paastoken> - name: DT_ONEAGENT_OPTIONS value: flavor=<FLAVOR>&include=<TECHNOLOGY> volumeMounts: - mountPath: /opt/dynatrace/oneagent name: oneagent # Make OneAgent available as a volume volumes: - name: oneagent emptyDir: {}
  • In the # initContainer to download OneAgent and # Make OneAgent available as a volume sections, add the initContainer, which will download OneAgent and make it available as a volume.

  • In the DT_ONEAGENT_OPTIONS section, set the OneAgent code module required for your compiler flavor (FLAVOR) and application (TECHNOLOGY).

    • Valid options for flavor are default, musl, or multidistro. Set default to download glibc binaries or set musl to download musl binaries. Set multidistro to download both the musl and glibc binaries and subsequently autodetect which binaries to use. Note that image size will be larger in this case, as it includes both flavors.
    • Valid options for technology are all, java, apache, nginx, nodejs, dotnet, php, go, and sdk.
    • For ARM, use the following value: flavor=default&arch=arm&include=<TECHNOLOGY>. For other architectures, see the list of valid values (scroll down to the arch parameter).
    • If you want to specify several code modules, use the following syntax: &include=technology1&include=technology2.

Note: If you include specific technology-support options rather than 'support for all technologies' options, you'll get a smaller OneAgent package.

What if my Docker image is based on Alpine Linux?
Dynatrace OneAgent supports the flavor musl for Alpine Linux based environments.
Valid options for technology are all, dotnet, go, php, java, apache, nginx, and nodejs.

  • In the # your application containers section, add the newly created volume to the container of your application. Also add the LD_PRELOAD environment variable.

  • optional In the # your application containers section, configure network zones:

yaml
containers: env: - name: DT_NETWORK_ZONE value: <your_network_zone>

See network zones for more information.

  • optional Configure a proxy address.

In case you run an environment with proxy, you need to set the DT_PROXY environment variable in the application container to pass the proxy credentials to OneAgent.

Note: For Alpine Linux-based containers, you might need to update the wget shipped with the Alpine image to allow for proxy authentication for the download of OneAgent.

Extend your deployment template as follows.

Note: This option refers to .NET applications in Windows containers.

yaml
# your application containers apiVersion: apps/v1 kind: Deployment metadata: name: sample labels: app: sample spec: replicas: 1 template: metadata: name: sample labels: app: sample spec: nodeSelector: "kubernetes.io/os": windows containers: - name: sample image: mcr.microsoft.com/dotnet/framework/samples:aspnetapp env: #.NET Framework - name: COR_ENABLE_PROFILING value: "0x01" - name: COR_PROFILER value: "{B7038F67-52FC-4DA2-AB02-969B3C1EDA03}" - name: COR_PROFILER_PATH_32 value: "C:\\oneagent\\agent\\lib\\oneagentloader.dll" - name: COR_PROFILER_PATH_64 value: "C:\\oneagent\\agent\\lib64\\oneagentloader.dll" #.NET Core - name: CORECLR_ENABLE_PROFILING value: "0x01" - name: CORECLR_PROFILER value: "{B7038F67-52FC-4DA2-AB02-969B3C1EDA03}" - name: CORECLR_PROFILER_PATH_32 value: "C:\\oneagent\\agent\\lib\\oneagentloader.dll" - name: CORECLR_PROFILER_PATH_64 value: "C:\\oneagent\\agent\\lib64\\oneagentloader.dll" - name: DT_AGENTACTIVE value: "true" - name: DT_BLOCKLIST value: "powershell*" volumeMounts: - mountPath: "C:\\OneAgent" name: oneagent # initcontainer to download OneAgent initContainers: - name: install-oneagent image: mcr.microsoft.com/windows/servercore:ltsc2019 command: - powershell args: - | Write-Host "Trustng all certificates..." add-type @" using System.Net; using System.Security.Cryptography.X509Certificates; public class TrustAllCertsPolicy : ICertificatePolicy { public bool CheckValidationResult( ServicePoint srvPoint, X509Certificate certificate, WebRequest request, int certificateProblem) { return true; } } "@ [System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy Write-Host "Downloading agent..." Invoke-WebRequest -Uri "$Env:DT_API_URL/v1/deployment/installer/agent/windows/paas/latest?Api-Token=$Env:DT_PAAS_TOKEN&$Env:DT_ONEAGENT_OPTIONS" -OutFile "installer.zip" Write-Host "Unpacking agent..." Expand-Archive -Path "installer.zip" -DestinationPath "C:\OneAgent" -Force Write-Host "Configuring agent..." $manifest = Get-Content "C:\OneAgent\manifest.json" | ConvertFrom-Json $config = New-Item -Path "C:\OneAgent\agent\conf" -Name "standalone.conf" -Force Add-Content -Path $config -Value "tenant $($manifest.tenantUUID)" Add-Content -Path $config -Value "tenanttoken $($manifest.tenantToken)" Add-Content -Path $config -Value "server $($manifest.communicationEndpoints -Join ';')" Add-Content -Path $config -Value "storage C:\OneAgent" Add-Content -Path $config -Value "loglevelcon NONE" env: - name: DT_API_URL value: https://<YourDTTenant>.live.dynatrace.com/api - name: DT_PAAS_TOKEN value: <PaaSToken (Dynatrace -> Settings -> Integration -> Platform as a Service Token) - name: DT_ONEAGENT_OPTIONS value: flavor=x86&include=dotnet volumeMounts: - mountPath: "C:\\OneAgent" name: oneagent # Make OneAgent available as a volume volumes: - name: oneagent emptyDir: {} selector: matchLabels: app: sample --- apiVersion: v1 kind: Service metadata: name: sample spec: type: LoadBalancer ports: - protocol: TCP port: 80 selector: app: sample
  • In the # initContainer to download OneAgent and # Make OneAgent available as a volume sections, add the initContainer, which will download OneAgent and make it available as a volume.
  • The # your application containers section contains environment variables that enable monitoring of .NET Framework and .NET Core applications. They can be set at the same time. For .NET Core, the COR_ prefix changes to CORECLR_, for example CORECLR_ENABLE_PROFILING.

Note: To report the correct memory limits in Kubernetes

  1. You have to specify the limit in the deployment.

    Example:

    plaintext
    spec: containers: ... resources: limits: memory: "32Gi" requests: memory: "4Gi"
  2. You have to enable access to the Kubernetes API so that OneAgent can read that value.

Update

Each time you want to leverage a new OneAgent version, you only need to redeploy your pods. In runtime injections, OneAgent is downloaded and injected within an initContainer. By default, the latest version of OneAgent is downloaded, but you can control which OneAgent version is downloaded by specifying it in the download URL.

Uninstall

To uninstall OneAgent from application-only monitoring

  1. Remove the install-oneagent YAML from your deployment template.
yaml
# your application containers containers: - name: customer-app image: tomcat env: - name: LD_PRELOAD value: /opt/dynatrace/oneagent/agent/lib64/liboneagentproc.so volumeMounts: - mountPath: /opt/dynatrace/oneagent name: oneagent # initContainer to download OneAgent initContainers: - name: install-oneagent image: alpine:3.8 command: - /bin/sh args: - -c - ARCHIVE=$(mktemp) && wget -O $ARCHIVE "$DT_API_URL/v1/deployment/installer/agent/unix/paas/latest?Api-Token=$DT_PAAS_TOKEN&$DT_ONEAGENT_OPTIONS" && unzip -o -d /opt/dynatrace/oneagent $ARCHIVE && rm -f $ARCHIVE env: - name: DT_API_URL value: https://<Your-environment-ID>.live.dynatrace.com/api - name: DT_PAAS_TOKEN value: <paastoken> - name: DT_ONEAGENT_OPTIONS value: flavor=<FLAVOR>&include=<TECHNOLOGY> volumeMounts: - mountPath: /opt/dynatrace/oneagent name: oneagent # Make OneAgent available as a volume volumes: - name: oneagent emptyDir: {}
  1. Redeploy your application.

Container build-time injection

Build-time injection puts OneAgent inside a container as it builds.

Note: This method of instrumenting applications has limitations in linking Kubernetes workloads with monitored containers/processes. To gain proper relationships and linking, use automatic application-only injection.

Prerequisites

  • Docker version 17.05+

  • Review the list of supported applications and versions.

  • Create a PaaS token.

  • Storage requirements:

    • ~325 MB for glibc
    • ~290 MB for musl
    • ~650 MB for glibc and musl combined
  • For ARM architecture, make sure you have wget and unzip installed.

Deploy

To integrate OneAgent into the application image, follow the steps below.

  1. Sign in to Docker with your Dynatrace environment ID as username and PaaS token as password.

    bash
    docker login -u <environmentID> <ADDRESS>
  2. Add the following lines of code to the application image, after the last FROM command:

    bash
    COPY --from=<ADDRESS>/linux/oneagent-codemodules:<TECHNOLOGY> / / ENV LD_PRELOAD /opt/dynatrace/oneagent/agent/lib64/liboneagentproc.so

    where:

  • <ADDRESS> is:

    • EnvironmentActiveGate: <ActiveGateaddress:9999>
    • SAAS: {yourenvid}.live.dynatrace.com
    • Managed: {ManagedAddress}
  • <TECHNOLOGY> is: The OneAgent code module required for your application. Valid options are all, java, apache, nginx, nodejs, dotnet, php, go, and sdk. You can specify several code modules, separated by hyphen (-), for example java-go. Including specific technology-support options, rather than support for all technology options, results in a smaller OneAgent package.

    What if my Docker image is based on Alpine Linux?
    Dynatrace OneAgent supports Alpine Linux based environments.
    Use this syntax:

    bash
    COPY --from=<ACTIVEGATE-ADDRESS>/linux/oneagent-codemodules-musl:<TECHNOLOGY> / / ENV LD_PRELOAD /opt/dynatrace/oneagent/agent/lib64/liboneagentproc.so

    Valid options here are all, dotnet, php, java, apache, nginx, nodejs, and go.

  1. Build your application image.

    Build the Docker image from your Dockerfile to use it in your Kubernetes environment.

    bash
    docker build -t yourapp .

    You can monitor your application containers with a different Dynatrace environment.

    For OneAgent version 1.139 or higher, if you have an existing application image where you have already added the OneAgent code modules for a specific Dynatrace environment, you can have the OneAgent report to another Dynatrace environment without rebuilding your application image.
    For this, you need to make a call to the REST endpoint of your second Dynatrace environment. Don't forget to adapt the respective placeholders <environmentID> and <token>.

    bash
    curl "https://<environmentID>.live.dynatrace.com/api/v1/deployment/installer/agent/connectioninfo?Api-Token=<token>"

    In return, you get a JSON object that covers the required information that needs to be passed as an environment variable to the application container.
    Make sure you set the environment variables of the application container as described below:

    • DT_TENANT: equals tenantUUID
    • DT_TENANTTOKEN: equals tenantToken
    • DT_CONNECTION_POINT: semi-colon separated list of communicationEndpoints
    1. optional Configure network zones

    You can configure network zones as an environment variable:

    • DT_NETWORK_ZONE: equals your.network.zone

    See network zones for more information.

  2. optional Configure a proxy address

    In case you run an environment with proxy, you need to set the DT_PROXY environment variable in the application container to pass the proxy credentials to OneAgent.

    Note: For Alpine Linux-based containers, you might need to update the wget shipped with the Alpine image to allow for proxy authentication for the download of OneAgent.

  1. Define variables with optional default values using ARG instructions, as shown below.

    bash
    ARG DT_API_URL="https://<environmentID>.live.dynatrace.com/api" ARG DT_PAAS_TOKEN="<token>" ARG DT_ONEAGENT_OPTIONS="flavor=default&include=<technology1>&include=<technology2>" ENV DT_HOME="/opt/dynatrace/oneagent"

    Notes:

    • You can override the default values within the OpenShift BuildConfig. Replace <environmentID> with your Dynatrace environment ID. If you’re using Dynatrace Managed, you need to provide your Dynatrace Server URL (https://<YourDynatraceServerURL>/e/<environmentID>/api). Replace <token> with the PaaS token mentioned above.
    • Technology support is enabled via include parameters. Valid options for flavor=default are all, java, apache, nginx, nodejs, dotnet, php, go, and sdk. Including specific technology-support options, rather than support for all technology options, results in a smaller OneAgent package.

    What if my Docker image is based on Alpine Linux?
    OneAgent supports the flavor musl for Alpine Linux based environments. Valid options for flavor=musl are all, java, apache, nginx, and nodejs.

  2. Add the following commands to your current Dockerfile to integrate OneAgent and activate instrumentation of your application.

    bash
    ARG DT_API_URL="https://<environmentID>.live.dynatrace.com/api" ARG DT_API_TOKEN="<token>" ARG DT_ONEAGENT_OPTIONS="flavor=default&include=<technology1>&include=<technology2>" ENV DT_HOME="/opt/dynatrace/oneagent" RUN mkdir -p "$DT_HOME" && \ wget -O "$DT_HOME/oneagent.zip" "$DT_API_URL/v1/deployment/installer/agent/unix/paas/latest?Api-Token=$DT_API_TOKEN&$DT_ONEAGENT_OPTIONS" && \ unzip -d "$DT_HOME" "$DT_HOME/oneagent.zip" && \ rm "$DT_HOME/oneagent.zip" ENV LD_PRELOAD="/opt/dynatrace/oneagent/agent/lib64/liboneagentproc.so"

    Note: The commands above that use wget and unzip might fail if they aren't provided by the base image.

  3. Build your application image.

    In an OpenShift context the above Dockerfile could be used for binary builds as follows:

    bash
    oc new-build --binary --strategy=docker --allow-missing-images yourapp oc patch bc/yourapp --type=json --patch='[{"op":"remove","path":"/spec/strategy/dockerStrategy/from"}]' oc start-build yourapp --from-dir=. --follow

    You can monitor your application containers with a different Dynatrace environment.

    For OneAgent version 1.139 or higher, if you have an existing application image where you have already added the OneAgent code modules for a specific Dynatrace environment, you can have the OneAgent report to another Dynatrace environment without rebuilding your application image.
    For this, you need to make a call to the REST endpoint of your second Dynatrace environment. Don't forget to adapt the respective placeholders <environmentID> and <token>.

    bash
    curl "https://<environmentID>.live.dynatrace.com/api/v1/deployment/installer/agent/connectioninfo?Api-Token=<token>"

    In return, you get a JSON object that covers the required information that needs to be passed as an environment variable to the application container.
    Make sure you set the environment variables of the application container as described below:

    • DT_TENANT: equals tenantUUID
    • DT_TENANTTOKEN: equals tenantToken
    • DT_CONNECTION_POINT: semi-colon separated list of communicationEndpoints
    1. optional Configure network zones

    You can configure network zones as an environment variable:

    • DT_NETWORK_ZONE: equals your.network.zone

    See network zones for more information.

  4. optional Configure a proxy address

    In case you run an environment with proxy, you need to set the DT_PROXY environment variable in the application container to pass the proxy credentials to OneAgent.

    Note: For Alpine Linux-based containers, you might need to update the wget shipped with the Alpine image to allow for proxy authentication for the download of OneAgent.

  1. Set the following build-time variables:
  • $DT_API_URL (The API URL of your Dynatrace environment)
  • $DT_PAAS_TOKEN (The PaaS token to download the code modules)
  • $DT_ONEAGENT_TECHNOLOGY (The module that is downloaded, for example php)
  1. Add the following commands to the Dockerfile:
bash
RUN mkdir -p /opt/dynatrace/oneagent && ARCHIVE=$(mktemp) && wget -O $ARCHIVE "$DT_API_URL/v1/deployment/installer/agent/unix/paas/latest?Api-Token=$DT_PAAS_TOKEN&flavor=default&arch=arm&include=$DT_ONEAGENT_TECHNOLOGY" && unzip -o -d /opt/dynatrace/oneagent $ARCHIVE && rm -f $ARCHIVE ENV LD_PRELOAD /opt/dynatrace/oneagent/agent/lib64/liboneagentproc.so

Update

You need to manually update OneAgent by rebuilding the container image every time a new version of a code module is needed.

Uninstall

To uninstall OneAgent from application-only monitoring

  1. Remove the two lines of code from the application image.

    plaintext
    COPY --from=<ACTIVEGATE-ADDRESS>/linux/oneagent-codemodules:<TECHNOLOGY> / / ENV LD_PRELOAD /opt/dynatrace/oneagent/agent/lib64/liboneagentproc.so
  2. Rebuild the application image.

    plaintext
    docker build -t yourapp .
  1. Remove the following commands from your Dockerfile.

    plaintext
    ARG DT_API_URL="https://<environmentID>.live.dynatrace.com/api" ARG DT_API_TOKEN="<token>" ARG DT_ONEAGENT_OPTIONS="flavor=default&include=<technology1>&include=<technology2>" ENV DT_HOME="/opt/dynatrace/oneagent" RUN mkdir -p "$DT_HOME" && \ wget -O "$DT_HOME/oneagent.zip" "$DT_API_URL/v1/deployment/installer/agent/unix/paas/latest?Api-Token=$DT_API_TOKEN&$DT_ONEAGENT_OPTIONS" && \ unzip -d "$DT_HOME" "$DT_HOME/oneagent.zip" && \ rm "$DT_HOME/oneagent.zip" ENTRYPOINT [ "/opt/dynatrace/oneagent/dynatrace-agent64.sh" ] CMD [ "executable", "param1", "param2" ] # the command of your application, for example, Java
  2. Rebuild the application image.

    plaintext
    docker build -t yourapp .
Related topics
  • Kubernetes/OpenShift monitoring

    Monitor Kubernetes/OpenShift with Dynatrace.

  • OneAgent platform and capability support matrix

    Learn which capabilities are supported by OneAgent on different operating systems and platforms.