Configuration options for Dynatrace Operator on Kubernetes/OpenShift

See below for a list of configuration options available for Dynatrace Operator.

Add a custom properties file optional

As part of getting started with Kubernetes monitoring, you may want to add a custom properties file. When you are finished, you can return to the installation instructions for your kubectl/oc or helm deployment.

You can add a custom properties file by providing it as a value or by referencing it from a secret.

• To add the custom properties file as a value, see the example below.

yamlcopydownload
customProperties:
    value: |
      [kubernetes_monitoring]
      ...

• To reference the custom properties file from a secret

1. Create a secret with the following content.

Note: The content of the secret has to be base64 encoded in order to work.

yamlcopydownload
apiVersion: v1
kind: Secret
metadata:
  name: <customproperties-secret>
  namespace: dynatrace
data:
  customProperties: <base64 encoded properties>

2. Add the secret to the custom properties.

yamlcopydownload
customProperties:
    valueFrom: <customproperties-secret>

Add a custom certificate for ActiveGate optional

As part of getting started with Kubernetes monitoring, you may want to add a custom certificate for ActiveGate. When you are finished, you can return to the installation instructions for your kubectl/oc or helm deployment.

To add a custom certificate for ActiveGate:

1. Create a secret.

bashcopydownload
kubectl -n dynatrace create secret generic dynakube-custom-certificate --from-file=server.p12 --from-literal=password=<password_to_server.p12> --from-file=server.crt

bashcopydownload
oc -n dynatrace create secret generic dynakube-custom-certificate --from-file=server.p12 --from-literal=password=<password_to_server.p12> --from-file=server.crt

2. In your custom resource, enable the tlsSecretName parameter and enter the name of the secret you created.

Example:

jsoncopydownload
apiVersion: dynatrace.com/v1beta1
kind: DynaKube
metadata:
  name: dynakube
  namespace: dynatrace
spec:
  apiUrl: https://FQDN/api
  activeGate:
    tlsSecretName: dynakube-custom-certificate
    capabilities:
      - kubernetes-monitoring

Note: HTTP clients connecting to the ActiveGate REST endpoint must trust provided certificates.

Configure proxy optional

As part of getting started with Kubernetes monitoring, you may want to configure proxy. When you are finished, you can return to the installation instructions for your kubectl/oc or helm deployment.

You can configure optional parameters like proxy settings in the DynaKube custom resource file in order to:

• Download the OneAgent installer
• Ensure communication between the OneAgent and your Dynatrace environment
• Ensure communication between Dynatrace Operator and the Dynatrace API.

There are two ways to provide the proxy, depending on whether your proxy uses credentials.

copydownload
apiVersion: dynatrace.com/v1beta1
kind: DynaKube
metadata:
  name: dynakube
  namespace: dynatrace
spec:
  apiUrl: https://environmentid.live.dynatrace.com/api
  proxy:
    value: http://mysuperproxy

Read-only file systems support

As part of getting started with Kubernetes monitoring, you may want to review the support for read-only file systems. When you are finished, you can return to the installation instructions for your kubectl/oc or helm deployment.

For read-only host file systems, support is enabled by default for cloudNativeFullStack and hostMonitoring with CSI driver configurations, so you don't need to set the ONEAGENT_ENABLE_VOLUME_STORAGE environment variable to true anymore.

To disable this feature, you can add the following annotation in your DynaKube custom resource.

yamlcopydownload
apiVersion: dynatrace.com/v1beta1
kind: DynaKube
metadata:
  name: dynakube
  namespace: dynatrace
  annotations:
    feature.dynatrace.com/disable-oneagent-readonly-host-fs: "true"

Configure monitoring for namespaces and pods

As part of getting started with Kubernetes monitoring, you may want to configure monitoring for namespaces and pods. When you are finished, you can return to the installation instructions for your kubectl/oc or helm deployment.

By default, Dynatrace Operator injects OneAgent into all namespaces, with the following exceptions:

• Namespaces starting with kube-or openshift-.
• The namespace where Dynatrace Operator was installed.

For more configuration options, see below.

• Option 1: Monitor all namespaces except selected pods.

To disable monitoring for selected pods, annotate the pods that should be excluded, as in the example below.

yamlcopydownload
...
metadata:
  annotations:
    ...
    oneagent.dynatrace.com/inject: "false"

For more pod annotation options, see Pod annotation list.

• Option 2: Monitor only specific namespaces.

If you don't want Dynatrace Operator to inject OneAgent in all namespaces, you can set the namespaceSelector parameter in the DynaKube custom resource, and enable monitoring for specific namespaces that have the chosen label.

To label namespaces, use the command below, making sure to replace the placeholder with your own value.

shcopydownload
kubectl label namespace <my_namespace> monitor=app

shcopydownload
oc label namespace <my_namespace> monitor=app

To enable monitoring for the namespace that was just labelled, edit the DynaKube custom resource file as in the example below.

yamlcopydownload
...
namespaceSelector:
  matchLabels:
    monitor: app

For details, see Labels and selectors.

Note: To add exceptions for specific pods within the selected namespaces, you can annotate the respective pods.

• Option 3: Exclude specific namespaces from being monitored.

To enable this option, edit the DynaKube custom resource file as in the example below. Note that

• key is the key of the label, for example monitor.
• value is the value of the label, for example app.

yamlcopydownload
...
spec:
 namespaceSelector:
   matchExpressions:
    - key: KEY
      operator: NotIn
      values:
       - VALUE

The webhook will inject every namespace that matches all namespaceselector.

The operator property can have the following values: In and NotIn.

• If you set In, the webhook will only inject the pods in the namespace that matches the namespace selector.
• If you set NotIn, the webhook will only inject the pods in all other namespaces that don't match the namespace selector. –> For details, see Resources that support set-based requirements.

Pod annotation list

• All applicable pod annotations for applicationMonitoring without CSI driver:

  • oneagent.dynatrace.com/inject: <"false">. If set to false, no modifications will be applied to the pod. If not set, the default on the namespace is used.
  • oneagent.dynatrace.com/flavor: <"default"> or <"musl">. If set, it indicates whether binaries for glibc or musl are to be downloaded. It defaults to glibc.
    Note: If your container uses musl (for example, Alpine base image), you must add the flavor annotation in order to monitor it.
  • oneagent.dynatrace.com/technologies: <"comma-separated technologies list">. If set, it filters which code modules are to be downloaded. It defaults to "all".
  • oneagent.dynatrace.com/install-path: <"path">. If set, it indicates the path where the unpacked OneAgent directory will be mounted. It defaults to "/opt/dynatrace/oneagent-paas".
  • oneagent.dynatrace.com/installer-url: <"url">. If set, it indicates the URL from where the OneAgent app-only package will be downloaded. It defaults to the Dynatrace environment API configured on the API URL of Dynakube.

• All applicable pod annotations for applicationMonitoring with CSI driver:

  • oneagent.dynatrace.com/inject: <"false">. If set to false, no modifications will be applied to the pod. If not set, the default on the namespace is used.

Example annotations:

yamlcopydownload
...
metadata:
  annotations:
    oneagent.dynatrace.com/technologies: "java,nginx"
    oneagent.dynatrace.com/flavor: "musl"
    oneagent.dynatrace.com/install-path: "/dynatrace"
    oneagent.dynatrace.com/installer-url: "https://my-custom-url/route/file.zip"

Import Kubernetes API certificates

As part of getting started with Kubernetes monitoring, you may want to check how importing Kubernetes API certificates works. When you are finished, you can return to the installation instructions for your kubectl/oc or helm deployment.

Starting with Dynatrace Operator version 0.3.0, Kubernetes API certificates are automatically imported for certification validation checks. Kubernetes automatically creates a kube-root-ca.crt configmap in every namespace. This certificate is automatically mounted into every container to /var/run/secrets/kubernetes.io/serviceaccount/ca.crt and merged into the ActiveGate truststore file using an initContainer. To get this feature, be sure to update Dynatrace Operator if you're using an earlier version.

Configure security context constraints (OpenShift)

As part of getting started with Kubernetes monitoring, you may want configure security context constraints for OpenShift. When you are finished, you can return to the installation instructions for your kubectl/oc or helm deployment.

Note: Configuring security context constraints is required for OpenShift for cloudNativeFullStack and applicationMonitoring with CSI driver deployments.

Dynatrace Operator needs permission to access the csi volumes, which are used to provide the necessary binaries to different pods. To allow pods access to the csi volumes you must add a security context constraint.

To add a security context constraint

1. Create a file called restricted-csi.yaml with the following content.

Note: You can configure the file according to your needs, just make sure you add csi to the volumes.

yamlcopydownload
apiVersion: security.openshift.io/v1
kind: SecurityContextConstraints
metadata:
  name: restricted-csi
runAsUser:
  type: MustRunAsRange
seLinuxContext:
  type: MustRunAs
fsGroup:
  type: MustRunAs
supplementalGroups:
  type: RunAsAny
allowHostDirVolumePlugin: true
allowHostIPC: false
allowHostNetwork: false
allowHostPID: false
allowHostPorts: false
allowPrivilegeEscalation: true
allowPrivilegedContainer: true
allowedCapabilities: null
defaultAddCapabilities: null
priority: null
readOnlyRootFilesystem: false
groups:
- system:authenticated
requiredDropCapabilities:
- KILL
- MKNOD
- SETUID
- SETGID
users: []
volumes:
- configMap
- downwardAPI
- emptyDir
- hostPath
- persistentVolumeClaim
- projected
- secret
- csi

2. Save the file.
3. Run the command below to create the security context constraint.

bashcopydownload
oc apply -f restricted-csi.yaml

Metadata metric enrichment

As part of getting started with Kubernetes monitoring, you may want configure metadata metric enrichment. When you are finished, you can return to the installation instructions for your kubectl/oc or helm deployment.

Metadata metric enrichment leverages data from OneAgent and Dynatrace Operator by adding additional context or relevant data to the metrics sent. Enrichment means the logs and data are related back to entities (pods, processes, hosts). Every metric prefixed with dt.entity is due to metadata enrichment.

Starting with Dynatrace Operator version 0.4+, every application pod that is instrumented by the Dynatrace Webhook is automatically enriched with metric metadata.

Activate metadata enrichment

To activate metadata enrichment, you need to create a special token for data ingest and add it to the secret.

1. Create a dataIngestToken token and enable the Ingest metrics permission (API v2).
2. Follow the deployment instructions, making sure the dynakube secret you create in step 4 of the instructions includes the dataIngestToken token.
3. Redeploy your monitored pods.

Note: You can add the dataIngestToken token manually at any time by editing the secret:

1. Edit the existing secret.

bashcopydownload
kubectl edit secret <dynakube>

bashcopydownload
oc edit secret <dynakube>

2. Add a new dataIngestToken key with your generated token to the secret, as in the example below:

yamlcopydownload
apiVersion: v1
kind: Secret
metadata:
  name: dynakube
  namespace: dynatrace
data:
  apiToken: <apiToken base64 encoded>
  dataIngestToken: <dataIngestToken base64 encoded>
type: Opaque

3. Redeploy your monitored pods.

Disable metadata enrichment

To disable the metadata enrichments, add the following annotation to the DynaKube custom resource:

yamlcopydownload
metadata:
  annotations:
    ...
    feature.dynatrace.com/disable-metadata-enrichment: "true"

Alternatively, you can disable the metadata enrichments by running the command below. Be sure to replace the placeholder (<...>) with the name of your DynaKube sample.

bashcopydownload
kubectl annotate dynakube -n dynatrace <your_DynaKube_CR> feature.dynatrace.com/disable-metadata-enrichment="true"

bashcopydownload
oc annotate dynakube -n dynatrace <your_DynaKube_CR> feature.dynatrace.com/disable-metadata-enrichment="true"

Enable AppArmor for enhanced security

As part of getting started with Kubernetes monitoring, you may want to enable AppArmor for enhanced security. When you are finished, you can return to the installation instructions for your kubectl/oc or helm deployment.

You can make Dynatrace Operator more secure by enabling AppArmor. Depending on whether you set up monitoring using kubectl/oc or helm, select one of the options below.

yamlcopydownload
apiVersion: dynatrace.com/v1beta1
kind: DynaKube
metadata:
  annotations:
    feature.dynatrace.com/activegate-apparmor: true

yamlcopydownload
kind: Deployment
metadata:
  name: dynatrace-webhook
spec:
  template:
    metadata:
      annotations:
        container.apparmor.security.beta.kubernetes.io/webhook: runtime/default

kind: Deployment
metadata:
  name: dynatrace-operator
spec:
  template:
    metadata:
      annotations:
        container.apparmor.security.beta.kubernetes.io/dynatrace-operator: runtime/default

yamlcopydownload
operator:
  apparmor: true

webhook:
  apparmor: true

activeGate:
  apparmor: true

High availability mode for Helm deployments

As part of getting started with Kubernetes monitoring, you may want to configure high availability. When you are finished, you can return to the installation instructions for your kubectl/oc or helm deployment.

Note: For now, this feature is limited to Helm deployments.

The high availability mode offers the following capabilities:

• Increases replicas to two replicas for webhook deployment.
• Adds pod topology spread constraints:
  • Pods are spread across different nodes, with the nodes in different zones where possible.
  • Multiple pods are allowed in the same zone.
• Adds pod disruption budget:
  • It restricts graceful shutdowns of the webhook pod, if it's the last remaining pod.

To enable this, you can add the following to the values.yaml:

yamlcopydownload
webhook:
  highAvailability: true