Set up the Dynatrace GCP log integration in a Kubernetes container (GKE)

Dynatrace version 1.230+

As an alternative to the main deployment, that provides Google Cloud Platform monitoring for both metrics and logs, you can choose to set up monitoring for logs only. In this scenario, you'll run the deployment script in Google Cloud Shell. Instructions will depend on the location where you want the deployment script to run:

• On a new GKE Autopilot cluster created automatically recommended

• On an existing GKE standard or GKE Autopilot cluster

During setup, a new Pub/Sub subscription will be created. GKE will run a log forwarder container. After installation, you'll get logs, dashboards, and alerts for your configured services in Dynatrace.

For other deployment options, see Alternative deployment scenarios.

Limitations

Dynatrace GCP log integration supports maximum 2 GB of data processing per hour.

Prerequisites

To deploy the integration, you need to make sure the following requirements are met on the machine where you are running the installation.

• Linux OS only

• Internet access

• GKE Cluster access

• Dynatrace environment access

  You need to configure the Dynatrace endpoint (environment, cluster or ActiveGate URL) to which the GKE autopilot cluster should send metrics and logs. Make sure that you have direct network access or, if there is a proxy or any other component present in between, that communication is not affected.

Tools

You can deploy the Dynatrace GCP integration in Google Cloud Shell or in bash.

If you use bash, you need to install:

• Google Cloud SDK
• kubectl
• helm
• jq (version 1.6)
• yq (version 4.9.x+)
• curl
• unzip

GCP permissions

Running the deployment script requires a list of permissions. You need to create a custom role (see below) and use it to deploy dynatrace-gcp-monitor.

1. Create a YAML file named dynatrace-gcp-monitor-helm-deployment-role.yaml with the following content:

yamlcopydownload
title: Dynatrace GCP Monitor helm deployment role
description: Role for Dynatrace GCP Monitor helm and pubsub deployment
stage: GA
includedPermissions:
  - container.clusters.get
  - container.configMaps.create
  - container.configMaps.delete
  - container.configMaps.get
  - container.configMaps.update
  - container.deployments.create
  - container.deployments.delete
  - container.deployments.get
  - container.deployments.update
  - container.namespaces.create
  - container.namespaces.get
  - container.pods.get
  - container.pods.list
  - container.secrets.create
  - container.secrets.delete
  - container.secrets.get
  - container.secrets.list
  - container.secrets.update
  - container.serviceAccounts.create
  - container.serviceAccounts.delete
  - container.serviceAccounts.get
  - iam.roles.create
  - iam.roles.list
  - iam.roles.update
  - iam.serviceAccounts.actAs
  - iam.serviceAccounts.create
  - iam.serviceAccounts.getIamPolicy
  - iam.serviceAccounts.list
  - iam.serviceAccounts.setIamPolicy
  - pubsub.subscriptions.create
  - pubsub.subscriptions.get
  - pubsub.subscriptions.list
  - pubsub.topics.attachSubscription
  - pubsub.topics.create
  - pubsub.topics.getIamPolicy
  - pubsub.topics.list
  - pubsub.topics.setIamPolicy
  - pubsub.topics.update
  - resourcemanager.projects.get
  - resourcemanager.projects.getIamPolicy
  - resourcemanager.projects.setIamPolicy
  - serviceusage.services.enable
  - serviceusage.services.get

2. Run the command below, replacing <your_project_ID> with the project ID where you want to deploy the Dynatrace integration.

bashcopydownload
gcloud iam roles create dynatrace_monitor.helm_deployment --project=<your_project_ID> --file=dynatrace-gcp-monitor-helm-deployment-role.yaml

Configure log export

1. Run the following shell script in the GCP project you've selected for deployment.

plaintextcopydownload
wget https://raw.githubusercontent.com/dynatrace-oss/dynatrace-gcp-monitor/master/scripts/deploy-pubsub.sh
chmod +x deploy-pubsub.sh
./deploy-pubsub.sh --topic-name <your-topic-name> --subscription-name <your-subscription-name>

2. Configure log export to send the desired logs to the GCP Pub/Sub topic created above.

Dynatrace permissions

• Create an API token and enable the following permission: Ingest logs (API v2).

Log ingestion

• If you are using Log Monitoring v1, enable the latest version of Dynatrace Log Monitoring.

• Determine where log ingestion will be performed, according to your deployment. This distinction is important when configuring the parameters for this integration.

  • For SaaS deployments: SaaS log ingest, where log ingestion is performed directly through the Cluster API. recommended

  • For Managed deployments: You can use an existing ActiveGate for log ingestion. For information on how to deploy it, see ActiveGate installation.

Install

Complete the steps below to finish your setup.

ownload the Helm deployment package in Google Cloud Shell

Configure parameter values

Connect your Kubernetes cluster

Run the deployment script

Download the Helm deployment package in Google Cloud Shell

bashcopydownload
wget -q "https://github.com/dynatrace-oss/dynatrace-gcp-monitor/releases/latest/download/helm-deployment-package.tar"; tar -xvf helm-deployment-package.tar; chmod +x helm-deployment-package/deploy-helm.sh

Configure parameter values

1. The Helm deployment package contains a values.yaml file with the necessary configuration for this deployment. Go to helm-deployment-package/dynatrace-gcp-monitorand edit the values.yaml file, setting the required and optional parameter values as follows.

   You might want to store this file somewhere for future updates, since it will be needed in case of redeployments. Also, keep in mind that its schema can change. In such case, you should use the new file and only copy over the parameter values.

   Parameter name	Description	Default value
   gcpProjectId	required The ID of the GCP project you've selected for deployment.	Your current project ID
   deploymentType	required Set to 'logs'.	all
   dynatraceAccessKey	required Your Dynatrace API token with the required permissions.
   dynatraceUrl	required For SaaS log ingestion, it's your environment URL (https://<your-environment-id>.live.dynatrace.com). For Managed log ingestion, it's your cluster URL (https:/<your_cluster_IP_or_hostname>/e/<your_environment_ID>). For Managed log ingestion with an existing ActiveGate, it's the URL of your ActiveGate (https://<your_activegate_IP_or_hostname>:9999/e/<your_environment_ID>). Note: To determine <your-environment-id>, see environment ID.
   logsSubscriptionId	required The ID of your log Sink Pub/Sub subscription. For details, see Configure log export.
   requireValidCertificate	optional If set to true, Dynatrace requires the SSL certificate of your Dynatrace environment. For SaaS log ingestion, we recommend leaving the default value. For Managed log ingestion with a new ActiveGate, we recommend setting this value to false.	true
   selfMonitoringEnabled	optional Send custom metrics to GCP to quickly diagnose if dynatrace-gcp-monitor processes and sends logs to Dynatrace properly. For details, see Self-monitoring metrics for the Dynatrace GCP integration.	false
   serviceAccount	optional Name of the service account to be created.
   dockerImage	optionalDynatrace GCP Monitor Docker image. We recommend using the default value, but you can adapt it if needed.	dynatrace/dynatrace-gcp-monitor:v1-latest
   logIngestContentMaxLength	optional The maximum content length of a log event. Should be the same as or lower than the setting on your Dynatrace environment.	8192
   logIngestAttributeValueMaxLength	optional The maximum length of the log event attribute value. If it exceeds the server limit, content will be truncated.	250
   logIngestRequestMaxEvents	optional The maximum number of log events in a single payload to the logs ingestion endpoint. If it exceeds the server limit, payload will be rejected with code 413.	5000
   logIngestRequestMaxSize	optional The maximum size in bytes of a single payload to the logs ingestion endpoint. If it exceeds the server limit, payload will be rejected with code 413.	1048576
   logIngestEventMaxAgeSeconds	optional Determines the maximum age of a forwarded log event. Should be the same as or lower than the setting on your Dynatrace environment.	86400

2. Choose which services you want Dynatrace to monitor.

   By default, the Dynatrace GCP integration starts monitoring a set of selected services. Go to Google Cloud Platform supported service metrics for a list of supported services.

Connect your Kubernetes cluster

• If you want to have a new GKE Autopilot cluster created by the deployment script, add --create-autopilot-cluster to the script. Setting up the connection to the cluster will happen automatically in this case and you can proceed to step 4.
• If you run the deployment script on an existing standard GKE or GKE Autopilot cluster, you can connect to your cluster from the GCP console or via terminal. Follow the instructions below.

shellcopydownload
gcloud container clusters get-credentials <cluster> --region <region> --project <project>

Run the deployment script

• If you run the deployment script on an existing standard GKE or GKE Autopilot cluster, the deployment script will create an IAM service account with the necessary roles and deploy dynatrace-gcp-monitor to your Kubernetes cluster.
• If you run the deployment script with the --create-autopilot-cluster option, the deployment script will automatically create the new GKE Autopilot cluster and deploy dynatrace-gcp-monitor to it.

To run the deployment script, follow the instructions below.

bashcopydownload
cd helm-deployment-package
./deploy-helm.sh

bashcopydownload
cd helm-deployment-package
./deploy-helm.sh [--role-name <role-to-be-created/updated>]

bashcopydownload
cd helm-deployment-package
./deploy-helm.sh --without-extensions-upgrade

bashcopydownload
cd helm-deployment-package
./deploy-helm.sh --create-autopilot-cluster

bashcopydownload
cd helm-deployment-package
./deploy-helm.sh --create-autopilot-cluster --autopilot-cluster-name <name-of-new-cluster>

bashcopydownload
cd helm-deployment-package
./deploy-helm.sh --create-autopilot-cluster --without-extensions-upgrade

Verify installation

To check whether installation was successful

1. Check if the container is running.

   After the installation, it may take couple of minutes until the container is up and running.

   plaintextcopydownload
   kubectl -n dynatrace get pods

2. Check the container logs for errors or exceptions. You have two options:

plaintextcopydownload
kubectl -n dynatrace logs -l app=dynatrace-gcp-monitor -c dynatrace-gcp-monitor-logs

3. Check if dashboards are imported.

   In the Dynatrace menu, go to Dashboards and filter by Tag for Google Cloud. A number of dashboards for Google Cloud Services should be available.

Enable alerting

To activate alerting, you need to enable metric events for alerting in Dynatrace.

To enable metric events

1. In the Dynatrace menu, go to Settings.
2. In Anomaly detection, select Metric events.
3. Filter for GCP alerts and turn on On/Off for the alerts you want to activate.

View logs

After deploying the integration, you can view and analyze GCP logs in the Dynatrace menu, if you go to Logs and filter by cloud.provider: gcp.

Change deployment settings

Change parameters from values.yaml

To load a new values.yaml file, you need to upgrade your Helm release.

To update your Helm release

1. Find out what Helm release version you're using.

   plaintextcopydownload
   helm ls -n dynatrace

2. Run the command below, making sure to replace <your-helm-release> with the value from the previous step.

   plaintextcopydownload
   helm upgrade <your-helm-release> dynatrace-gcp-monitor -n dynatrace

For details, see Helm upgrade.

Change deployment type

To change the deployment type (all, metrics, or logs)

1. Find out what Helm release version you're using.

   plaintextcopydownload
   helm ls -n dynatrace

2. Uninstall the release.

   Be sure to replace <your-helm-release> with the release name from the previous output.

   plaintextcopydownload
   helm uninstall <your-helm-release> -n dynatrace

3. Edit deploymentType in values.yaml with the new value and save the file.

4. Run the deployment command again. For details, see Run the deployment script.

Troubleshoot

To investigate potential deployment and connectivity issues

1. Verify installation
2. Enable self-monitoring optional
3. Check the dynatrace_gcp_<date_time>.log log file created during the installation process.

• This file will be created each time the installation script runs.
• The debug information won't contain sensitive data such as the Dynatrace access key.

4. If you are contacting a Dynatrace product expert via live chat:
   • Make sure to provide the dynatrace_gcp_<date_time>.log log file described in the previous step.
   • Provide version information.
     • For issues during installation, check the version.txt file.
     • For issues during runtime, check container logs.

Uninstall

1. Find out what Helm release version you're using.

plaintextcopydownload
helm ls -n dynatrace

2. Uninstall the release.

plaintextcopydownload
helm uninstall <your-helm-release> -n dynatrace

Alternatively, you can delete the namespace.

bashcopydownload
kubectl delete namespace dynatrace

3. To remove all monitoring assets (dashboards, alerts, etc) from Dynatrace, you need to remove all GCP extensions.

To remove an extension

1. In the Dynatrace menu, go to Extensions and search for the GCP extensions.
2. Select an extension you want to remove, and then select the trash icon in the Actions column to remove it.

Repeat the procedure until you remove all GCP extensions.

Monitoring consumption

DDU consumption applies to cloud Log Monitoring. See DDUs for Log Monitoring for details.