How do I monitor Cloud Foundry cluster components and applications?

The following guidelines apply to the deployment of OneAgent to Cloud Foundry VMs, including Cloud Foundry components, Diego cells, and Windows Diego cells.

Before you begin

Check the following requirements:

  • Dynatrace environment credentials
  • BOSH CLI
  • Cloud Foundry deployment with Garden-runC v1.0.0 or higher
  • Dynatrace Garden container monitoring (Enable Dynatrace Garden container monitoring by going to Settings > Monitoring > Monitored technologies. On the Monitored technologies tab, enable the Garden container switch.)

Get your Dynatrace environment ID and PaaS token

The first step is to get your environment ID and the PaaS token for your Dynatrace environment.

  1. Login with your Dynatrace account.

  2. Select Deploy Dynatrace from the navigation menu.

  3. Click the Set up PaaS integration button.

  4. Your environment ID appears in the Environment ID text box. You'll need this ID to link your Dynatrace account with your PaaS environment. Click Copy to copy the ID to the clipboard. You can do this at any time by revisiting this page.

  5. To generate a PaaS token, click the Generate new token button. The PaaS token is essentially an API token that's used in combination with your environment ID to download Dynatrace OneAgent. As you will notice, there is already a default InstallerDownload token available that you could alternatively use. However, for security reasons, it is recommended that you create several discrete tokens for different environments.

  6. Type in a meaningful name for your PaaS token. A meaningful token name might be the name of the PaaS platform you want to monitor (for example, azure, cloud-foundry, or openshift). To view and manage your existing PaaS tokens, go to Settings -> Integration -> Platform as a Service.

  7. Click Generate to create the PaaS token. The newly created PaaS token will appear in the list below. Click Copy to copy the generated token to the clipboard. You can do this at any time by revisiting this page and clicking Show token next to the relevant PaaS token.

Upload and deploy the Dynatrace BOSH add-on

Ensure that your bosh CLI successfully connected to the BOSH director. For details, please refer to Cloud Foundry documentation or Pivotal documentation.

Get the link to the latest BOSH release on Github. You can also run the following command to get the URL of the latest release, provided that you have jq installed.

RELEASE=$(curl -s https://api.github.com/repos/dynatrace/bosh-oneagent-release/releases/latest | jq -r ".assets[] | select(.name) | .browser_download_url")

Upload the BOSH release of Dynatrace OneAgent to the BOSH Director:

  • For Ops Manager v1.10 or earlier:
    bosh upload release PATH-TO-BINARY/dynatrace-release.tar.gz

  • For Ops Manager v1.11 or later:
    bosh2 -e my-env upload-release PATH-TO-BINARY/dynatrace-release.tar.gz

Optionally, from the command line, you can confirm that upload of the Dynatrace software binary is complete. You should see the Dynatrace binary file:

bosh releases

Update your runtime configuration to include the Dynatrace OneAgent for BOSH add-on

Create a Dynatrace OneAgent manifest runtime-config-dynatrace.yml. Please see the sample runtime configuration. The Dynatrace OneAgent release that you uploaded in the previous step consists of two jobs. The dynatrace-oneagent job is applicable for Linux VMs. The dynatrace-oneagent-windows job is for Windows VMs. Since the runtime configuration needs to split jobs for Linux and Windows, you need to define two add-on sections in the manifest file.
Note: If you’ve installed other BOSH add-ons, you must merge the Dynatrace manifest into your existing add-on manifest.

releases:
- name: dynatrace-oneagent
  version: 1.0.3
  #specify version of latest release

addons:
- name: dynatrace-oneagent-addon
  jobs:
  - name: dynatrace-oneagent
    release: dynatrace-oneagent
  include:
    stemcell:
    - os: ubuntu-trusty
  exclude:
    jobs:
    - {name: smoke-tests, release: cf}
    - {name: push-apps-manager, release: push-apps-manager-release}
    - {name: deploy-notifications, release: notifications}
    - {name: deploy-notifications-ui, release: notifications-ui}
    - {name: push-pivotal-account, release: pivotal-account}
    - {name: deploy-autoscaling, release: cf-autoscaling}
    - {name: register-broker, release: cf-autoscaling}
    - {name: nfsbrokerpush, release: nfs-volume}
    - {name: bootstrap, release: cf-mysql}
    - {name: rejoin-unsafe, release: cf-mysql}
    - {name: broker-registrar, release: cf-mysql}
    - {name: deregister-and-purge-instances, release: cf-mysql}
    - {name: smoke-tests, release: cf-mysql}
    - {name: install-hwc-buildpack, release: hwc-buildpack}
  properties:
    dynatrace:
      environmentid: <environmentId>
      apitoken: <token>
      ###
      # optional properties below
      ###
      # Replace with your Dynatrace Managed URL, including the environment ID.
      # An example URL might look like the following
      apiurl: https://{your-managed-cluster.com}/e/{environmentid}/api
      # Set to 'all' if you want to accept all self-signed SSL certificates.
      sslmode: all
      # Specify a direct download URL for Dynatrace OneAgent.
      # If this propery is set, BOSH will download OneAgent from this location.
      downloadurl: https://direct-download-url-for-agent
      # Specify the proxy to be used for communication.
      proxy: https://your-proxy-url

#optional: extra addon configuration for Windows cells
- name: dynatrace-oneagent-windows-addon
  jobs:
  - name: dynatrace-oneagent-windows
    release: dynatrace-oneagent
  include:
    stemcell:
    - os: windows2012R2
  properties:
    dynatrace:
      environmentid: <environmentid>
      apitoken: <token>
      apiurl: https://{your-managed-cluster.com}/e/{environmentid}/api # optional: base URL for API endpoint for Dynatrace Managed

Update the BOSH Director runtime configuration

  • For Ops Manager v1.10 or earlier:
    bosh update runtime-config PATH/runtime-config-dynatrace.yml

  • For Ops Manager v1.11 or later:
    bosh2 -e my-env update-runtime-config PATH/runtime-config-dynatrace.yml

The updated runtime configuration applies to all new BOSH deployments going forward.

Deploy changes and run OneAgent

Since existing BOSH deployments won’t be automatically updated with the jobs specified in the runtime configuration, you need to re-deploy those deployments so that BOSH rolls out OneAgent:

  • For Ops Manager v1.10 or earlier:
    bosh deploy

  • For Ops Manager v1.11 or later:
    bosh2 -e my-env -d deployment deploy

Uninstalling OneAgent with BOSH

Uninstalling the Dynatrace OneAgent for BOSH add-ons requires that you update the runtime configuration with an “empty” manifest and re-deploy all BOSH deployments that the add-ons have executed.
Following is an example empty runtime configuration:

releases:
    - name: dynatrace-onagent
      version: 1.0.3
  #specify version of latest release

Update the runtime configuration without Dynatrace-related jobs.

  • For Ops Manager v1.10 or earlier:
    bosh update runtime-config PATH/runtime-config-uninstall-dynatrace.yml

  • For Ops Manager v1.11 or later:
    bosh2 -e my-env update-runtime-config PATH/runtime-config-uninstall-dynatrace.yml

Limitations

Find out how to solve problems that you may encounter when monitoring (Pivotal) Cloud Foundry foundations in version 1.12.x or earlier.

[PCF PAS 1.12.x or earlier] Outdated nginx upload module causes malfunction of nginx-based Cloud Foundry components

If you’ve enabled monitoring for nginx process groups, HTTP uploads to these nginx instances can lead to failures due to issues in the nginx HTTP-upload module. One example of this problem is cloud_controller. The bug is fixed in CAPI 1.44.0 or later.

In the meantime, please use the following workaround to disable monitoring for nginx process groups:

  1. From the Dynatrace navigation menu, click Settings > Monitoring overview > Process groups.
  2. Search for the nginx process group and disable monitoring of the process group.
  3. Re-start the affected nginx processes.

[PCF PAS 1.11.x or earlier] Apps from Apps Manager may not start properly

If you’ve enabled the errands for Apps Manager in your PCF deployments, starting p-invitations or apps-manager-js-[blue|green] applications can lead to failures due to restrictive memory limits. PCF version 1.12.x deploys these applications with less restrictive memory limits. However, PCF version 2.0.x again relies on restrictive memory limits.

Memory limits have been increased for p-invitations and apps-manager-js* in two phases.

  1. PCF version 1.11.6 or higher: p-invitations memory limit was increased to 256m with the internal push-apps-manager-release-661.1.29-3421.9.0.tgz release.
  2. PCF version 1.12.0 or higher: apps-manager-js-[blue|green] memory limit was increased to 256m with the internal push-apps-manager-release-662.0.11-3445.7.0.tgz release.

If you encounter problems with Apps Manager please use the following workaround to disable monitoring for these apps:

  1. From the Dynatrace navigation menu, click Settings > Monitoring overview > Process groups.
  2. Search for all p-invitations and apps-manager-js-[blue|green] process groups and disable monitoring of these process groups.
  3. Re-run the Apps Manager errands. The Apps Manager deployments should now complete successfully.