Use this extension to ingest metrics from your Databricks Clusters via the embedded Ganglia metric repository.
This OneAgent custom extension allows you to collect metrics from your embedded Ganglia instance on your Databricks Cluster.
This is intended for users who:
This enables you to:
Upload OneAgent Extension Zip file to Dynatrace Cluster.
#settings/monitoredtechnologies/customextensions
Inside the zip file is a dashboard Databricks_Ganglia_Metrics.dashboard.json
which can be uploaded to your Dynatrace Cluster via the import dashboard button from /ui/dashboards
Ensure the EEC is enabled on each host. This can be done globally from /ui/settings/builtin:eec.local
and turning on the first two options.
Create Databricks API Token from inside your Databricks Cluster
Copy your Databricks URL
Copy Linux OA Installation wget
command from #install/agentlinux;gf=all
Create Dynatrace API with ReadConfig
Permissions
Set up Global Init Script on Databricks Cluster
#!/usr/bin/env bash
wget -O Dynatrace-OneAgent-Linux.sh "https://<TENANT>.live.dynatrace.com/api/v1/deployment/installer/agent/unix/default/latest?arch=x86&flavor=default" --header="Authorization: Api-Token <Installer API-TOKEN>"
/bin/sh Dynatrace-OneAgent-Linux.sh --set-infra-only=true --set-app-log-content-access=true --set-host-id-source="ip-addresses"
# token with 'ReadConfig' permissions
wget -O custom_python_databricks_ganglia.zip "https://<TENANT>.live.dynatrace.com/api/config/v1/extensions/custom.python.databricks_ganglia/binary" --header="Authorization: Api-Token <ReadConfig API-TOKEN>"
unzip custom_python_databricks_ganglia.zip -d /opt/dynatrace/oneagent/plugin_deployment/
# Add Databricks Workspace URL Environment Variable
cat <<EOF | sudo tee /etc/databricks_env
DB_WS_URL=https://adb-XXXXXXXXX.XX.azuredatabricks.net
DB_WS_TOKEN=dapiXXXXXXXXXXXXXXXXXXXXXXXXXXX
EOF
Create Dynatrace API Token with entities.read
& entitie.write
permissions.
Configure OneAgent Extension in Dynatrace Cluster from #settings/monitoredtechnologies/customextension;id=custom.python.databricks_ganglia;gf=all
Restart Databricks Cluster(s)
Verify Metrics are showing up on Dashboard