Databricks Workspace

Databricks Workspace

Remotely monitor your Databricks Workspaces!

Extension

Product information
Release notes

Overview

With Dynatrace, you can remotely monitor your Databricks Workspaces. This extension works in harmony with the OneAgent based Databricks extension but, is also ideal for workspaces and clusters where the OneAgent cannot be installed such as Databricks Serverless compute.

Use cases

Gather Databricks Job Run metrics including success rate and job duration
For Databricks Jobs running on All-purpose and Job Compute Clusters understand cost of these jobs (currently Azure Databricks is supported)
Ingest Job and Task run information as traces allowing for further analysis

Get started

Install Dynatrace Environment ActiveGate
Ensure connectivity between this ActiveGate and your Databricks Workspace URL
Create a Databricks Access Token for your Databricks Workspace
Create a Dynatrace Access Token with the openTelemetryTrace.ingest scope
Create a New Monitoring Configuration in Dynatrace, using the URL and tokens above

Details

This extension remotely queries the Databricks List Job Runs and Get cluster info APIs using the provided Databricks URL and Access token. With that information, it calculates and reports the various metrics selected from the Feature Sets in the Dynatrace Monitoring configuraiton.

If trace ingestion is configured, the extension will transform the data from the Databricks APIs into OpenTelemetry traces with the Job as the parent span and the tasks in that job as child spans.

NOTE: Only once a job is completed will it's metrics and trace be ingested, meaning that as the jobs are running, data about it is not ingested.

Compatibility information

Databricks API version 2.1 is used for the APIs below :

For the databricks.job.cost metric, currently only Azure Databricks workspaces are supported.

License Consumption

If all the Feature Sets are enabled, the number of metric datapoints is :

7 * # of Jobs

If traces are configured to be ingested, the number of spans is :

# of Jobs * (1 + Tasks per Job)

Subscribe to new releases

Copy to clipboard

Extension content

Content typeNumber of items included

metric metadata

7

document dashboard

1

dashboards

1

Feature sets

Below is a complete list of the feature sets provided in this version. To ensure a good fit for your needs, individual feature sets can be activated and deactivated by your administrator during configuration.

Feature setsNumber of metrics included

Metric name	Metric key	Description	Unit
Job Run Duration	databricks.job.duration.run	-	MilliSecond
Job Success Rate	databricks.job.success_rate	-	Percent

Metric name	Metric key	Description	Unit
Job Setup Duration	databricks.job.duration.setup	-	MilliSecond
Job Excecution Duration	databricks.job.duration.execution	-	MilliSecond
Job Cleanup Duration	databricks.job.duration.cleanup	-	MilliSecond
Job Queue Duration	databricks.job.duration.queue	-	MilliSecond

Metric name	Metric key	Description	Unit
Job Cost (Approx)	databricks.job.cost	-	Unspecified

Related to Databricks Workspace

Databricks

Monitor your Databricks Clusters via its multiple APIs!

Full version history

To have more information on how to install the downloaded package, please follow the instructions on this page.

ReleaseDate

Full version history

V1.0.1

Initial version with updated Platform Dashboard link

Discover recent additions to Dynatrace

Problems

Analyze abnormal system behavior and performance problems detected by Davis AI.

Logs

Explore all your logs without writing a single query.

Security Investigator

Fast and precise forensics for security and logs on Grail data with DQL queries.

Business Flow

Track, analyze, and optimize your critical business processes.

Cost & Carbon Optimization

Track, analyze, and optimize your IT carbon footprint and public cloud costs.

Davis Anomaly Detection

Detect anomalies in timeseries using the Davis AI

Analyze your data

Understand your data better with deep insights and clear visualizations.

Notebooks

Create powerful, data-driven documents for custom analytics and collaboration.

Dashboards

Transform complex data into clear visualizations with custom dashboards.

Automate your processes

Turn data and answers into actions, securely, and at scale.

Workflows

Automate tasks in your IT landscape, remediate problems, and visualize processes

Jira

Create, query, comment, transition, and resolve Jira tickets within workflows.

Slack

Automate Slack messaging for security incidents, attacks, remediation, and more.

Secure your cloud application

See vulnerabilities and attacks in your environment.

Security Overview

Get a comprehensive overview of the security of your applications.

Code-Level Vulnerabilities

Detect vulnerabilities in your code in real time.

Threats & Exploits

Understand, triage, and investigate application security findings and alerts.

Are you looking for something different?

We have hundreds of apps, extensions, and other technologies to customize your environment

Leverage our newest innovations of Dynatrace Saas

Kick-start your app creation

Kick-start your app creation

Whether you’re a beginner or a pro, Dynatrace Developer has the tools and support you need to create incredible apps with minimal effort.

Go to Dynatrace Developer

Upgrading from Dynatrace Managed to SaaS

Upgrading from Dynatrace Managed to SaaS

Drive innovation, speed, and agility in your organization by seamlessly and securely upgrading.

Log Management and Analytics

Log Management and Analytics

Innovate faster and more efficiently with unified log management and log analytics for actionable insights and automation.