Skip to technology filters Skip to main content
Dynatrace Hub

Extend the platform,
empower your team.

Popular searches:
Home hero bg
Google DataflowGoogle Dataflow
Google Dataflow

Google Dataflow

Get insights into Google Dataflow service metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Free trialDocumentation
  • Product information
  • Release notes

Overview

This Dynatrace extension leverages data collected from the Google Operations API to constantly monitor health and performance of Google Dataflow services. This extension combines all relevant data into pre-configured dashboards and provides alerting and event tracking.

This is intended for users, who want to:

  • enrich their monitoring data with metrics and logs from Google Operations API

This enables you to:

  • View and analyze 23 metrics that are specific to Google Dataflow, like Billable shuffle data processed, Current number of vCPUs in use, Current shuffle slots in use, and more.

  • Build custom dashboards for your cloud infrastructure.

  • Analyze Google Dataflow logs.

  • Set custom alerts that trigger remediation workflows.

Google Dataflow metric and log ingestion requires advanced GCP integration.

Compatibility requirements

  • Dynatrace GCP integration version 1.0.0+. To enable metric and log ingestion, see Dynatrace Documentation.
  • Dynatrace version 1.230+

Details

This extension package contains:

  • Configuration for Google Dataflow metric ingest based on your selected feature set (see full list of feature sets and metrics)

To provide correlation and causation analysis all ingested metrics and logs are analyzed by the Dynatrace Davis AI engine, which consumes DDUs.

Get started

To add this extension to your environment:

  • Follow the instructions detailed in Dynatrace Documentation. (Ignore the “Add to environment” button below.)
  • Ensure that you have GCP integration running in your environment and that Google Dataflow service is configured.

Following GCP integration and Google Dataflow configuration:

  • The first data points will be ingested by Dynatrace Davis within ~5 minutes.
  • Explore Google Dataflow metrics in Data Explorer and create custom charts.
  • Set custom events for alerting
Dynatrace
Documentation
By Dynatrace
Dynatrace support center
Subscribe to new releases
Copy to clipboard

Extension content

Content typeNumber of items included
metric metadata
44
generic type
1

Feature sets

Below is a complete list of the feature sets provided in this version. To ensure a good fit for your needs, individual feature sets can be activated and deactivated by your administrator during configuration.

Feature setsNumber of metrics included
Metric nameMetric keyDescriptionUnit
Per-stage backlog in bytescloud.gcp.dataflow_googleapis_com.job.backlog_bytes-Byte
Per-stage backlog in elementscloud.gcp.dataflow_googleapis_com.job.backlog_elements-Count
BigQueryIO.Write Requestscloud.gcp.dataflow_googleapis_com.job.bigquery.write_count-Count
Billable shuffle data processedcloud.gcp.dataflow_googleapis_com.job.billable_shuffle_data_processed-Byte
Bundle user processing latenciescloud.gcp.dataflow_googleapis_com.job.bundle_user_processing_latencies-MilliSecond
Current number of vCPUs in usecloud.gcp.dataflow_googleapis_com.job.current_num_vcpus-Count
Current shuffle slots in usecloud.gcp.dataflow_googleapis_com.job.current_shuffle_slots-Count
Data watermark lagcloud.gcp.dataflow_googleapis_com.job.data_watermark_age-Second
Disk Space Capacitycloud.gcp.dataflow_googleapis_com.job.disk_space_capacity-Byte
Duplicate message count per stagecloud.gcp.dataflow_googleapis_com.job.duplicates_filtered_out_count-Count
Elapsed timecloud.gcp.dataflow_googleapis_com.job.elapsed_time-Second
Element countcloud.gcp.dataflow_googleapis_com.job.element_count.gauge-Count
Elements Producedcloud.gcp.dataflow_googleapis_com.job.elements_produced_count-Count
Estimated byte countcloud.gcp.dataflow_googleapis_com.job.estimated_byte_count.gauge-Byte
Estimated Bytes Producedcloud.gcp.dataflow_googleapis_com.job.estimated_bytes_produced_count-Count
Failedcloud.gcp.dataflow_googleapis_com.job.is_failed-Count
Memory Capacitycloud.gcp.dataflow_googleapis_com.job.memory_capacity-Byte
Per-stage data watermark lagcloud.gcp.dataflow_googleapis_com.job.per_stage_data_watermark_age-Second
Per-stage system lagcloud.gcp.dataflow_googleapis_com.job.per_stage_system_lag-Second
The approximate number of parallel processing keyscloud.gcp.dataflow_googleapis_com.job.processing_parallelism_keys-Count
Pub/Sub Late Messages Countcloud.gcp.dataflow_googleapis_com.job.pubsub.late_messages_count-Count
Pub/Sub Published Messages Countcloud.gcp.dataflow_googleapis_com.job.pubsub.published_messages_count-Count
Pub/Sub Pulled Message Agescloud.gcp.dataflow_googleapis_com.job.pubsub.pulled_message_ages-MilliSecond
PubsubIO.Read requests from Dataflow jobscloud.gcp.dataflow_googleapis_com.job.pubsub.read_count-Count
Pub/Sub Pull Request Latenciescloud.gcp.dataflow_googleapis_com.job.pubsub.read_latencies-MilliSecond
Percentage of active/terminated Streaming Pull connectionscloud.gcp.dataflow_googleapis_com.job.pubsub.streaming_pull_connection_status-Percent
Pub/Sub Publish Requestscloud.gcp.dataflow_googleapis_com.job.pubsub.write_count-Count
Pub/Sub Publish Request Latenciescloud.gcp.dataflow_googleapis_com.job.pubsub.write_latencies-MilliSecond
Statuscloud.gcp.dataflow_googleapis_com.job.status-Unspecified
Storage bytes readcloud.gcp.dataflow_googleapis_com.job.streaming_engine.persistent_state.read_bytes_count-Count
Storage bytes writtencloud.gcp.dataflow_googleapis_com.job.streaming_engine.persistent_state.write_bytes_count-Count
Storage write latenciescloud.gcp.dataflow_googleapis_com.job.streaming_engine.persistent_state.write_latencies-MilliSecond
Per stage end to end latencies.cloud.gcp.dataflow_googleapis_com.job.streaming_engine.stage_end_to_end_latencies-MilliSecond
System lagcloud.gcp.dataflow_googleapis_com.job.system_lag-Second
Thread Timecloud.gcp.dataflow_googleapis_com.job.thread_time-MilliSecond
Timers pending count per stagecloud.gcp.dataflow_googleapis_com.job.timers_pending_count-Count
Timers processed count per stagecloud.gcp.dataflow_googleapis_com.job.timers_processed_count-Count
Total memory usage timecloud.gcp.dataflow_googleapis_com.job.total_memory_usage_time-GigaByte
Total PD usage timecloud.gcp.dataflow_googleapis_com.job.total_pd_usage_time-GigaByte
Total SECU usagecloud.gcp.dataflow_googleapis_com.job.total_secu_usage-Unspecified
Total shuffle data processedcloud.gcp.dataflow_googleapis_com.job.total_shuffle_data_processed-Byte
Total streaming data processedcloud.gcp.dataflow_googleapis_com.job.total_streaming_data_processed-Byte
Total vCPU timecloud.gcp.dataflow_googleapis_com.job.total_vcpu_time-Second
User Countercloud.gcp.dataflow_googleapis_com.job.user_counter-Count

Full version history

To have more information on how to install the downloaded package, please follow the instructions on this page.
ReleaseDate

Full version history

Version 1.0.5

Updated

  • Updated set of supported metrics

Full version history

Version 1.0.4

Updated

  • Feature sets & metrics section in HUB Product Information page

Full version history

Version 1.0.3

Added

  • Release notes to published extension
  • Auto publishing extensions to Hub

Full version history

No release notes

Full version history

No release notes

Dynatrace Hub
Hub homeCertified PartnersGet data into DynatraceBuild your own app
All (715)Log Management and AnalyticsKubernetesAI/ML ObservabilityAutomationsApplication ObservabilityApplication SecurityDigital ExperienceBusiness Analytics
Filter
Type
Built and maintained by
Deployment model
SaaS
  • SaaS
  • Managed
Dynatrace Developer
Filter
Type
Built and maintained by
Deployment model
SaaS
  • SaaS
  • Managed

Infrastructure Observability

76 Results filtered by:

Active Directory extended monitoring logo

Active Directory extended monitoring

Enhance Active Directory services monitoring with advanced metrics

Extension
Active Directory services logo

Active Directory services

Monitor the health and performance of Microsoft Active Directory services.

Extension
Apache CouchDB logo

Apache CouchDB

Intelligently observe and optimize health and performance of your database.

Extension
Cisco Firepower logo

Cisco Firepower

Monitor Cisco Firepower devices using SNMP

Extension
Cloudflare (extension) logo

Cloudflare (extension)

Collect notification history and various metrics from your Cloudflare accounts.

Extension
Control-M Jobs v2 logo

Control-M Jobs v2

Monitor your Control-M jobs execution status.

Extension
Databricks logo

Databricks

Monitor your Databricks Clusters via its multiple APIs!

Extension
Dell EMC PowerMax logo

Dell EMC PowerMax

Monitor PowerMax Arrays for performance and health, with Dell Unisphere API.

Extension
Dell PowerStore logo

Dell PowerStore

Monitor PowerStore Arrays for performance and health, with Dell PowerStore API.

Extension
Elasticsearch logo

Elasticsearch

Monitor Elasticsearch Clusters, Nodes, Indexes, remotely or locally, via API.

Extension
F5 BIG-IP logo

F5 BIG-IP

Gain insights into the performance of your F5 BIG-IP LTM and DNS solutions.

Extension
Fortinet FortiGate logo

Fortinet FortiGate

Gain insights into the performance of your Fortinet FortiGate solution.

Extension
Generic Cisco Device logo

Generic Cisco Device

Monitor Cisco devices using SNMP to feed Dynatrace with metrics and alerts.

Extension
Generic network device logo

Generic network device

Level up your network infrastructure observability by monitoring your network equipment within the Dynatrace platform.

Extension
Google AI Platform logo

Google AI Platform

Get insights into Google AI Platform service metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Google Apigee logo

Google Apigee

Get insights into Google Apigee service metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Google App Engine. Monitor with GCP integration logo

Google App Engine. Monitor with GCP integration

Get insights into Google App Engine service metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Google Assistant Smart Home logo

Google Assistant Smart Home

Get insights into Google Assistant Smart Home metrics collected from the Google Operations API to ensure health of the cloud infrastructure.

Extension
Google Big Query logo

Google Big Query

Get insights into Google BigQuery service metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Google Cloud APIs logo

Google Cloud APIs

Get insights into Google Cloud APIs service metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Google Cloud Bigtable logo

Google Cloud Bigtable

Get insights into Google Cloud Bigtable metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Google Cloud Composer logo

Google Cloud Composer

Get insights into Google Cloud Composer metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Google Cloud Data Loss Prevention logo

Google Cloud Data Loss Prevention

Get insights into Google Data Loss Prevention metrics collected from the Google Operations API to ensure health of the cloud infrastructure.

Extension
Google Cloud DNS logo

Google Cloud DNS

Get insights into Google Cloud DNS service metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Google Cloud Filestore logo

Google Cloud Filestore

Monitor your Google Cloud Filestore services using metrics ingested to your Dynatrace cluster.

Extension
Google Cloud Firestore logo

Google Cloud Firestore

Get insights into Google Cloud Firestore metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Google Cloud Functions logo

Google Cloud Functions

E2E observability for serverless and hybrid environments using Google Functions.

Extension
Google Cloud IoT Core logo

Google Cloud IoT Core

Get insights into Google Cloud IoT Core metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Google Cloud Load Balancing logo

Google Cloud Load Balancing

Get insights into Google Load Balancing metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Google Cloud Router logo

Google Cloud Router

Get insights into Google Cloud Router service metrics collected from the Google Operations API to ensure health of the cloud infrastructure.

Extension
Google Cloud Run logo

Google Cloud Run

Get insights into Google Cloud Run service metrics.

Extension
Google Cloud's operations suite logo

Google Cloud's operations suite

Get insights into Google Cloud's operations suite metrics collected from the Google Operations API to ensure health of cloud infrastructure.

Extension
Google Cloud Spanner logo

Google Cloud Spanner

Get insights into Google Cloud Spanner metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Google Cloud SQL logo

Google Cloud SQL

Get insights into Google Cloud SQL service metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Google Cloud Storage logo

Google Cloud Storage

Get insights into Google Cloud Storage metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Google Cloud Storage Transfer logo

Google Cloud Storage Transfer

Get insights into Google Cloud Storage Transfer metrics collected from the Google Operations API to ensure health of cloud infrastructure.

Extension
Google Compute Engine (GCP integration) logo

Google Compute Engine (GCP integration)

Gather Google Compute Engine metrics collected from the Google Operations API

Extension
Google Dataflow logo

Google Dataflow

Get insights into Google Dataflow service metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Google Dataproc logo

Google Dataproc

Get insights into Google Dataproc service metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Google Firebase logo

Google Firebase

Get insights into Google Firebase service metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Google Firestore in Datastore mode logo

Google Firestore in Datastore mode

Get insights into Google Firestore in Datastore mode metrics collected from the Google Operations API to ensure health of infrastructure.

Extension
Google Hybrid Connectivity logo

Google Hybrid Connectivity

Get insights into Google Hybrid Connectivity metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Google Kubernetes (GKE) GCP integration logo

Google Kubernetes (GKE) GCP integration

Insights into GKE metrics collected from the Google Operations API.

Extension
Google Managed Microsoft AD logo

Google Managed Microsoft AD

Get insights into Google Managed Microsoft AD metrics collected from the Google Operations API to ensure health of the cloud infrastructure.

Extension
Google Memorystore logo

Google Memorystore

Get insights into Google Memorystore service metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Google Network Security logo

Google Network Security

Get insights into Google Network Security metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Google Network Topology logo

Google Network Topology

Get insights into Google Network Topology metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Google Pub/Sub logo

Google Pub/Sub

Get insights into Google Pub/Sub service metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Google Pub/Sub Lite logo

Google Pub/Sub Lite

Get insights into Google Pub/Sub Lite service metrics collected from the Google Operations API to ensure health of the cloud infrastructure.

Extension
Google reCAPTCHA Enterprise logo

Google reCAPTCHA Enterprise

Get insights into Google reCAPTCHA Enterprise metrics collected from the Google Operations API to ensure health of your cloud infrastructure

Extension

Reach out to certified Dynatrace partners to solve your unique use-case

Spica Solutions logo

Spica Solutions

Spindox logo

Spindox

Eviden logo

Eviden

Omnilogy logo

Omnilogy

Matrix logo

Matrix

Phenisys logo

Phenisys