Extend the platform,
empower your team.
Get insights into Google Dataflow service metrics collected from the Google Operations API to ensure health of your cloud infrastructure.
Extension by DynatraceThis Dynatrace extension leverages data collected from the Google Operations API to constantly monitor health and performance of Google Dataflow services. This extension combines all relevant data into pre-configured dashboards and provides alerting and event tracking.
View and analyze 23 metrics that are specific to Google Dataflow, like Billable shuffle data processed, Current number of vCPUs in use, Current shuffle slots in use, and more.
Build custom dashboards for your cloud infrastructure.
Analyze Google Dataflow logs.
Set custom alerts that trigger remediation workflows.
Google Dataflow metric and log ingestion requires advanced GCP integration.
This extension package contains:
To provide correlation and causation analysis all ingested metrics and logs are analyzed by the Dynatrace Davis AI engine, which consumes DDUs.
Below is a complete list of the feature sets provided in this version. To ensure a good fit for your needs, individual metrics can be activated and deactivated by your administrator during configuration.
Metric name | Metric key | Description | Unit |
---|---|---|---|
Billable shuffle data processed | cloud.gcp.dataflow_googleapis_com.job.billable_shuffle_data_processed | - | Byte |
Current number of vCPUs in use | cloud.gcp.dataflow_googleapis_com.job.current_num_vcpus | - | Count |
Current shuffle slots in use | cloud.gcp.dataflow_googleapis_com.job.current_shuffle_slots | - | Count |
Data watermark lag | cloud.gcp.dataflow_googleapis_com.job.data_watermark_age | - | Second |
Elapsed time | cloud.gcp.dataflow_googleapis_com.job.elapsed_time | - | Second |
Element count | cloud.gcp.dataflow_googleapis_com.job.element_count.gauge | - | Count |
Elements Produced | cloud.gcp.dataflow_googleapis_com.job.elements_produced_count | - | Count |
Estimated byte count | cloud.gcp.dataflow_googleapis_com.job.estimated_byte_count.gauge | - | Byte |
Estimated Bytes Produced | cloud.gcp.dataflow_googleapis_com.job.estimated_bytes_produced_count | - | Count |
Failed | cloud.gcp.dataflow_googleapis_com.job.is_failed | - | Count |
Per-stage data watermark lag | cloud.gcp.dataflow_googleapis_com.job.per_stage_data_watermark_age | - | Second |
Per-stage system lag | cloud.gcp.dataflow_googleapis_com.job.per_stage_system_lag | - | Second |
PubsubIO.Read requests from Dataflow jobs | cloud.gcp.dataflow_googleapis_com.job.pubsub.read_count | - | Count |
Pub/Sub Pull Request Latencies | cloud.gcp.dataflow_googleapis_com.job.pubsub.read_latencies | - | MilliSecond |
Pub/Sub Publish Requests | cloud.gcp.dataflow_googleapis_com.job.pubsub.write_count | - | Count |
Pub/Sub Publish Request Latencies | cloud.gcp.dataflow_googleapis_com.job.pubsub.write_latencies | - | MilliSecond |
System lag | cloud.gcp.dataflow_googleapis_com.job.system_lag | - | Second |
Total memory usage time | cloud.gcp.dataflow_googleapis_com.job.total_memory_usage_time | - | GigaByte |
Total PD usage time | cloud.gcp.dataflow_googleapis_com.job.total_pd_usage_time | - | GigaByte |
Total shuffle data processed | cloud.gcp.dataflow_googleapis_com.job.total_shuffle_data_processed | - | Byte |
Total streaming data processed | cloud.gcp.dataflow_googleapis_com.job.total_streaming_data_processed | - | Byte |
Total vCPU time | cloud.gcp.dataflow_googleapis_com.job.total_vcpu_time | - | Second |
User Counter | cloud.gcp.dataflow_googleapis_com.job.user_counter | - | Count |
No release notes
No release notes