Google Dataproc monitoring
Dynatrace GCP integration leverages data collected from the Google Operation API to constantly monitor health and performance of Google Cloud Platform Services. While combining all relevant data into dashboards, it also enables alerting and event tracking.
Prerequisites
Add services and feature sets optional
After integration, Dynatrace automatically monitors a number of preset GCP services and feature sets (metrics). Besides these, you can add more services or feature sets to monitoring. For details, see Add or remove services.
For a list of feature sets available for this service, see Metric table.
Extensions for GCP services
After adding a service to monitoring, it appears as an enabled extension in Dynatrace. Then, your dashboards and metric events are instantly available in your Dynatrace environment. To see the list of extensions, in the Dynatrace menu, go to Extensions.
Note: If you want to stop a service from sending metrics to Dynatrace, removing the relevant extension isn't enough. You also need to remove the service from monitoring. For details, see Add or remove services.
View metrics
After deploying the integration, you can see metrics from monitored services in the Metrics browser, the Data explorer, and your dashboard tiles.
Metric table
The following feature sets are available for Google Dataproc.
Feature set | Name | Unit | GCP metric identifier |
---|---|---|---|
cloud_dataproc_cluster/default_metrics | HDFS DataNodes | Count | dataproc.googleapis.com/cluster/hdfs/datanodes |
cloud_dataproc_cluster/default_metrics | HDFS capacity | GibiByte | dataproc.googleapis.com/cluster/hdfs/storage_capacity |
cloud_dataproc_cluster/default_metrics | HDFS storage utilization | Count | dataproc.googleapis.com/cluster/hdfs/storage_utilization |
cloud_dataproc_cluster/default_metrics | Unhealthy HDFS blocks by status | Count | dataproc.googleapis.com/cluster/hdfs/unhealthy_blocks |
cloud_dataproc_cluster/default_metrics | Job duration | Second | dataproc.googleapis.com/cluster/job/completion_time |
cloud_dataproc_cluster/default_metrics | Job state duration | Second | dataproc.googleapis.com/cluster/job/duration |
cloud_dataproc_cluster/default_metrics | Failed jobs | Count | dataproc.googleapis.com/cluster/job/failed_count |
cloud_dataproc_cluster/default_metrics | Running jobs | Count | dataproc.googleapis.com/cluster/job/running_count |
cloud_dataproc_cluster/default_metrics | Submitted jobs | Count | dataproc.googleapis.com/cluster/job/submitted_count |
cloud_dataproc_cluster/default_metrics | Operation duration | Second | dataproc.googleapis.com/cluster/operation/completion_time |
cloud_dataproc_cluster/default_metrics | Operation state duration | Second | dataproc.googleapis.com/cluster/operation/duration |
cloud_dataproc_cluster/default_metrics | Failed operations | Count | dataproc.googleapis.com/cluster/operation/failed_count |
cloud_dataproc_cluster/default_metrics | Running operations | Count | dataproc.googleapis.com/cluster/operation/running_count |
cloud_dataproc_cluster/default_metrics | Submitted operations | Count | dataproc.googleapis.com/cluster/operation/submitted_count |
cloud_dataproc_cluster/default_metrics | YARN allocated memory percentage | Count | dataproc.googleapis.com/cluster/yarn/allocated_memory_percentage |
cloud_dataproc_cluster/default_metrics | YARN active applications | Count | dataproc.googleapis.com/cluster/yarn/apps |
cloud_dataproc_cluster/default_metrics | YARN containers | Count | dataproc.googleapis.com/cluster/yarn/containers |
cloud_dataproc_cluster/default_metrics | YARN memory size | GibiByte | dataproc.googleapis.com/cluster/yarn/memory_size |
cloud_dataproc_cluster/default_metrics | YARN NodeManagers | Count | dataproc.googleapis.com/cluster/yarn/nodemanagers |
cloud_dataproc_cluster/default_metrics | YARN pending memory size | GibiByte | dataproc.googleapis.com/cluster/yarn/pending_memory_size |
cloud_dataproc_cluster/default_metrics | YARN virtual cores | Count | dataproc.googleapis.com/cluster/yarn/virtual_cores |