Skip to technology filters Skip to main content
Dynatrace Hub

Extend the platform,
empower your team.

Popular searches:
Home hero bg
HadoopHadoop
Hadoop

Hadoop

Enhanced insights for Hadoop HDFS and Yarn services

Extension
Free trialDocumentation
Overview DashboardExtension configuration and featuresetsExtra Hadoop metricsHadoop HDFS service entityHadoop Yarn service entityProcess Group Injection
  • Product information
  • Release notes

Overview

Hadoop monitoring in Dynatrace provides a high-level overview of the main Hadoop components within your cluster. These enhanced insights provides additional metrics directly from the HDFS and YARN services.

Use cases

The extension enables insights into the overall health of Hadoop HDFS and YARN services

Compatibility information

  • Hadoop version 2.4.1+
  • Linux OS
  • For full Hadoop visibility, OneAgent must be installed on all machines running the following Hadoop processes: NameNode, ResourceManager, NodeManager, DataNode, and MRAppMaster
Dynatrace
Documentation
By Dynatrace
Dynatrace support center
Subscribe to new releases
Copy to clipboard

Extension content

Content typeNumber of items included
screen injections
2
generic relationship
1
document dashboard
1
screen dql table
1
dashboards
1
screen entities lists
1
generic type
1
metric metadata
91

Feature sets

Below is a complete list of the feature sets provided in this version. To ensure a good fit for your needs, individual feature sets can be activated and deactivated by your administrator during configuration.

Feature setsNumber of metrics included
Metric nameMetric keyDescriptionUnit
AllocatedContainershadoop.yarn.resourcemanager.AllocatedContainersQueue Current number of allocated containersCount
AllocatedMBhadoop.yarn.resourcemanager.AllocatedMBCurrent allocated memory in MBMegaByte
AllocatedVCoreshadoop.yarn.resourcemanager.AllocatedVCoresCurrent allocated CPU in virtual coresCount
AppsCompletedhadoop.yarn.resourcemanager.AppsCompleted.countTotal number of completed applicationsCount
AppsFailedhadoop.yarn.resourcemanager.AppsFailed.countTotal number of failed applicationsCount
AppsKilledhadoop.yarn.resourcemanager.AppsKilled.countTotal number of killed applicationsCount
AppsPendinghadoop.yarn.resourcemanager.AppsPending.countCurrent number of applications that have not yet been assigned by any containersCount
AppsRunninghadoop.yarn.resourcemanager.AppsRunning.countCurrent number of running applicationsCount
AppsSubmittedhadoop.yarn.resourcemanager.AppsSubmitted.countTotal number of submitted applicationsCount
AvailableMBhadoop.yarn.resourcemanager.AvailableMBCurrent available memory in MBCount
AvailableVCoreshadoop.yarn.resourcemanager.AvailableVCoresCurrent available CPU in virtual coresCount
PendingMBhadoop.yarn.resourcemanager.PendingMBCurrent memory requests in MB that are pending to be fulfilled by the schedulerMegaByte
PendingVCoreshadoop.yarn.resourcemanager.PendingVCoresCurrent CPU requests in virtual cores that are pending to be fulfilled by the schedulerCount
ReservedMBhadoop.yarn.resourcemanager.ReservedMBCurrent reserved memory in MBMegaByte
ReservedVCoreshadoop.yarn.resourcemanager.ReservedVCoresCurrent reserved CPU in virtual coresCount
Metric nameMetric keyDescriptionUnit
NumberOfMissingBlockshadoop.hdfs.namenode.NumberOfMissingBlocks-Count
CacheCapacityhadoop.hdfs.namenode.CacheCapacityThe total cache capacity of all DataNodesByte
CacheUsedhadoop.hdfs.namenode.CacheUsedThe total cache used by all DataNodesByte
Metric nameMetric keyDescriptionUnit
NumActiveNMshadoop.yarn.resourcemanager.NumActiveNMsCurrent number of active NodeManagersCount
NumDecommissioningNMshadoop.yarn.resourcemanager.NumDecommissionedNMsCurrent number of NodeManagers being decommissionedCount
NumLostNMshadoop.yarn.resourcemanager.NumLostNMsCurrent number of lost NodeManagers for not sending heartbeatsCount
NumRebootedNMshadoop.yarn.resourcemanager.NumRebootedNMsCurrent number of rebooted NodeManagersCount
NumUnhealthyNMshadoop.yarn.resourcemanager.NumUnhealthyNMsCurrent number of unhealthy NodeManagersCount
Metric nameMetric keyDescriptionUnit
FilesTotalhadoop.hdfs.namenode.FilesTotalCurrent number of files and directoriesCount
PendingReplicationBlockshadoop.hdfs.namenode.PendingReplicationBlocksCurrent number of blocks pending to be replicatedCount
UnderReplicatedBlockshadoop.hdfs.namenode.UnderReplicatedBlocksCurrent number of blocks under replicatedCount
ScheduledReplicationBlockshadoop.hdfs.namenode.ScheduledReplicationBlocksCurrent number of blocks scheduled for replicationsCount
NumLiveDataNodeshadoop.hdfs.namenode.NumLiveDataNodesNumber of datanodes which are currently liveCount
NumDeadDataNodeshadoop.hdfs.namenode.NumDeadDataNodesNumber of datanodes which are currently deadCount
NumDecomLiveDataNodeshadoop.hdfs.namenode.NumDecomLiveDataNodesNumber of datanodes which have been decommissioned and are now liveCount
NumDecomDeadDataNodeshadoop.hdfs.namenode.NumDecomDeadDataNodesNumber of datanodes which have been decommissioned and are now deadCount
VolumeFailuresTotalhadoop.hdfs.namenode.VolumeFailuresTotalTotal number of volume failures across all DatanodesCount
EstimatedCapacityLostTotalhadoop.hdfs.namenode.EstimatedCapacityLostTotalAn estimate of the total capacity lost due to volume failuresCount
NumDecommissioningDataNodeshadoop.hdfs.namenode.NumDecommissioningDataNodesNumber of datanodes in decommissioning stateCount
NumStaleDataNodeshadoop.hdfs.namenode.NumStaleDataNodesNumber of datanodes marked stale due to delayed hearbeat.Count
Metric nameMetric keyDescriptionUnit
JobsCompletedhadoop.yarn.mrappmaster.JobsCompletedNumber of completed jobsCount
JobsFailedhadoop.yarn.mrappmaster.JobsFailedNumber of failed jobsCount
JobsKilledhadoop.yarn.mrappmaster.JobsKilledNumber of killed jobsCount
JobsPreparinghadoop.yarn.mrappmaster.JobsPreparingNumber of preparing jobsCount
JobsRunninghadoop.yarn.mrappmaster.JobsRunningNumber of running jobsCount
MapsCompletedhadoop.yarn.mrappmaster.MapsCompletedNumber of maps completedCount
MapsFailedhadoop.yarn.mrappmaster.MapsFailedNumber of maps failedCount
MapsKilledhadoop.yarn.mrappmaster.MapsKilledNumber of maps killedCount
MapsRunninghadoop.yarn.mrappmaster.MapsRunningNumber of maps runningCount
MapsWaitinghadoop.yarn.mrappmaster.MapsWaitingNumber of maps waitingCount
ReducesCompletedhadoop.yarn.mrappmaster.ReducesCompletedNumber of completed reducesCount
ReducesFailedhadoop.yarn.mrappmaster.ReducesFailedNumber of failed reducesCount
ReducesKilledhadoop.yarn.mrappmaster.ReducesKilledNumber of killed reducesCount
ReducesRunninghadoop.yarn.mrappmaster.ReducesRunningNumber of running reducesCount
ReducesWaitinghadoop.yarn.mrappmaster.ReducesWaitingNumber of waiting reducesCount
Metric nameMetric keyDescriptionUnit
CapacityTotalhadoop.hdfs.namenode.CapacityTotalCurrent raw capacity of DataNodes in bytesByte
CapacityUsedhadoop.hdfs.namenode.CapacityUsedCurrent used capacity across all DataNodes in bytesByte
CapacityRemaininghadoop.hdfs.namenode.CapacityRemainingCurrent remaining capacity in bytesByte
TotalLoadhadoop.hdfs.namenode.TotalLoadCurrent number of connectionsCount
BlocksTotalhadoop.hdfs.namenode.BlocksTotalCurrent number of allocated blocks in the systemCount
PendingDeletionBlockshadoop.hdfs.namenode.PendingDeletionBlocksCurrent number of blocks pending deletionCount
CorruptBlockshadoop.hdfs.namenode.CorruptBlocksCurrent number of blocks with corrupt replicas.Count
CapacityUsedNonDFShadoop.hdfs.namenode.CapacityUsedNonDFSCurrent space used by DataNodes for non DFS purposes in bytesCount
Metric nameMetric keyDescriptionUnit
ShuffleConnectionshadoop.yarn.nodemanager.ShuffleConnections.countNumber of Shuffle connectionsCount
ShuffleOutputByteshadoop.yarn.nodemanager.ShuffleOutputBytes.countTotal shuffle output bytesByte
ShuffleOutputsFailedhadoop.yarn.nodemanager.ShuffleOutputsFailed.countNumber of failed shuffle outputsCount
ShuffleOutputsOKhadoop.yarn.nodemanager.ShuffleOutputsOK.countNumber of OK shuffle outputsCount
Metric nameMetric keyDescriptionUnit
DataNode BlocksReadhadoop.hdfs.datanode.BlocksRead.countTotal number of blocks read from DataNodeCount
DataNode BlocksRemovedhadoop.hdfs.datanode.BlocksRemoved.countTotal number of blocks removed from DataNodeCount
DataNode BlocksReplicatedhadoop.hdfs.datanode.BlocksReplicated.countTotal number of blocks replicatedCount
DataNode BlocksVerifiedhadoop.hdfs.datanode.BlocksVerified.countTotal number of blocks verifiedCount
DataNode BlocksWrittenhadoop.hdfs.datanode.BlocksWritten.countTotal number of blocks written to DataNodeCount
DataNode BytesReadhadoop.hdfs.datanode.BytesRead.countTotal number of bytes read from DataNodeByte
DataNode BytesWrittenhadoop.hdfs.datanode.BytesWritten.countTotal number of bytes written to DataNodeByte
Metric nameMetric keyDescriptionUnit
DataNode CacheCapacityhadoop.hdfs.datanode.CacheCapacityThe cache capacity of the DataNodeByte
DataNode CacheUsedhadoop.hdfs.datanode.CacheUsedThe cache used the DataNodeByte
DataNode Capacityhadoop.hdfs.datanode.CapacityCurrent raw capacity of the DataNodes in bytesByte
DataNode DfsUsedhadoop.hdfs.datanode.DfsUsedThe storage space that has been used up by HDFS.Byte
DataNode NumBlocksCachedhadoop.hdfs.datanode.NumBlocksCachedThe number of blocks cached on the DataNodeCount
DataNode NumBlocksFailedToCachehadoop.hdfs.datanode.NumBlocksFailedToCacheThe number of blocks that failed to cache on the DataNodeCount
DataNode NumBlocksFailedToUncachehadoop.hdfs.datanode.NumBlocksFailedToUncacheThe number of failed blocks to remove from cache.Count
DataNode NumFailedVolumeshadoop.hdfs.datanode.NumFailedVolumesNumber of failed volumes.Count
DataNode Remaininghadoop.hdfs.datanode.RemainingThe remaining DataNode disk space left in PercentPercent
Metric nameMetric keyDescriptionUnit
NodeManager AllocatedContainershadoop.yarn.nodemanager.AllocatedContainersNodeManager Current number of allocated containersCount
AllocatedGBhadoop.yarn.nodemanager.AllocatedGBCurrent allocated memory in GBGigaByte
AvailableGBhadoop.yarn.nodemanager.AvailableGBCurrent available memory in GBGigaByte
ContainersCompletedhadoop.yarn.nodemanager.ContainersCompleted.countTotal number of successfully completed containersCount
ContainersFailedhadoop.yarn.nodemanager.ContainersFailed.countTotal number of failed containersCount
ContainersInitinghadoop.yarn.nodemanager.ContainersIniting.countCurrent number of initializing containersCount
ContainersKilledhadoop.yarn.nodemanager.ContainersKilled.countTotal number of killed containersCount
ContainersLaunchedhadoop.yarn.nodemanager.ContainersLaunched.countTotal number of launched containersCount
ContainersRunninghadoop.yarn.nodemanager.ContainersRunning.countCurrent number of running containersCount
Metric nameMetric keyDescriptionUnit
FilesAppendedhadoop.hdfs.namenode.FilesAppended.countTotal number of files appendedCount
FilesCreatedhadoop.hdfs.namenode.FilesCreated.countTotal number of files and directories created by create or mkdir operationsCount
FilesDeletedhadoop.hdfs.namenode.FilesDeleted.countTotal number of files and directories deleted by delete or rename operationsCount
FilesRenamedhadoop.hdfs.namenode.FilesRenamed.countTotal number of rename operations (NOT number of files/dirs renamed)Count

Full version history

To have more information on how to install the downloaded package, please follow the instructions on this page.
ReleaseDate

Full version history

  • New Platform/Gen3 dashboard bundled with extension
  • Added DQL support for Platform/Gen3 screens
  • Enriched entities with dt.security_context attribute

Full version history

No release notes

Dynatrace Hub
Hub HomeGet data into DynatraceBuild your own app
Log Management and AnalyticsKubernetesAI and LLM ObservabilityInfrastructure ObservabilitySoftware DeliveryApplication ObservabilityApplication SecurityDigital ExperienceBusiness Observability
Filter
Type
Built and maintained by
Deployment model
SaaS
  • SaaS
  • Managed
Partner FinderBecome a partnerDynatrace Developer

All

195 Results filtered by:

Palo Alto firewalls logo

Palo Alto firewalls

Palo Alto extension for problems detection

Extension
Confluent Cloud (Kafka) logo

Confluent Cloud (Kafka)

Remotely monitor your Confluent Cloud Kafka Clusters and other resources!

Extension
Kong - Prometheus logo

Kong - Prometheus

Monitor Prometheus metrics exposed by Kong and proxied upstream services

Extension
Nutanix Clusters logo

Nutanix Clusters

Monitor Nutanix clusters' performance, usage and availability, with Nutanix API.

Extension
Luna Network HSM Device logo

Luna Network HSM Device

Monitor your Luna Network Hardware Security Module (HSM) Devices through SNMP.

Extension
Consul Service Mesh (StatsD) logo

Consul Service Mesh (StatsD)

Extend visibility into your Consul Service Mesh instances to monitor health and improve performance.

Extension
Microsoft IIS logo

Microsoft IIS

Flexible and secure web server for hosting with Windows Server.

Extension
Kubernetes Monitoring Statistics logo

Kubernetes Monitoring Statistics

Troubleshoot your Dynatrace Kubernetes monitoring and Prometheus integration.

Extension
Snyk logo

Snyk

Ingest Snyk vulnerability findings, scans, and audit logs.

Extension
Citrix DaaS & Virtual Apps and Desktops logo

Citrix DaaS & Virtual Apps and Desktops

Gain insight into your Citrix DaaS & Virtual Apps and Desktops environments

Extension
Google Memorystore logo

Google Memorystore

Get insights into Google Memorystore service metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Databricks Workspace logo

Databricks Workspace

Remotely monitor your Databricks Workspaces!

Extension
UPS Device logo

UPS Device

Monitor your Uninterruptible Power Supplies (UPS) over SNMP

Extension
Google App Engine (integration) logo

Google App Engine (integration)

Insights into Google App Engine service metrics collected from Operations API

Extensioncoming soon
Traceroute logo

Traceroute

Run traceroute commands and collect step performance metrics

Extension
[Deprecated] Kubernetes PVCs logo

[Deprecated] Kubernetes PVCs

Monitor your Kubernetes persistent volume claims and alert on capacity limits.

Extension
Google Cloud Storage Transfer logo

Google Cloud Storage Transfer

Get insights into Google Cloud Storage Transfer metrics collected from the Google Operations API to ensure health of cloud infrastructure.

Extension
NVIDIA GPU logo

NVIDIA GPU

Monitor base parameters of the GPU, including load, memory and temperature

Extension
Oracle Database logo

Oracle Database

Observe, analyze and optimize the usage, health and performance of your database

Extension
Cisco ACI/APIC logo

Cisco ACI/APIC

Get insights into your Cisco Application Centric Infrastructure (ACI)

Extension
Dell iDRAC logo

Dell iDRAC

Connect to the Redfish API to get insights into your Dell iDRAC environment

Extension
Azure Managed Apache Cassandra logo

Azure Managed Apache Cassandra

Gain insights into your Azure Managed Cassandra Instance health and performance

Extension
PayShield HSM Device logo

PayShield HSM Device

Monitor PayShield Payment Hardware Security Module (HSM) Devices through SNMP.

Extension
NetApp OnTap (Remote) logo

NetApp OnTap (Remote)

Remote extension that collects NetApp OnTap metrics from the OnTap 9.6+ API.

Extension
Google Firestore in Datastore mode logo

Google Firestore in Datastore mode

Get insights into Google Firestore in Datastore mode metrics collected from the Google Operations API to ensure health of infrastructure.

Extension
Redis (2.0) logo

Redis (2.0)

Collect important additional data for your Redis instances.

Extension
PHP-FPM logo

PHP-FPM

Monitor the PHP-FPM status of your applications with this extension.

Extension
Timedrift Monitoring logo

Timedrift Monitoring

Monitor your host's NTP/Chrony Time Offset!

Extension
Apache Kafka logo

Apache Kafka

Automatic and intelligent observability with trace and metric insights.

Extension
MongoDB (local or remote monitoring) logo

MongoDB (local or remote monitoring)

Monitor your MongoDB servers either locally or remotely!

Extension
Connection Pools: C3P0 logo

Connection Pools: C3P0

Application server method of pooling and sharing connections to a database.

Extension
AWS Entities for Metric Streaming logo

AWS Entities for Metric Streaming

Analyse metrics in the context of an entity based on AWS Metric Streaming.

Extension
MongoDB Atlas logo

MongoDB Atlas

Remotely monitor your SaaS installation of MongoDB (Atlas)

Extension
Microsoft SQL Server logo

Microsoft SQL Server

Improve the health and performance monitoring of your Microsoft SQL Servers.

Extension
IBM MQ Appliance logo

IBM MQ Appliance

Monitor your IBM MQ Appliances over SNMP

Extension
Google Apigee logo

Google Apigee

Get insights into Google Apigee service metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Oracle Autonomous Database on OCI logo

Oracle Autonomous Database on OCI

Monitor health and performance of the Oracle Autonomous Database.

Extension
Google Pub/Sub Lite logo

Google Pub/Sub Lite

Get insights into Google Pub/Sub Lite service metrics collected from the Google Operations API to ensure health of the cloud infrastructure.

Extension
Infoblox DDI logo

Infoblox DDI

Monitor Infoblox DDI using SNMP

Extension
SAP HANA Database (remote monitoring) logo

SAP HANA Database (remote monitoring)

Easily understand the health and performance of your SAP HANA databases.

Extension
Connection Pools: WebSphere Liberty logo

Connection Pools: WebSphere Liberty

Application server method of pooling and sharing connections to a database.

Extension
Google Cloud Composer logo

Google Cloud Composer

Get insights into Google Cloud Composer metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
Google Cloud Spanner logo

Google Cloud Spanner

Get insights into Google Cloud Spanner metrics collected from the Google Operations API to ensure health of your cloud infrastructure.

Extension
IBM i logo

IBM i

Collect performance data from your IBM i Hosts via this Remote extension.

Extension
Google reCAPTCHA Enterprise logo

Google reCAPTCHA Enterprise

Get insights into Google reCAPTCHA Enterprise metrics collected from the Google Operations API to ensure health of your cloud infrastructure

Extension
.NET logo

.NET

Automatic end-to-end observability for .NET applications and processes.

Extension
Google Cloud's operations suite logo

Google Cloud's operations suite

Get insights into Google Cloud's operations suite metrics collected from the Google Operations API to ensure health of cloud infrastructure.

Extension
Google Vertex AI logo

Google Vertex AI

Get insights into Google Vertex AI service metrics.

Extension
Oracle Exadata logo

Oracle Exadata

Monitor Oracle Exadata systems for performance, usage and availability

Extension
Cisco Catalyst Center (DNA Center) logo

Cisco Catalyst Center (DNA Center)

Get insights into the Cisco Catalyst Center infrastructure (f.k.a. DNA Center).

Extension