Skip to technology filters Skip to main content
Dynatrace Hub

Extend the platform,
empower your team.

Popular searches:
Home hero bg
HadoopHadoop
Hadoop

Hadoop

Enhanced insights for Hadoop HDFS and Yarn services.

Extension
Free trialDocumentation
Overview DashboardExtension configuration and featuresetsExtra Hadoop metricsHadoop HDFS service entityHadoop Yarn service entityProcess Group Injection
  • Product information
  • Release notes

Overview

Hadoop monitoring in Dynatrace provides a high-level overview of the main Hadoop components within your cluster. These enhanced insights provides additional metrics directly from the HDFS and YARN services.

Use cases

The extension enables insights into the overall health of Hadoop HDFS and YARN services

Get started

For more information on the installation and configuration, please see Hadoop extension in the Dynatrace Documentation.

Details

Compatibility information

  • Hadoop version 2.4.1+
  • Linux OS
  • For full Hadoop visibility, OneAgent must be installed on all machines running the following Hadoop processes: NameNode, ResourceManager, NodeManager, DataNode, and MRAppMaster
Dynatrace
Documentation
By Dynatrace
Dynatrace support center
Subscribe to new releases
Copy to clipboard

Feature sets

Below is a complete list of the feature sets provided in this version. To ensure a good fit for your needs, individual feature sets can be activated and deactivated by your administrator during configuration.

Feature setsNumber of metrics included
Metric nameMetric keyDescriptionUnit
AllocatedContainershadoop.yarn.resourcemanager.AllocatedContainersQueue Current number of allocated containersCount
AllocatedMBhadoop.yarn.resourcemanager.AllocatedMBCurrent allocated memory in MBMegaByte
AllocatedVCoreshadoop.yarn.resourcemanager.AllocatedVCoresCurrent allocated CPU in virtual coresCount
AppsCompletedhadoop.yarn.resourcemanager.AppsCompleted.countTotal number of completed applicationsCount
AppsFailedhadoop.yarn.resourcemanager.AppsFailed.countTotal number of failed applicationsCount
AppsKilledhadoop.yarn.resourcemanager.AppsKilled.countTotal number of killed applicationsCount
AppsPendinghadoop.yarn.resourcemanager.AppsPending.countCurrent number of applications that have not yet been assigned by any containersCount
AppsRunninghadoop.yarn.resourcemanager.AppsRunning.countCurrent number of running applicationsCount
AppsSubmittedhadoop.yarn.resourcemanager.AppsSubmitted.countTotal number of submitted applicationsCount
AvailableMBhadoop.yarn.resourcemanager.AvailableMBCurrent available memory in MBCount
AvailableVCoreshadoop.yarn.resourcemanager.AvailableVCoresCurrent available CPU in virtual coresCount
PendingMBhadoop.yarn.resourcemanager.PendingMBCurrent memory requests in MB that are pending to be fulfilled by the schedulerMegaByte
PendingVCoreshadoop.yarn.resourcemanager.PendingVCoresCurrent CPU requests in virtual cores that are pending to be fulfilled by the schedulerCount
ReservedMBhadoop.yarn.resourcemanager.ReservedMBCurrent reserved memory in MBMegaByte
ReservedVCoreshadoop.yarn.resourcemanager.ReservedVCoresCurrent reserved CPU in virtual coresCount
Metric nameMetric keyDescriptionUnit
NumberOfMissingBlockshadoop.hdfs.namenode.NumberOfMissingBlocks-Count
CacheCapacityhadoop.hdfs.namenode.CacheCapacityThe total cache capacity of all DataNodesByte
CacheUsedhadoop.hdfs.namenode.CacheUsedThe total cache used by all DataNodesByte
Metric nameMetric keyDescriptionUnit
NumActiveNMshadoop.yarn.resourcemanager.NumActiveNMsCurrent number of active NodeManagersCount
NumDecommissioningNMshadoop.yarn.resourcemanager.NumDecommissionedNMsCurrent number of NodeManagers being decommissionedCount
NumLostNMshadoop.yarn.resourcemanager.NumLostNMsCurrent number of lost NodeManagers for not sending heartbeatsCount
NumRebootedNMshadoop.yarn.resourcemanager.NumRebootedNMsCurrent number of rebooted NodeManagersCount
NumUnhealthyNMshadoop.yarn.resourcemanager.NumUnhealthyNMsCurrent number of unhealthy NodeManagersCount
Metric nameMetric keyDescriptionUnit
FilesTotalhadoop.hdfs.namenode.FilesTotalCurrent number of files and directoriesCount
PendingReplicationBlockshadoop.hdfs.namenode.PendingReplicationBlocksCurrent number of blocks pending to be replicatedCount
UnderReplicatedBlockshadoop.hdfs.namenode.UnderReplicatedBlocksCurrent number of blocks under replicatedCount
ScheduledReplicationBlockshadoop.hdfs.namenode.ScheduledReplicationBlocksCurrent number of blocks scheduled for replicationsCount
NumLiveDataNodeshadoop.hdfs.namenode.NumLiveDataNodesNumber of datanodes which are currently liveCount
NumDeadDataNodeshadoop.hdfs.namenode.NumDeadDataNodesNumber of datanodes which are currently deadCount
NumDecomLiveDataNodeshadoop.hdfs.namenode.NumDecomLiveDataNodesNumber of datanodes which have been decommissioned and are now liveCount
NumDecomDeadDataNodeshadoop.hdfs.namenode.NumDecomDeadDataNodesNumber of datanodes which have been decommissioned and are now deadCount
VolumeFailuresTotalhadoop.hdfs.namenode.VolumeFailuresTotalTotal number of volume failures across all DatanodesCount
EstimatedCapacityLostTotalhadoop.hdfs.namenode.EstimatedCapacityLostTotalAn estimate of the total capacity lost due to volume failuresCount
NumDecommissioningDataNodeshadoop.hdfs.namenode.NumDecommissioningDataNodesNumber of datanodes in decommissioning stateCount
NumStaleDataNodeshadoop.hdfs.namenode.NumStaleDataNodesNumber of datanodes marked stale due to delayed hearbeat.Count
Metric nameMetric keyDescriptionUnit
JobsCompletedhadoop.yarn.mrappmaster.JobsCompletedNumber of completed jobsCount
JobsFailedhadoop.yarn.mrappmaster.JobsFailedNumber of failed jobsCount
JobsKilledhadoop.yarn.mrappmaster.JobsKilledNumber of killed jobsCount
JobsPreparinghadoop.yarn.mrappmaster.JobsPreparingNumber of preparing jobsCount
JobsRunninghadoop.yarn.mrappmaster.JobsRunningNumber of running jobsCount
MapsCompletedhadoop.yarn.mrappmaster.MapsCompletedNumber of maps completedCount
MapsFailedhadoop.yarn.mrappmaster.MapsFailedNumber of maps failedCount
MapsKilledhadoop.yarn.mrappmaster.MapsKilledNumber of maps killedCount
MapsRunninghadoop.yarn.mrappmaster.MapsRunningNumber of maps runningCount
MapsWaitinghadoop.yarn.mrappmaster.MapsWaitingNumber of maps waitingCount
ReducesCompletedhadoop.yarn.mrappmaster.ReducesCompletedNumber of completed reducesCount
ReducesFailedhadoop.yarn.mrappmaster.ReducesFailedNumber of failed reducesCount
ReducesKilledhadoop.yarn.mrappmaster.ReducesKilledNumber of killed reducesCount
ReducesRunninghadoop.yarn.mrappmaster.ReducesRunningNumber of running reducesCount
ReducesWaitinghadoop.yarn.mrappmaster.ReducesWaitingNumber of waiting reducesCount
Metric nameMetric keyDescriptionUnit
CapacityTotalhadoop.hdfs.namenode.CapacityTotalCurrent raw capacity of DataNodes in bytesByte
CapacityUsedhadoop.hdfs.namenode.CapacityUsedCurrent used capacity across all DataNodes in bytesByte
CapacityRemaininghadoop.hdfs.namenode.CapacityRemainingCurrent remaining capacity in bytesByte
TotalLoadhadoop.hdfs.namenode.TotalLoadCurrent number of connectionsCount
BlocksTotalhadoop.hdfs.namenode.BlocksTotalCurrent number of allocated blocks in the systemCount
PendingDeletionBlockshadoop.hdfs.namenode.PendingDeletionBlocksCurrent number of blocks pending deletionCount
CorruptBlockshadoop.hdfs.namenode.CorruptBlocksCurrent number of blocks with corrupt replicas.Count
CapacityUsedNonDFShadoop.hdfs.namenode.CapacityUsedNonDFSCurrent space used by DataNodes for non DFS purposes in bytesCount
Metric nameMetric keyDescriptionUnit
ShuffleConnectionshadoop.yarn.nodemanager.ShuffleConnections.countNumber of Shuffle connectionsCount
ShuffleOutputByteshadoop.yarn.nodemanager.ShuffleOutputBytes.countTotal shuffle output bytesByte
ShuffleOutputsFailedhadoop.yarn.nodemanager.ShuffleOutputsFailed.countNumber of failed shuffle outputsCount
ShuffleOutputsOKhadoop.yarn.nodemanager.ShuffleOutputsOK.countNumber of OK shuffle outputsCount
Metric nameMetric keyDescriptionUnit
DataNode BlocksReadhadoop.hdfs.datanode.BlocksRead.countTotal number of blocks read from DataNodeCount
DataNode BlocksRemovedhadoop.hdfs.datanode.BlocksRemoved.countTotal number of blocks removed from DataNodeCount
DataNode BlocksReplicatedhadoop.hdfs.datanode.BlocksReplicated.countTotal number of blocks replicatedCount
DataNode BlocksVerifiedhadoop.hdfs.datanode.BlocksVerified.countTotal number of blocks verifiedCount
DataNode BlocksWrittenhadoop.hdfs.datanode.BlocksWritten.countTotal number of blocks written to DataNodeCount
DataNode BytesReadhadoop.hdfs.datanode.BytesRead.countTotal number of bytes read from DataNodeByte
DataNode BytesWrittenhadoop.hdfs.datanode.BytesWritten.countTotal number of bytes written to DataNodeByte
Metric nameMetric keyDescriptionUnit
DataNode CacheCapacityhadoop.hdfs.datanode.CacheCapacityThe cache capacity of the DataNodeByte
DataNode CacheUsedhadoop.hdfs.datanode.CacheUsedThe cache used the DataNodeByte
DataNode Capacityhadoop.hdfs.datanode.CapacityCurrent raw capacity of the DataNodes in bytesByte
DataNode DfsUsedhadoop.hdfs.datanode.DfsUsedThe storage space that has been used up by HDFS.Byte
DataNode NumBlocksCachedhadoop.hdfs.datanode.NumBlocksCachedThe number of blocks cached on the DataNodeCount
DataNode NumBlocksFailedToCachehadoop.hdfs.datanode.NumBlocksFailedToCacheThe number of blocks that failed to cache on the DataNodeCount
DataNode NumBlocksFailedToUncachehadoop.hdfs.datanode.NumBlocksFailedToUncacheThe number of failed blocks to remove from cache.Count
DataNode NumFailedVolumeshadoop.hdfs.datanode.NumFailedVolumesNumber of failed volumes.Count
DataNode Remaininghadoop.hdfs.datanode.RemainingThe remaining DataNode disk space left in PercentPercent
Metric nameMetric keyDescriptionUnit
NodeManager AllocatedContainershadoop.yarn.nodemanager.AllocatedContainersNodeManager Current number of allocated containersCount
AllocatedGBhadoop.yarn.nodemanager.AllocatedGBCurrent allocated memory in GBGigaByte
AvailableGBhadoop.yarn.nodemanager.AvailableGBCurrent available memory in GBGigaByte
ContainersCompletedhadoop.yarn.nodemanager.ContainersCompleted.countTotal number of successfully completed containersCount
ContainersFailedhadoop.yarn.nodemanager.ContainersFailed.countTotal number of failed containersCount
ContainersInitinghadoop.yarn.nodemanager.ContainersIniting.countCurrent number of initializing containersCount
ContainersKilledhadoop.yarn.nodemanager.ContainersKilled.countTotal number of killed containersCount
ContainersLaunchedhadoop.yarn.nodemanager.ContainersLaunched.countTotal number of launched containersCount
ContainersRunninghadoop.yarn.nodemanager.ContainersRunning.countCurrent number of running containersCount
Metric nameMetric keyDescriptionUnit
FilesAppendedhadoop.hdfs.namenode.FilesAppended.countTotal number of files appendedCount
FilesCreatedhadoop.hdfs.namenode.FilesCreated.countTotal number of files and directories created by create or mkdir operationsCount
FilesDeletedhadoop.hdfs.namenode.FilesDeleted.countTotal number of files and directories deleted by delete or rename operationsCount
FilesRenamedhadoop.hdfs.namenode.FilesRenamed.countTotal number of rename operations (NOT number of files/dirs renamed)Count

Full version history

To have more information on how to install the downloaded package, please follow the instructions on this page.
ReleaseDate

Full version history

  • New Platform/Gen3 dashboard bundled with extension
  • Added DQL support for Platform/Gen3 screens
  • Enriched entities with dt.security_context attribute

Full version history

No release notes

Dynatrace Hub
Hub HomeGet data into DynatraceBuild your own app
Dynatrace Intelligence - Agentic Operations SystemThe Dynatrace Agentic AI ecosystem
Log Management and AnalyticsKubernetesAI and LLM ObservabilityInfrastructure ObservabilitySoftware DeliveryApplication ObservabilityApplication SecurityBusiness ObservabilityDigital Experience
Filter
Type
Built and maintained by
Deployment model
SaaS
  • SaaS
  • Managed
Partner FinderBecome a partnerDynatrace Developer

All

221 Results filtered by:

Akamai logo

Akamai

Ingest logs and security events from Akamai products.

Extension
Alibaba Cloud logo

Alibaba Cloud

Automatically discover, baseline, and monitor Alicloud environment components.

Extension
Anypoint MQ logo

Anypoint MQ

Gather metrics about your Anypoint MQ Queues.

Extension
Apache ActiveMQ Classic logo

Apache ActiveMQ Classic

Automatic and smart observability for ActiveMQ with trace and metric insights.

Extension
Apache Cassandra logo

Apache Cassandra

Improve Apache Cassandra observability.

Extension
Apache CouchDB logo

Apache CouchDB

Intelligently observe and optimize health and performance of your database.

Extension
Apache Kafka logo

Apache Kafka

Automatic and intelligent observability with trace and metric insights.

Extension
Apache Solr logo

Apache Solr

Open-source enterprise-search platform, written in Java.

Extension
Apache Spark logo

Apache Spark

Enhanced insights for Spark Components.

Extension
Apache Zookeeper logo

Apache Zookeeper

Monitor and analyze your Apache Zookeeper with this JMX-based extension.

Extension
Apigee logo

Apigee

Remotely monitor your Apigee proxies.

Extension
AWS Cloud Monitoring logo

AWS Cloud Monitoring

New and enhanced monitoring capabilities for your AWS cloud platforms.

Extension
AWS Entities for Metric Streaming logo

AWS Entities for Metric Streaming

Analyse metrics in the context of an entity based on AWS Metric Streaming.

Extension
Azure Cloud Monitoring logo

Azure Cloud Monitoring

New and enhanced monitoring capabilities for your Azure cloud platforms.

Extensioncoming soon
Azure Managed Apache Cassandra logo

Azure Managed Apache Cassandra

Gain insights into your Azure Managed Cassandra Instance health and performance.

Extension
Black Duck logo

Black Duck

Ingest Black Duck security events and audit logs.

Extension
Ceph storage logo

Ceph storage

Monitor usage of Ceph storage system at both client side and host level.

Extension
Checkmarx logo

Checkmarx

Ingest Checkmarx security events and audit logs.

Extension
Check Point Firewall logo

Check Point Firewall

Gain insights into the performance of your Check Point Firewall devices.

Extension
Cisco ACI/APIC logo

Cisco ACI/APIC

Get insights into your Cisco Application Centric Infrastructure (ACI).

Extension
Cisco Catalyst Center (DNA Center) logo

Cisco Catalyst Center (DNA Center)

Get insights into the Cisco Catalyst Center infrastructure (f.k.a. DNA Center).

Extension
Cisco Catalyst SD-WAN Manager (Viptela) logo

Cisco Catalyst SD-WAN Manager (Viptela)

Get insights into the Cisco SD-WAN performance through the vManage API.

Extension
Cisco Firepower logo

Cisco Firepower

Monitor Cisco Firepower devices using SNMP.

Extension
Cisco UCS C-Series logo

Cisco UCS C-Series

Get insights into your Cisco UCS C-Series devices.

Extension
Cisco UCS M-Series logo

Cisco UCS M-Series

Get insights into your Cisco UCS M-Series devices.

Extension
Citrix DaaS & Virtual Apps and Desktops logo

Citrix DaaS & Virtual Apps and Desktops

Gain insight into your Citrix DaaS & Virtual Apps and Desktops environments.

Extension
Citrix Netscaler ADC logo

Citrix Netscaler ADC

Remotely monitor your Citrix NetScaler devices.

Extension
Citrix NetScaler SDX logo

Citrix NetScaler SDX

Remotely monitor your Citrix NetScaler SDX devices.

Extension
Cloudflare (extension) logo

Cloudflare (extension)

Collect notification history and various metrics from your Cloudflare accounts.

Extension
Cloud Foundry logo

Cloud Foundry

Monitor your Cloud Foundry resource definitions by connecting to the foundation.

Extension
Confluent Cloud (Kafka) logo

Confluent Cloud (Kafka)

Remotely monitor your Confluent Cloud Kafka Clusters and other resources!.

Extension
Connection Pools: Apache Commons logo

Connection Pools: Apache Commons

Application method of pooling and sharing multiple connections to a database.

Extension
Connection Pools: C3P0 logo

Connection Pools: C3P0

Application server method of pooling and sharing connections to a database.

Extension
Connection Pools: JBoss logo

Connection Pools: JBoss

Application server method of pooling and sharing connections to a database.

Extension
Connection Pools: WebLogic logo

Connection Pools: WebLogic

Application server method of pooling and sharing connections to a database.

Extension
Connection Pools: WebSphere Liberty logo

Connection Pools: WebSphere Liberty

Application server method of pooling and sharing connections to a database.

Extension
Consul Service Mesh (StatsD) logo

Consul Service Mesh (StatsD)

Extend visibility into your Consul Service Mesh instances to monitor health and improve performance.

Extension
Control-M Jobs v2 logo

Control-M Jobs v2

Monitor your Control-M jobs execution status.

Extension
CoreDNS (2.0) logo

CoreDNS (2.0)

Collect important data from your CoreDNS processes.

Extension
Couchbase logo

Couchbase

Couchbase Capella (SaaS) database monitoring.

Extension
CPU Core Monitoring logo

CPU Core Monitoring

Get insights into CPU Core utilization on your OneAgent hosts.

Extension
CrowdStrike logo

CrowdStrike

Ingest CrowdStrike detection findings.

Extension
CyberArk logo

CyberArk

Ingest CyberArk audit logs via SIEM integration for reporting and analysis.

Extension
Databricks logo

Databricks

Monitor your Databricks Clusters via its multiple APIs!.

Extension
Databricks Workspace logo

Databricks Workspace

Remotely monitor your Databricks Workspaces!.

Extension
definity logo

definity

Monitor and optimize your Lakehouse, Spark, and Databricks pipelines.

Extension
Defunct Process Monitoring logo

Defunct Process Monitoring

Monitoring defunct processes stuck on your linux servers.

Extension
Dell EMC Isilon - PowerScale logo

Dell EMC Isilon - PowerScale

Monitor your Dell EMC Isilon / Powerscale clusters through the OneFS API.

Extension
Dell EMC PowerMax logo

Dell EMC PowerMax

Monitor PowerMax Arrays for performance and health, with Dell Unisphere API.

Extension
Dell iDRAC logo

Dell iDRAC

Connect to the Redfish API to get insights into your Dell iDRAC environment.

Extension