Monitor your Apache Storm environment with this ActiveGate extension

Metrics

• Cluster Memory Used
• Cluster CPU Used
• Supervisor CPU Used
• Supervisor Memory Used
• Supervisor Uptime
• Supervisor Slots Free
• Supervisor Slots Used
• Nimbus Uptime
• Nimbus Status
• Topology Replication
• Topology Tasks
• Topology Uptime
• Topology Workers
• Topology Assigned CPU
• Topology Assigned Memory
• Topology Status
• Spout Acked
• Spout Complete Latency
• Spout Emitted
• Spout Failed
• Spout Transferred
• Bolt Acked
• Bolt Capacity
• Bolt Emitted
• Bolt Execute Latency
• Bolt Failed
• Bolt Process Latency
• Bolt Transferred

Configuration

Create an Endpoint for every Apache Storm cluster you would like to monitor.
This is done in Settings > Monitored technologies > Custom extensions > Apache Storm.

The parameters are:

• Storm UI URL
• Storm Cluster Name
• Frequency (minutes)
• Log level

Visualization

Under Technologies > Apache Storm three groups will be created

• Topology (Cluster Name)
• Supervisors (Cluster Name)
• Nimbus (Cluster Name)

DDUs per year

(2 + (5 * supervisors) + (2 * nimbus) + (7 * topologies) + (5 * spouts) + (7 * bolts)) / frequency * 525.6

Example:

1 cluster with 6 supervisors, 3 nimbus servers, 6 topologies, 8 spouts and 10 bolts, monitored every 5 minutes:

(2 + (5 * 6) + (2 * 3) + (7 * 6) + (5 * 8) + (7 * 10)) / 5 * 525.6 = 19950 DDUs/year

Requirements

• Environment Activegate with access to the Apache Storm UI url

Installation

There are two steps to install an Activegate extension:

1. Environment Activegate server - Extract the extension zip file to the plugin_deployment folder of the remote plugin module
   • In a default installation this is done with unzip -o -d /opt/dynatrace/remotepluginmodule/plugin_deployment custom.remote.python.apache_storm.zip
   • Adjust the path if the Activegate was installed somewhere other than /opt/dynatrace
2. Browser - Upload the same extension zip file to your tenant
   • Settings > Monitored technologies > Custom extensions > Upload extension

Extension Logs

The logs under %PROGRAMDATA% (windows) or /var/lib (linux) give us more details in case of failures

The full path is /var/lib/dynatrace/remotepluginmodule/log/remoteplugin/custom.remote.python.apache_storm/ApacheStormExtension.log

or C:\ProgramData\dynatrace\remotepluginmodule\log\remoteplugin\custom.remote.python.apache_storm\ApacheStormExtension.log