Customizing Dynatrace process detection

Dynatrace automatically detects process types like Tomcat, JBoss, Apache web server, MongoDB, and many others. For each process type Dynatrace looks for specific patterns to identify processes that should logically be grouped together into “process groups” for monitoring purposes. A “process group” is a set of processes that perform the same function across multiple hosts. It also ensures that Dynatrace can present continuous charts and information for a particular Application server after restarts or configuration and deployment changes.

With Tomcat, for instance, Dynatrace uses CATALINA_HOME and CATALINA_BASE to distinguish between different Tomcat clusters. For JBoss, Dynatrace uses JBOSS_HOME and the JBoss cluster configuration. For generic Java processes, Dynatrace uses the JAR file or main class used to start the process. Learn more about how Dynatrace tracks host processes

This approach to process detection works fine in most cases, but isn’t perfect. This is why we’ve enabled you to customize how Dynatrace detects and identifies host processes in your environment.

Each automatic deployment results in creation of a new service and process

Tomcat, JBoss, and other applications are often deployed automatically to production. In some cases this involves the creation of new containers and directories with each deployment. Often build numbers, dates, GUIDs, and even version numbers are automatically appended to directory names (see Tomcat example below).
This presents a challenge for long-term process monitoring because it results in each new deployment being considered by Dynatrace to be a newly discovered process.

You can see if this is the case with your processes by examining the Properties section of any Process page (for example, Catalina base and Catalina home in the example below).

Process detection

Using the example above, Dynatrace previously would have considered each new deployment of CATALINA_HOME to be a new process representing a new process group. This would have made continuous baselining and performance tracking of this process impossible. To prevent this problem, we’ve added a feature that will automatically detect IDs, dates, GUIDsand other numbers in directory-names of your executable during detection and filter them out. This is default for all new tenants of Dynatrace. You should not change it.

Existing tenants can this feature on in the process group naming settings. Please be aware that any process that is affected will have to be restarted and that this will result in new Services being detected. You should do this if the services and processes detected by Dynatrace change after you deploy a new version of your application.

  1. Go to Settings > Monitoring overview > Process group naming.
  2. Set the Ignore versions, builds, dates, and GUIDs in process directory names switch to On.
    By enabling this setting, any newly generated numbers, dates, or GUIDs in directory names will be ignored when determining if processes simply belong to the same cluster or if they are indeed the same logical entity. This ensures continuity of service and process-level data following the deployment of new versions.
    Process detection

Adjusting process detection for Java processes

For a while now you’ve had the ability to adjust process group detection via environment variables. While this works well, it’s often not convenient for monitoring Java processes that use Java system properties rather than environment variables. Now you can specify which Java system property Dynatrace should use to identify process groups (and specific nodes in that group) by creating a new process-group detection rule. In the example below Dynatrace is set to use the Java system property jetty.home to detect processes.

Process detection

Process detection

This results in the process with the following command line to be detected as “/opt/solr/server”. Note the identifying system property and value in the properties of the process (see CUSTOM_ENTRY).
Every Java process that has the same jetty.home value will be considered part of the same process group. Dynatrace will detect a cluster of these processes even if they’re deployed across multiple hosts.

polkitd 12448 11914 1 03:35 pts/0 00:04:57 java -server -Xss256k -Xms512m -Xmx512m -XX:NewRatio=3 -XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=8 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:ConcGCThreads=4 -XX:ParallelGCThreads=4 -XX:+CMSScavengeBeforeRemark -XX:PretenureSizeThreshold=64m -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=50 -XX:CMSMaxAbortablePrecleanTime=6000 -XX:+CMSParallelRemarkEnabled -XX:+ParallelRefProcEnabled -verbose:gc -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:/opt/solr/server/logs/solr_gc.log -Djetty.port=8983 -DSTOP.PORT=7983 -DSTOP.KEY=solrrocks -Duser.timezone=UTC -Djetty.home=/opt/solr/server -Dsolr.solr.home=/opt/solr/server/solr -Dsolr.install.dir=/opt/solr -jar start.jar --module=http

Note: To take advantage of this feature you must upgrade to OneAgent v1.81 or higher.