When your application experiences memory leaks or high object churn, it’s important that you get your hands on the memory dumps so you can analyze these issues. In production environments, this is often quite a challenge as you most likely can’t log into the environment, and you may have no other means of triggering memory dumps. Dynatrace enables you to both trigger and securely download memory dumps to subsequently analyze them in your analysis tool of choice.
Trigger memory dumps for Java and Node.js
To trigger a memory dump:
Navigate to the Host page, Service page, Process page, or Process group page of the entity that you want to analyze.
Click the Browse […] button and select Memory dump details.
The Memory dumps page displays a list of the memory dumps that occurred during the selected time frame for the selected entity (host, service, process, or process group).
Click the Trigger new dump button to generate a new memory dump (you’ll need the Configure capture of sensitive data user permission to do this). To trigger a memory dump first select the process. Note: Dynatrace currently supports memory dumps for Java and Node.js processes only.
Click the Trigger dump button.
It takes a few minutes to generate a memory dump. The time required varies widely based on application type. Java applications that have multiple GBs of heap memory may take several minutes; smaller dumps will be available almost immediately.
Note for Java applications: Dynatrace uses the JVM Tool Interface (JVM TI) to generate memory dumps. For this reason, your JVM may stall during memory-dump generation. Please restart your Java-based application following trigger of a memory dump.
Return to the Memory dumps page. The newly created dump now appears in the list.
Click the memory dump Download link to view the memory dump (see example below).
Memory dumps are first stored locally on the disk of the monitored application-server machine. Dynatrace OneAgent can subsequently upload the memory dumps automatically to a Security Gateway, which can act as a long-term storage center for memory dumps. The provided download links direct you to the Security Gateway that received that memory dump. This approach ensures that memory dumps are only available to users who have access to the network location of your Security Gateway. This precaution provides an additional security layer that ensures that no sensitive data leaves your data center without you explicitly configuring it that way.
In the case of Java applications, the download provides the memory dump in hprof format, which can be analyzed using a number of tools, including Eclipse Memory analyzer and VisualVM (see example below).
The process works the same for Node.js applications. Node.js memory dumps can be opened in Google Chrome’s integrated memory heap snapshot analysis tool (see example below).
Configure a Security Gateway for memory dump storage
If you don’t already have a Dynatrace Security Gateway setup, you need to install a Security Gateway.
Add the following code to the
custom.properties file and adjust the values based on your requirements.
[collector] ... # collector supports collecting of memory dumps from agents (default: false) # Add this to the [collector] section DumpSupported = true ... # Add this section to the end of the file [dump] # relative to security gateway installation directory or absolute path # The Security Gateway runs with reduced permissions to the file system. Ensure that the user that runs the Security gateway has write permissions to this directory dumpDir = dump # maximum size in gb for stored memory dump maxSizeGb = 100 # maximum age in days for keeping stored memory dumps maxAgeDays = 7 # maximum number of concurrent file uploads supported maxConcurrentUploads = 5
The configured directory must be created manually and the permissions must be set so that the user who runs the Security Gateway has read and write permissions.
- On Linux, this requires that the user have the
Read & executepermission on the directory.
- On Windows, this requires that the
Usersuser group have the
Restart the Security Gateway. All instances of Dynatrace OneAgent will automatically reconnect and recognize that the Security Gateway can now store memory dumps. Going forward, whenever a memory dump is successfully triggered, the dump will be uploaded automatically to the correct Security Gateway. Once safely stored on the Security Gateway, OneAgent deletes the local file to clean up its storage.
This feature requires OneAgent v1.127 or higher and Security Gateway v1.127 or higher.
- Memory dumps are not available for IBM Java.
- Memory dumps are not yet available for PaaS integrations.
- Your application server must have adequate space available to store the heap dump.