Average instrumentation time for agent was too slow

Environment

Appmon: 3.5 +

>=3.5

Symptoms

Agent starts successfully, but without instrumentation. Similar entries can be seen in the agent's startup logs:

warning [native] Average transformation time for 1333 classes/modules expected to be 60ms maximum, but was 91ms
info [native] The expected transformation time can be set via the Agent option: transformationmaxavgwait=<ms>. Please consult the documentation for further information.

Solution

Either solve the underlying transformation slowness issue, which can be:

  • Host performance (High CPU usage, low disk IO performance etc)
  • Network latency between agent and collector

Or, increase the threshold for transformation-wait timeout via one of the following ways:

  • Via agent instrumentation string option transformationmaxavgwait, eg(example for 90ms threshold)
    -agentpath=...,transformationmaxavgwait=90
    
    
  • For .NET agents, set the value via environment variable DT_TRANSFORMATIONMAXAVGWAIT

The default thresholds for transformation times are different for Java and .NET, due to the different transformation strategies and required data amounts to be transferred between agent and collector(single classes in case of Java vs whole modules in case of .NET). Defaults are:

  • Java: 60ms
  • .NET: 6000ms

Root cause

During agent startup, we test its class/module transformation performance by performing a sample transformation(always the identical and same amount of classes/modules across all agents). If the transformation turns out to be too slow, a failsafe mechanism disables all transformation for the agent to prevent slowing down the instrumented application's startup time. The thresholds for this failsafe are editable by the user according to their requirements, as described in the solution section