Android auto setup and instrumentation

Android auto-instrumentation lets anyone instrument Android applications for monitored using the mobile agent. The instrumentation process is an automated way to add the mobile agent to an application without manually modifying the source code. The mobile agent is added in the auto-instrumentation process to the application. Data collection is the same with an auto-instrumented application as it is with a manually instrumented application.

There are two options how you can start the auto-instrumentation process:

  • Call instrument.<cmd_or_sh> apk=<app-name>.apk prop=<prop-name>.properties from the command line.
  • Integrate the auto-instrumentation process into your Gradle build process with the Dynatrace Gradle plugin

Auto-instrumentation from the command line

See the Android system requirements before running the auto-instrumentor the first time.

  1. Download the newest mobile agent. The file name is dynatrace-mobile-agents-<version>.zip. Unpack it to a directory of your choice.
  2. Copy the APK-Instr.properties property template file from the <MobileAgentUnzipDir>/Android/auto-instrumentor directory to a directory of your choice, and rename it.
  3. Edit the properties file to customize the auto-instrumentation process. The DTXApplicationID and DTXAgentStartupPath properties are mandatory.
  4. Run the instrument command with the apk and prop parameters: instrument.<cmd_or_sh> apk=<app-name>.apk prop=<prop-name>.properties.
  5. Sign the instrumented APK <pathToYourApk>/<nameOfAPK>/dist/<nameOfAPK>.apk with your debug or Google playstore certificate, or use the <pathToYourApk>/<nameOfAPK>/dist/<nameOfAPK>-final.apk APK, which is signed with the auto-instrumentor debug certificate.
  6. Start AppMon, including the web server (master) agent and your instrumented web or app server.
  7. Install and run your instrumented APK file.

Auto-Instrumentation features

This level of instrumentation provides visibility into:

  • Automatic mobile agent startup. You must use instrumentation keys to configure the automatic startup.
  • Lifecycle data.
  • Crash detection and reporting.
  • Real user experiences of the flow within the application, such as responsiveness on user actions like a button touch.
  • Web requests ‐ by tagging them.
  • Web request events, such as reporting execution time and identifying third-party requests.

Mobile agent start-up

The auto-instrumentor inserts the mobile agent start-up into the beginning of the application start procedure. Therefore you can't control the start-up parameters with manual instrumentation, because the agent ignores multiple start-up calls.

The DTXApplicationID and DTXAgentStartupPath properties are mandatory, because they are needed for the Dynatrace.startup() call. For the DTXApplicationID property, specify a unique application ID. For the DTXAgentStartupPath property, specify the address of your instrumented web or application server. For example:

DTXApplicationID=YourUniqueAppId
DTXAgentStartupPath=http://<instrumentedWebOrAppServer>[:portIfNot80]

Specifying an HTTPS address in the DTXAgentStartupPath property lets the agent verify the server certificate and the host name. The agent communication with the instrumented web or application server fails when verification steps can't complete successfully.

If you don't have a root certificate for your instrumented web or application server, you must provide a BKS keystore file, which contains the certificate(s) for SSL communication. Add the keystore file to the raw directory of the APK and specify the BKS file name (without the file extension) in the DTXBKSFileName property. You also must specify the keystore file password in the DTXBKSPassword property. For example:

# for file raw/mykeystore.bks use
DTXBKSFileName=myKeyStore
DTXBKSPassword=myPassword

The keystore password is not secured and can be retrieved by decompiling the application.

Another option is to deactivate the certificate validation with the DTXAllowAnyCert property. Host name verification can't be deactivated.

Crash reporting

Crash reporting is enabled by default. The mobile agent captures all unhandled exceptions/errors and sends the crash report immediately to the server. The Android crash report includes the occurrence time and the full stack trace of the exception. The crash report is attached to the user action that has triggered the exception.

Use the DTXCrashReportingEnabled property to disable crash reporting. Then the auto-instrumentor does not insert the method call Dynatrace.enableCrashReporting(boolean sendCrashData) into your applications start procedure. If you deactivate crash reporting, the agent ignores the corresponding value from your user experience configuration.

Lifecycle monitoring

The auto-instrumentor automatically instrument all activities listed in the AndroidManifest.xml file that are not manually instrumented. Use the DTXInstrumentLifecycleMonitoring property to deactivate lifecycle instrumentation. You can find more information in the chapter Collected lifecycle data.

User action monitoring

Auto-instrumentation allows you to automatically detect user actions. Therefore the auto-instrumentor instruments the following listeners and methods:

  • Android
    • method: android.app.Activity.onOptionsItemSelected
    • android.view.MenuItem$OnMenuItemClickListener
    • android.view.View$OnClickListener
    • android.widget.AdapterView$OnItemClickListener
    • android.widget.AdapterView$OnItemSelectedListener
  • Android Support
    • android.support.v4.view.ViewPager$OnPageChangeListener
    • android.support.v4.widget.SwipeRefreshLayout$OnRefreshListener
  • ActionBarSherlock
    • method: com.actionbarsherlock.app.SherlockActivity.onOptionsItemSelected
    • com.actionbarsherlock.internal.widget.IcsAdapterView$OnItemSelectedListener
    • com.actionbarsherlock.view.MenuItem$OnMenuItemClickListener

While an user action is open, the mobile agent adds all triggered events such as web requests and crashes to this user action. But most user actions are generated in the main thread and their web requests are triggered in a background thread after the user actions are closed. Normally these web requests would not be added to the user action. Therefore the auto-instrumentor configures a time frame. In this time frame the mobile agent adds web requests to the closed user action. You can set this time duration in milliseconds with the DTXAutoActionTimeoutMilliseconds property. The default value for this property is 500 ms and the valid range is 100 ms - 5,000 ms.

You can also configure the maximum duration of the user action with the DTXAutoActionMaxDurationMilliseconds property. The property default value is 60,000 ms (one minute) and the valid range is 100 ms - 540,000 ms. After the first time frame expires the mobile agent keeps the user action until all outstanding web requests or child actions complete, or the user action maximum duration is reached. In this waiting time frame the mobile agent does not accept web requests. These web requests are added to other user actions. When the maximum duration is reached, then the mobile agent closes the user action and all outstanding web requests or child actions.

Note

You can only configure one value for the DTXAutoActionTimeoutMilliseconds and DTXAutoActionMaxDurationMilliseconds properties, and these values must fit all user actions on all devices.

By default, the mobile agent discards empty user actions to reduce the amount of data that is sent to the server. If you are interested in these user actions, you can change this behavior with the DTXSendEmptyAutoAction property. You can also deactivate user action monitoring with the DTXInstrumentAutoUserAction property.

Note

The auto-instrumentor only instruments your byte-code. It does not instrument listener methods defined in xml files. You must manually instrument these listener methods.

Also, the auto-instrumentor does not instrument controls placed in a WebView. See How to instrument a hybrid app for more information.

Web request monitoring

The auto-instrumentor activates web request tagging and timing for your web requests. The mobile agent automatically truncates the query from the captured URL and only reports the domain name and path of your web requests.

The following list contains the supported HTTP frameworks:

Note

The auto-instrumentor does not support OkHttp 3 library obfuscation. You must add the following rule to your ProGuard rules file:

-keep class okhttp3.** { *; }

The mobile agent automatically adds your web requests to the user action that triggered these web requests. For Web requests that can't be assigned to a user action, mobile agent generates a new action and attach the web request to it. The name of this action contains the key word WebRequest and the domain name of the web requests. For example WebRequest(www.domain.com). Use the DTXShowFullWebRequestURL property to replace the name of this wrapping action with the full URL. There is an action name length limit and the web request URL is truncated if it exceeds this limit. The DTXShowFullWebRequestURL property only changes the name of the wrapping action. The web request event is not affected by the property and does not show the full URL.

Deactivate web request tagging and timing with the DTXInstrumentWebRequestTagging and DTXInstrumentWebRequestTiming properties. Because web request tagging is a precondition for web request timing, you can't deactivate tagging without deactivating timing. The auto-instrumentor ignores the DTXInstrumentWebRequestTagging property value when the DTXInstrumentWebRequestTiming property is activated.

Note

When you use auto-instrumentation and activate web request and user action monitoring, is it recommended to renounce manual web request instrumentation for the supported HTTP frameworks. Otherwise, your implementation can have unexpected results.

Additional customization

Auto-instrumentation instruments all packages by default. To skip third party content, you can use the DTXExcludePackages property to exclude packages, classes, or methods. For example:

DTXExcludePackages=com.xyz.IncludedClass.excludeThisMethod,com.xyz.ExcludedClass

If you only want to instrument your application package, use the DTXIncludePackages property to select the packages and deactivate DTXIncludeAllPackages. You can't include packages using the DTXIncludePackages property, if they are already excluded by the DTXExcludePackages property.

DTXIncludeAllPackages=false
DTXIncludePackages=com.this.pkg,com.that.pkg

To refresh the uninstrumented application with the instrumented application on Google Play, you must change the version number. By default, the auto-instrumentator does not change the version code and name. Use the DTXVersionCode and DTXVersionName properties to change these two values. If you have build a new APK file with a new version name and code, then you do not have to change these values, because the auto-instrumentor uses your new version name and code.

Activate the DTXInstrumentGPSLocation property to monitor the user location. This lets the auto-instrumentor instrument all your location listeners, and appends the captured user position to the sent data. The agent captures only three fractional digits. Your application must request the location permission and implement the location listener. The auto-instrumentor supports the following location listeners:

  • android.location.LocationListener
  • com.google.android.gms.location.LocationListener

Set the DTXLogLevel property to the value debug to activate the agent and auto-instrumentor debug logs. The auto-instrumentor and mobile agent log additional information in this mode that helps support troubleshoot problems. It is recommended recommend to remove this property when building your playstore or production application, because the additional logging can slow down the application. The auto-instrumentor creates a log file AIA_<timestamp>.log in the log directory of your working directory, depending on the terminal where you execute the instrument command. The mobile agent logs are available through the logcat tool or view.

Full property list

The following list shows the properties used to fine-tune your instrumentation:

Key Key Type Description
DTXApplicationID String This key’s value is used as your application ID to identify it on the AppMon Server. If the key is not present, the auto-instrumentor throws an error.
DTXAgentStartupPath String This key with a valid value lets the mobile agent start automatically, using this value as the serverUrl, and ignores any manual Dynatrace.startup calls in your app. The serverUrl needs to start with the transport mechanism to use (http://  or https://). If it is not present, the auto-instrumentor throws an error.
DTXAllowAnyCert boolean This key is equivalent to the useAnyCert parameter in the Dynatrace.startup call. If set to true, all certificates are accepted. If set to false, valid certificates from known certificate authorities are accepted. Self-signed certificates (see DTXBKSFileName and DTXBKSPassword) are checked for validity. The default value is false.
DTXBKSFileName String This key (no file extension) defines the name of a BKS keystore file (must be in the APKs raw directory), which contains a (self-signed) certificate and which is used as an additional anchor to validate HTTPS communication. This key is needed if DTXAllowAnyCert is false and a self-signed certificate is used on the server.
DTXBKSPassword String The password for the BKS keystore file (see DTXBKSFileName).
DTXCrashReportingEnabled boolean The default value is true. Set to false to deactivate crash reporting.
DTXInstrumentLifecycleMonitoring boolean The default value is true. Set to false to disable automatic lifecycle monitoring.
DTXInstrumentWebRequestTagging boolean The default value is true. Enables web request tagging for all supported HTTP frameworks. Set to false to disable automatic web request tagging.
DTXInstrumentWebRequestTiming boolean The default value true. Enables web request timing for all supported HTTP frameworks. Set to false to disable automatic web request timing. If web request timing is enabled, the value from DTXInstrumentWebRequestTagging is ignored, because web request tagging is a precondition for web request timing.
DTXInstrumentAutoUserAction boolean The default value true. If enabled, the mobile agent automatically creates user actions for user interactions such as button presses. Set to false to disable automatic user action creation.
DTXAutoActionTimeoutMilliseconds int The default value is 500 ms. Sets the value for how long a particular automatic user action is active. The purpose of this setting is to catch all web requests, that happen when an automatic user action is active. If the automatic user action has completed web requests, the mobile agent leaves the action at the end of this time. The minimum allowed value is 100 ms, the maximum allowed value is 5000 ms (five seconds).
DTXAutoActionMaxDurationMilliseconds int The default value is 60000 ms (60 seconds). Sets the value for how long a particular automatic user action is retained before being deleted. The purpose of this setting is to catch all web requests, that happen when an automatic user action is active. If the automatic user action has pending web requests (because they are taking a long time to complete), the mobile agent waits for this amount of time for the web requests to complete before leaving the user action. The minimum allowed value is 100 ms, the maximum allowed value is 540000 ms (nine minutes).
DTXSendEmptyAutoAction boolean The default value is false. If enabled, the mobile agent sends automatic user actions that do not contain any child elements.
DTXMultiDexKeep String Defines a list of classes and packages that should stay in the primary .dex file. The auto-instrumentor also analyzes the dependencies of these classes and ensures these dependecies stay in the primary .dex file. Separate the list elements with a comma, and mark packages with an asterisk.
DTXMultiDexKeepFile String Defines the absolute path to a file containing a list of classes that should stay in the primary .dex file. It is recommend to define the maindexlist.txt file, which is generated by the Android Gradle plugin.
DTXPrimaryDexLimit int Defines the limit of referenced methods for the primary .dex file (before byte-code instrumentation). The default value is around 62K.
DTXSecondaryDexLimit int Defines the referenced methods limit for the secondary .dex files (before byte-code instrumentation). The default value is nearly 64K.
DTXMonitorCookie String The given value is automatically added as a cookie to the http requests of the agent-server communication, so they can pass your infrastructure's requirements.
DTXSetCookiesForDomain String For hybrid applications using the JavaScript agent or JavaScript bridge, cookies need to be set for each instrumented domain or server the application communicates with. You can specify domains, host or IP addresses. Domains or sub-domains must start with a dot. Separate the list elements with a comma.
DTXAllowFileCookies boolean The default value is false. Set to true to use file cookies.
DTXNameUseValueOf String For KonyOne based applications, use this property to use the serviceID runtime to fetch the operation name, which in turn is used as the wrapping action name.
DTXIncludeAllPackages boolean The auto-instrumentor instruments all classes. Set the value to false, if you only want to instrument a part of your app (see DTXIncludePackages). The default value is true.
DTXIncludePackages String Select a list of packages which should be instrumented. Separate the packages names with a comma.
DTXExcludePackages String Select a list of packages, classes, or methods which should not be instrumented. Separate the list elements with a comma.
DTXVersionCode int The default value is the version code value of your APK. The auto-instrumentor changes the instrumented APK file version code if you use a different value.
DTXVersionName String The default value is the version name value of your APK. The auto-instrumentor changes the instrumented APK file version code if you use a different value.
DTXInstrumentGPSLocation boolean The default value is false. If set to true, the auto-instrumentor instruments your LocationListener classes and sends the captured location as metric to the server.
DTXLogLevel String Default logs of the mobile agent can't be deactivated. Debug logs can be activated with the value debug.

Requirements

  • Supported OS versions (dependence on Google Android SDK system requirements):
    • Windows Vista to 10; other versions may work.
    • Ubuntu 12 and 14, 32 or 64-bits with the right ia32-libs installed; other versions and Linux flavors may work.
    • Mac OSX 10.8 to 10.9; other versions may work.
  • Java SDK 1.8
  • On Linux and OSX make sure that <MobileAgentUnzipDir>/Android/auto-instrumentor/instrument.sh has the execute permission chmod +x instrument.sh.
  • Install ia32-libs on 64-bit Linux.

Limitations

  • The auto-instrumentor has to add the agent code into the primary .dex file. There are some limitations for multi-dex applications.
  • If you obfuscate your application and you use both manual instrumentation and auto-instrumentation, you must exclude the Dynatrace.jar library from obfuscation. See the section ProGuard Settings for more information.
  • The keystore password, if specified, is not secured. It is just as if you coded the password into the application. A possible attacker can retrieve the password by decompiling the application.
  • Auto-instrumentation only provides limited information for hybrid applications. See How to instrument a hybrid app for more information.
  • Avoid using special characters and spaces in file and path names (such as ( ) [ ] $ % *).  If you are having trouble instrumenting your app, rename your file to use only alphanumeric characters (a-z, A-Z, 0-9) and the dot character (.) and try again.
  • APK protection tools (like DexGuard) are not fully supported.

Multi-dex applications

There is a limit on the number of referenced methods in a given .dex file. The specification limit for this number is 65,536. Android refers to that limit as the 64K reference limit.

Android introduced the ART runtime with Android 5.0, which natively supports multidex applications. But for platforms prior to Android 5.0, Android uses the Davlik runtime, which only supports single-dex applications. With the multidex support library lets you use additional .dex files after the MultiDex.install() task completes. Therefore all classes loaded at the app start must be part of the primary .dex file (classes.dex).

Because the auto-instrumentor inserts the agent start-up routine into the application start procedure, the agent code has to be part of the primary .dex file. The auto-instrumentor moves some classes from the primary .dex file into another .dex file if there is not enough space in the primary .dex file, . For most applications, the default settings should work. But for some apps you have to fine-tune the settings.

Note

The auto-instrumentor analyzes class dependencies based on the byte code of for application start procedure. If you hide your dependencies, for example with reflection, the auto-instrumentor can't detect all your application start dependencies. If you use the Android Gradle properties multiDexKeepFile and/or multiDexKeepProguard, adding the specified classes also to the corresponding auto-instrumentor properties is recommended, or use the DTXMultiDexKeepFile property as descripted in the next section.

You should test the application start on a device with Android 4.4 or lower. If the application crashes with a java.lang.NoClassDefFoundError, then you have to add the missing class to the corresponding auto-instrumentation properties.

Fine-tune the multidex preparation step

Following properties fine-tune auto-instrumentation for mutlidex applications.

  • Use the DTXPrimaryDexLimit and DTXSecondaryDexLimit properties to configure the limit of referenced methods for the primary and the secondary .dex files.
  • Use the DTXMultiDexKeep and DTXMultiDexKeepFile properties to configure the auto-instrumentor internal main dex list.

The instrumentation step increases the number of referenced methods (depending on your implementation) and fails if the number of referenced methods exceeds the 64K limit. The auto-instrumentor also adds the agent code into the primary .dex file. For every .dex file, the number of referenced methods is analyzed and adapted before the instrumentation step. The auto-instrumentator adapts the number of referenced methods by moving a precalculated number of classes from the .dex file to another .dex file. Use the DTXPrimaryDexLimit and DTXSecondaryDexLimit properties to configure the final limit of referenced methods for this step. Use the DTXPrimaryDexLimit property to set the limit for the primary .dex file (classes.dex) and the DTXSecondaryDexLimit property sets the limit for the secondary .dex files, such as classes2.dex.

Currently, the auto-instrumentor can't calculate the number of added methods and therefore uses default values for the limits. The default value for DTXPrimaryDexLimit is around 62K and for DTXSecondaryDexLimit it is nearly 64K. This configuration should fit most applications and also keeps the changes to a minimum. If the auto-instrumentation fails for your application, then you have to update the two properties.

For the primary .dex file the auto-instrumentor generates an internal main dex list and does not move the classes to another .dex file. If using the Android Gradle plugin to build you APK file, set the DTXMultiDexKeepFile property. The Android Gradle plugin generates a maindexlist.txt file. This file contains a list of classes, which have to be part of the primary .dex file. You should specify this file with the DTXMultiDexKeepFile property. For example:

DTXMultiDexKeepFile=<path_to_project>/<project>/<module>/build/intermediates/multi-dex/<build_variant>/maindexlist.txt

You can also specify you own class list in a file, to be part of the primary .dex file. Only one class per line is accepted. The class name must end with the .class suffix. Separate the package parts with a slash (for example com/example/Foo.class). You can only specify one file for the DTXMultiDexKeepFile property. The auto-instrumentor does not analyze the dependecies of the specified classes. You have to track them manually.

Use the DTXMultiDexKeep property if you don't want to track dependencies manually. The auto-instrumentor analyzes the specified classes and keeps these classes with their dependencies in the primary .dex file. Use the DTXMultiDexKeep to create a list of packages and classes. For example:

DTXMultiDexKeep=com.example.*, com.example2.Foo

This property is very useful when for resolving a java.lang.NoClassDefFoundError message on application start-up for an Android 4.x device. Add the classes (mentioned in the error message) to the DTXMultiDexKeep property. The auto-instrumentor detects the class dependencies.

Build an ajusted APK file

If you want to reduce the execution time of the auto-instrumentor, you can also fine-tune the Android build process to generate an adjusted APK file. With this file, you can skip the multidex preparation step from the auto-instrumentor. Ensure there is enough space for the mobile agent in the primary .dex file. You can reach this goal with the --minimal-main-dex or --set-max-idx-number option from the dx command. You can set this option with using additionalParameters property from the Android Gradle plugin.

--minimal-main-dex option: Use this option to have Android store classes only needed for the application start procedure in the primary .dex file.

dexOptions {
        additionalParameters '--minimal-main-dex'
}

--set-max-idx-number option: Use this option to have Android reduce the referenced method limit to the specified value. This setting is used for all your APK .dex files.

dexOptions {
        additionalParameters '--set-max-idx-number=62000'
}

Note

Before you modify your application settings, you should aware of the main dex list and how you can add missing classes to the main dex list.