Unit/integration and performance tests

Java test framework

To capture data, enable the Java Tests Sensor Packs in the System Profile Preferences. The Java Tests pack captures data from JUnit and TestNG. 

Java tests sensor pack

Placing the Java Tests Sensor Pack instruments the following methods.

Framework Instrumentation Details
JUnit All methods annotated with org.junit.Test are instrumented unless the annotation uses the expected property for expected Exceptions; this feature is not supported by the AppMon Sensor.

Methods annotated with org.junit.Ignore are not instrumented.

Any method with a name starting with test and testclass is considered a subclass of junit.framework.TestCase, to maintain compatibility with JUnit 3.x.
Test NG All methods annotated with org.testng.annotations.Test are instrumented, unless the annotation property expectedExceptions is used or the enabled property is set to false.

Some other methods are also instrumented to gain additional information on, for example, parameterized tests: constructor of classed annotated with org.junit.runner.RunWith and the internal method org.junit.runners.Parameterized$TestClassRunnerForParameters.computeParameters().

.NET test framework

To capture data, enable the .NET Tests Sensor Packs in the System Profile Preferences. The .NET Tests pack captures data from NUnit and Visual Studio Unit Testing Framework.

.NET tests sensor pack

Placing the .NET Tests Sensor instruments the following methods.

Framework Instrumentation Comment
Visual Studio Unit Testing Framework Classes with the attribute Microsoft.VisualStudio.TestTools.UnitTesting.TestClassAttribute and methods with the attribute Microsoft.VisualStudio.TestTools.UnitTesting.TestMethodAttribute, are instrumented, unless the method has the attribute Microsoft.VisualStudio.TestTools.UnitTesting.ExpectedExceptionAttribute.
NUnit Classes with the attribute NUnit.Framework.TestFixtureAttribute and methods with the attribute NUnit.Framework.TestAttribute or NUnit.Framework.TestCaseAttribute are instrumented; also all test() methods (for backward compatibility), unless the method has the attribute NUnit.Framework.ExpectedExceptionAttribute.

Instrumentation

Depending on the technology and test framework, you must instrument processes for automated testing.

Sensorpack Framework Process to instrument
.NET Tests NUnit nunit.exe
.NET Tests Visual Studio Unit Testing Framework (tests run directly from Visual Studio) vstest.executionengine.x86.exe
.NET Tests Visual Studio Unit Testing Framework (tests run from command line with vstest.console.exe) vstest.console.exe,
QTAgent.exe, QTAgent32.exe,
QTAgent_35.exe, QTAgent32_35.exe,
QTAgent_40.exe, QTAgent32_40.exe
.NET Tests MSTest (tests run with the console application MSTest.exe) QTAgent.exe, QTAgent32.exe,
QTAgent_35.exe, QTAgent32_35.exe,
QTAgent_40.exe, QTAgent32_40.exe
.NET Tests MSTest (tests run with reSharper) JetBrains.ReSharper.TaskRunner.CLR4.MSIL.exe

See .NET Agent Configuration for information on how to instrument .NET processes.

Known limitations

When using vstest.console.exe with the command-line option /EnableCodeCoverage, data capturing does not work. This usually applies if you analyze code coverage on the build service. See Analyzing code coverage in the build service for information on how to enable/disable this option from the Visual Studio Team Explorer.

PHP test framework

To capture data, enable the PHP Tests Sensor Packs in the System Profile Preferences.

Measures

Measures based on following Metric / Metric Groups are captured during test executions:

Unit Tests

Category Metric Name Notes
Remoting Count  
Database Execution Count  
Methods Invocation  
Exceptions Count  
Logging Count  
Web Requests Number of Requests  
Web Services Count  

Performance Tests

Category Metric Name Notes
Remoting Count  
Database Execution Count  
Methods Invocation  
Exceptions Count  
Logging Count  
Web Requests Number of Requests  
Web Services Count  
PurePaths PurePath response time  
PurePaths PurePath CPU duration  
PurePaths PurePath duration  
PurePaths PurePath duration without suspension