Regression analysis

Regression analysis estimates the differences between application executions. If you fix bugs, introduce new features, or refactor an application, you affect other application areas. Regression analysis estimates differences in application performance if you:

  • Have two different application versions or releases.
  • Execute the application on two different environments or platforms.
  • Change third-party libraries.

Goal of this tutorial

To describe how to do a simple regression analysis based on a problem pattern in easyTravel.

Scenario

Assume that you have recently refactored your application and modified some APIs. Also, you fixed some bugs and created new features. The main features remain the same. You need to determine whether the changes affect the main features negatively or positively. You can do it by comparing two stored sessions—the one created before the refactor and the other created after refactor.

To imitate the deployment with easyTravel, you can record the first session under standard scenario, and the second one under the same scenario and additional problem pattern. Then you can explore the differences in AppMon Client. For example, you can record the first session under UEM > Lightweight scenario, and for the second session add the SlowAuthentication problem pattern. This pattern causes slow login for certain users.

You can record those sessions manually in the AppMon Client, and also manually use easyTravel web page to log in or you can automate any of these. On pre-production editions you can use Live Session REST interface via any REST client to automate session recording. To automate logins you can use any web UI testing software, for example the Selenium IDE plugin for Mozilla Firefox.

Requirements

Detailed steps

Here's a high level overview of the process:

  1. Record the "pre-deployment" session under UEM > Lightweight easyTravel scenario.
  2. Record the "after-deployment" session under UEM > Lightweight easyTravel scenario with addition of SlowAuthentication problem pattern.
  3. Create the Regression Analysis dashboard to show the performance differences.

If you already have two comparable stored sessions, you can go straightforward to the Step three.

1. Record the "pre-deployment" session

At first you need to record a session, reflecting system behavior before deployment of the new code.

  1. In the easyTravel configuration UI, click UEM > Lightweight to start a scenario.
  2. In the Agent Overview dashlet, make sure that all Agents are connected to the AppMon Server.
  3. In the PurePaths dashlet, make sure you get events and PurePaths in your live session.
  4. Start session recording. Use the Start session recording REST command or do it manually:
    • AppMon 2017 May In the Cockpit right-click the System Profile, and select Session Storage > Start Recording from the context menu.
    • AppMon 2018 April In the sidebar, click Settings > Start Session Recording.
  5. In the Start Session Recording dialog box, give session the pre-deployment name, select the PurePaths and time series radio button in the Content pane, then click Record.
    Start session recording
  6. Go to the easyTravel web page.
  7. In the top right corner, click Login.
  8. Click the lock icon icon and select any username to log in.
  9. Click Logout.
  10. Repeat steps 7 to 9 several times.
  11. Stop session recording. Use the Stop session recording REST command or do it manually:
    • AppMon 2017 May In the Cockpit right-click the System Profile, and select Session Storage > Stop Recording from the context menu.
    • AppMon 2018 April In the sidebar, click Settings > Stop Session Recording.

2. Record the "after-deployment" session

Now you need to record a session, reflecting system behavior after deployment of the new code. To imitate a problem, use the SlowAuthentication problem pattern of easyTravel. It creates noticeable lags on login for certain users and also puts heavy load to the database.

  1. In the easyTravel configuration UI, click Problem Patterns > SlowAuthentication.

    Note

    If you don't see the Problem Patterns tab, click the gear icon in the top right corner and select Show Problem Patterns.

  2. Start session recording. Use the Start session recording REST command or do it manually:

    • AppMon 2017 May In the Cockpit right-click the System Profile, and select Session Storage > Start Recording from the context menu.
    • AppMon 2018 April In the sidebar, click Settings > Start Session Recording.
  3. In the Start Session Recording dialog box, give session the after-deployment name, select the PurePaths and time series radio button in the Content pane, then click Record.

  4. Go to the easyTravel web page.

  5. In the top right corner, click Login.

  6. Click the lock icon icon and select either monica or hainer username to log in.

  7. Click Logout.

  8. Repeat steps 5 to 7 several times.

  9. Stop session recording. Use the Stop session recording REST command or do it manually:

    • AppMon 2017 May In the Cockpit right-click the System Profile, and select Session Storage > Stop Recording from the context menu.
    • AppMon 2018 April In the sidebar, click Settings > Stop Session Recording.

3. Create the Regression Analysis dashboard

Now, when you have the required data, you can compare it.

  1. In the AppMon Client, create a new empty dashboard.
  2. Right-click the dashboard tab, and select Properties from the context menu.
  3. In the Properties dialog box, click the Source tab.
  4. AppMon 2018 April only Clear the Use selected client data source checkbox.
  5. Set the pre-deployment session as a data source.
    Source - first session
  6. Click the Comparison tab.
  7. Select the Compare with checkbox.
  8. Set the after-deployment session as a data source.
    Comparison - second session
  9. Click OK to save the changes.
    Now all the dashlets of this dashboard will show the difference between two sessions.
  10. Add the following dashlets to the dashboard:
  11. Drag dashlets to arrange them in convenient way.
    Regression analysis dashboard
  12. As you can see, due to the SlowAuthentication artificial regression, some timings differ drastically. You may want to explore this deeper.

Conclusion

AppMon provides regression information and quickly puts you on the path to discovery, even if you are not familiar with the application. For best results, use the AppMon automation interface to perform regression analysis in an automated environment.