Error analysis

To view the Error analysis page:

Use Error analysis to identify patterns of availability errors, focus on tests that have errors, and view information that can help you troubleshoot and resolve performance problems. For a selected test or for all tests, you can view the availability percentage, error types and counts, and the error distribution over agent locations, tests, and test-run times. At a glance, you can identify the most frequently occurring errors, and the tests and locations with the most errors.

From the Error analysis page, you can go to the Error list to get more details about the errors. From that page, you can view screen captures (if Screen Capture on Error is enabled for the test), and you can drill down through the Waterfall summary page to a waterfall chart for each step in the test execution.

Error analysis page

The Error analysis page displays:

  1. Aggregate test statistics.
  2. Circle charts for Error types, test Locations where errors occurred, and the Tests that have errors.
  3. An Error count chart that shows the number of errors over time.

Test statistics

When you drill down to the Error analysis page from the Operational summary page or Operations dashboard, the error statistics are calculated for the selected test. When you use the menu to go to the Error analysis page, the statistics are calculated based on all active tests that have errors.

These statistics are aggregated for the tests:

  • Availability – The percentage of successful test executions, calculated as
    (Total test Executions – Failed Test Executions)/(Total Test Executions)
  • Runs failing – The number of tests runs that failed.
  • Tests failing – The number of tests that failed.
  • Runs total – The total number of test executions for all included tests.
  • Tests total – The number of tests included in this Error analysis.
  • Error types – The number of different error types that occurred in all the test executions.
  • Locations failing – The number of measurement locations where test executions failed.

Circle charts

The interactive circle charts show the top 10 items, by percentage, for Error types, measurement Locations with errors, and Tests with errors.

For example, the Locations graph shows the locations with the highest percentage of availability errors. The location name and the percentage of the total availability errors are shown for each segment. The segment size corresponds to the percentage of errors.

When the Error analysis page displays data for just one test, the Tests chart contains segments for the test’s steps where errors occurred.

Click a chart segment to filter the page by that item, as described below.

If a circle chart has more than 10 items — for example, more than 10 Tests have errors — the top 9 items are graphed individually and the rest are grouped in a segment labeled Other. Click the Other segment to display a list of the items in the Other category. Click an item in the list to filter the Error analysis page.

Error count

The Error count chart at the bottom of the page shows the number of errors that occurred at specific times during the selected time range. The chart’s resolution depends on the time range. For example:

  • Last 1 hour – Every 15 minutes
  • Last 48 hours – Every 6 hours

Hover over a bar to display the error count and exact time.

You can use the Error count chart to filter the data, as described below.

Filtering the error analysis data

You can filter the aggregate view by:

  • Time
  • Error types
  • Measurement locations
  • Tests/steps

When you filter the page, the filters are listed across the top of the page, with the exception of the time range menu selection.

Aggregate statistics are recalculated based on the filter. For example, when you filter by a Location, the statistics are aggregated for the tests that ran on that location.

Filtering by time range

By default, the Error analysis page displays the time range that’s set for the Operational summary page.

To view error data for a specific time range, use the time range menu at the top of the page.

Changing the time range in the Error analysis page also changes it in the Operational summary page.

You can define a Quick or Custom time range.

Filtering by error count timestamp

To focus on error data from test executions at a specific time, click the bar for that time in the Error count chart at the bottom of the page. When you filter the page this way, the filter is displayed at the top of the page.

Using the circle charts to filter

Click a segment of a circle chart to filter the page by that item. Chart filters are cumulative: you can filter the page by an error type, and a location, and a test/step.

  • Error types – By default, the error types are grouped into categories: Network errors, timeouts, HTTP errors, etc. When you click an error group, the circle chart displays the error types in that group; for example, Network may drill down to show that DNS Lookup Failures and Connection Timeouts occurred.

  • Locations – By default, this chart shows the continents were errors occurred. Drill down through countries and cities to the measurement locations (Backbone nodes, peer populations, mobile carriers/sites).

  • Tests – By default, the tests with errors are grouped into test types. Click a test type to drill down to the individual tests. The top 10 tests with errors are displayed; all other tests are grouped into “Other”.

    • Clicking a Mobile, Last Mile, or Private Last Mile test drills down to the batch groups that contain the test; clicking a batch group drills down to the test’s steps. if the test is not in a batch group or is in only one, it drills down directly to the steps.
    • Clicking a Backbone test drills down to the steps.

When a circle-chart filter is applied, the filter is listed at the top of the page and the test statistics are recalculated for the filtered tests.

As you apply each filter, the other circle charts are filtered automatically; for example, when you click a Location that ran only Mobile tests, the Tests chart is filtered to display only those Backbone tests.

Disable or remove filters

Click a filter to temporarily disable it; the text is dimmed to show the filter is disabled. Click the filter again to enable it.

Click the X on the right side of a filter to remove it.

If a circle-chart filter is applied, you can also click the center of the chart to remove the filter.

Error list

To view the Error list, click Inspect errors at the top right of the Error analysis page.

The Error list displays errors for the time range selected in the Error analysis page. Inspect errors is available for a maximum 48-hour time range:

  • In the Quick time range menu, select a time range of Last 48 hours or less.
  • In the Custom menu, select start date within the last 45 days, with a maximum time range of 48 hours.

If filters are applied to the Error analysis page, the same filters are applied to the Error list.

The error table lists the following information:

  • Test – The test name, with icons that identify the test type and browser type.
  • Step – The step in which the error occurred, if the test has more than one step.
  • Error code – For more information, see the Error Codes help page.
  • Error – A brief description of the error.
  • Test time – The date and time of the test execution.

To sort the table, click a column head. By default, the table lists errors by Test time, from most recent to oldest.

Filtering the error list

To find specific errors in a long error list, you can filter the list by Error description, measurement Location, SCoE Available, Step name, and Test name.

Click the filter field to display the criteria list.

When you select a criterion, it is added to the field and a list of items matching that criterion appears. You can only select one item for each criterion.

Filters are cumulative. The list is immediately filtered when you add a criterion, so the next criterion you select displays only the items available in the filtered list. For example:

  • The unfiltered list displays errors for a test consisting of three steps. When you filter the list for an error type and then select Step for the next filter criterion, the Step list only displays the step(s) where that error occurred.
  • The selections for SCoE available are true (screen capture are available) and false. If none of the errors in the list have screen captures available, the selection list for SCoE available only displays false.

Viewing error details

Click the expand icon for an error to see the geographic Location, the Site (node or peer population), and the number of Objects that failed.

If Screen Capture on Error is enabled in the test settings, a thumbnail of the first screen captured is displayed. Click the thumbnail to view the screen capture in a popup window.

Drilldown for data analysis

Click View waterfall under the error details to go to the Waterfall summary page for the test execution. From the waterfall summary, you can drill down to the waterfall chart for a step.

If SCoE is enabled for the test, click View screen capture to open the Screen Capture page in a new browser tab. (You must allow popups for to be able to open the new tab.)