PurePath® method-level transaction analysis

The first step in your analysis should be to locate the requests that you want to analyze at a granular level. Filters can be used to narrow down many thousands of requests to just those few that are relevant to your analysis. You have a couple of ways to achieve this result

Follow the root cause analysis drill-downs.
Segment your requests using the advanced filtering mechanisms in service flow and outlier analysis.

Ultimately, you’ll likely discover a handful of requests that require deeper analysis. This is where distributed-trace analysis comes in.

Detect a single request via service flow

You can narrow down your requests by segmenting them.

In the Dynatrace menu, go to Services.
Select the service you want to analyze.
On the service overview page, under Understand dependencies, select View service flow.

Select a called service > Apply filter to obtain a subset of requests based on its chain of calls (see example below).
Select the caller service > Distributed traces to access its traces filtered by the chosen chain of calls.
In the Configure view tile, select a node to browse the Filter requests list and apply filters.
Select a trace from the refined list of traces to proceed with its code-level analysis.

Example: how to narrow down requests via service flow.

In this example, the service easyTravel Customer Frontend received 249,000 requests during the selected 2-hour timeframe.

Let's suppose that you're interested specifically in the requests from easyTravel Customer Frontend that call first AuthenticationService (11,600 overall calls) and then easyTravel-Business (14,400 overall calls).

To focus on this subset of requests, select easyTravel-Business> Apply filter .

In the picture above, 94% of the easyTravel Customer Frontend requests calling AuthenticationService also call VerificationService. To add VerificationService as a second filter parameter, select VerificationService. Then, select easyTravel Customer Frontend > Distributed traces to access the list of distributed traces.

The Most recent traces list features the requests initiated by easyTravel Customer Frontend that match the filter criteria. You can filter the list or sort it by Start time, Name, Response time, Processing time, HTTP method, or Response code.

To visualize only easyTravel Customer Frontend requests with Response time slower than or equal to 80 ms

Select the easyTravel Customer Frontend node.
From the Filter requests list, select Response time.
Select greater than or equal to ≥, type 80 in the input field, and select Apply.
Select Apply.

Only 3 requests out of the initial 249,000 justify in-depth distributed trace analysis. Select a request to analyze its trace.

Distributed-trace analysis of a single web request

Distributed-traces analysis provides a waterfall visualization of all requests. Each service in the call chain is represented in the analysis.

The Execution breakdown section at the top of the page shows how the time spent by the transaction is distributed. The waterfall chart shows much more detail. You can see:

Which other services are called and in which order.
Other services called and the order in which they're called.
Subsequent calls to services called by the ones under analysis.

Like service flow, distributed-trace analysis provides end-to-end web-request visualizations—in this case, that of a single request.

The colors and positions of the horizontal bars in the chart indicate both the sequence and response time of each of the requests. You can easily see which calls were executed synchronously and which were executed in parallel.

By selecting one of the service or execution bars, you can analyze the details of each request in the distributed-trace analysis via multiple tabs. Depending on the type and the technology of the service under analysis, additional tabs, such as Threads and Integration, are available.

The Summary tab includes details on metadata, request headers, request parameters, and more. You can also see information about the proxy through which the request was sent.

You might encounter values obscured with five asterisks (*****). This happens because the values are confidential and the current user account doesn’t have permission to view confidential data. If the user account has permission to view personal data, the values are visible.

Personal data and data obscured for aggregation

Five asterisks (*****) imply hidden confidential data, which can be unmasked by users with sufficient permissions. Three or fewer asterisks (***) are used for data aggregation purposes and can't be unmasked.

The Timing tab hosts timing-specific details of services. The example below shows timing details for the /orange.jsf web service call. In this case, you see that the request lasts 57.7ms. More details on timing contributions (for example, CPU, wait, sync, lock, and execution time) are available by selecting View method hotspots.

Each request executes some code, for example, Java, .NET, PHP, Node.js, Apache webserver, NGINX, or IIS. The distributed traces view enables you to look at the code execution of each and every request. Simply select a particular service and select the Code level tab.

This view shows you code-level method executions and their timings. Dynatrace tells you the exact sequence of events with all respective timings. As you can see above, Dynatrace tells you exactly which method in the /orange.jsf request on the easyTravel Customer Frontend service called the respective web services and which specific web-service methods were called. The timings displayed here are the timings as experienced by the easyTravel Customer Frontend, which, in the case of calls to services on remote tiers, represent the client time.

Notice that some execution trees are more detailed than others. Some contain full-stack traces. Dynatrace automatically adapts the level of information it captures based on importance, timing, and estimated overhead. Because of this, slower parts of a request typically contain more information than faster parts.

You can look at any request in the distributed traces view and navigate between the respective code-level trees. This gives you access to the full execution tree.

By combining logs with distributed traces, you can check log records in the full context of a transaction. For a selected waterfall node, all available span IDs are included by default. You can filter by logs relevant to the selected waterfall node by turning on Strict span filter.

To learn how to enhance your distributed-trace analysis with logs, see the Log analysis with PurePath® technology page.

The Errors tab is available for failed requests. Here you can analyze occurring exceptions. To learn more about PurePath® error analysis, visit the dedicated page.

Analyze a called service

Each distributed trace tracks a request from start to finish. This means that the traces always start at the first fully monitored process group. However, you might want to change perspective and focus only on a service in the call chain.

In the Dynatrace menu, go to Services.
Select the service you want to analyze.
On the service overview page, under Understand dependencies, select View service flow.

Select the last called service > Apply filter to obtain a subset of requests based on a specific chain of calls.
Select the called service you are interested in > Distributed traces to access its traces filtered by the chosen chain of calls.
optional To apply filters, in the Configure view tile, select the node of the choosen called service.
Select a trace from the refined list of traces to proceed with its code-level analysis.

You can always go back and extend your analysis:

To see where the request originated, you can select More > Service backtrace.
To see the entire trace from the first fully monitored process group, select Show full trace.

Example

In the call chain in this example, easyTravel Customer Frontend is the first service and it calls AuthenticationService and VerificationService. To look into the called service AuthenticationService's requests, select AuthenticationService > Distributed traces .

Filtering AuthenticationService traces by response time, you can choose to exclusively view requests slower than 15ms. The refined list now shows requests of AuthenticationService slower than 15ms called by easyTravel Customer Frontend when the frontend request also calls VerificationService.

To analyze AuthenticationService requests without including the easyTravel Customer Frontend service in the analysis, select a trace.

To look at the same distributed trace from the perspective of the first caller service, easyTravel Customer Frontend, select Show full trace.

This is the same distributed trace we began our analysis with. You can still see the AuthenticationService call and its two database calls, but now they are embedded in a larger request.