Comey Hearings: What digital experience monitoring means to news media

Politics aside, today’s testimony by James Comey provides a fascinating look at how events can impact Digital Experience Monitoring for news media organizations. I’m using the term Digital Experience Monitoring (DEM) because the industry (Including Gartner and Forrester Research) has identified that Digital Experience needs to be considered and managed in a unique way. DEM draws the relationship between performance, availability, and end-user/consumer behavior when they interact with digital properties like web sites, mobile applications, etc.

I’m looking at various news providers using Dynatrace technology to illustrate how complex web applications require a new methodology and approach for understanding DEM impact.  Performance metrics are key here and I’ll explain more on this in a moment.

To give you an example of what we are seeing, below is a performance comparison of 20 different news media organizations as observed from locations across the US.

As you can see there is a huge difference from a performance perspective between these different news media organizations. The performance of these sites can be impacted by a wide variety of variables. Some of these include object size (page weight), cached resources, third-party contributors, client side code (javascript), and even server-side responsiveness.

Below is an example of how Dynatrace analyzes an individual page load for a real user and identifies key performance items.

On which web performance metrics should digital experience monitoring focus?

Below is an example of an Analysis Dashboard of the front page for a major news outlet.

Let’s go over what these “tiles” would tell a digital business owner at a news organization.

The Response Time & Success Rate tile (top left) provides a performance trend view which shows aberrations and events which could be impacting end users. It’s also useful to know when you have recovered from an event and the degree to which performance has been change by an event.

The Geographic Response Time tile (bottom left) shows response time by specific regions.  This is important especially if you are using a CDN (Content Delivery Network) as high regional response times can be associated with an oversubscribed PoP (CDN Point of Presence) or misrouted traffic. CDN services are expensive, and this is a way to help manage your technology investments.

The Contribution by Domain tile (second in from the top) highlights the impact that third parties like social media, ad networks, analytics/tracking tools are having on end-user performance.  This view helps you manage technology investment and risk associated with a third-party touching your customer.

The Key Delivery Indicators tile (second in from the bottom) shows observed byte count (how much data was delivered). This often gets overlooked by retailers but will show issues related content that is not optimized (what happens when the creative team release a 10MB juggling monkey image to the landing page), or malicious activity (what happens when a hacker re-routes your site to their page). Metrics like Object, Connection and Host count also provide an indication as to the complexity of the site and if something unexpected is occurring.

Let’s switch to the right side of the screen.

The DNS performance tile (top second from the right) shows DNS resolution time. DNS can be thought of as phone book, routing site names to server addresses. Again this often goes overlooked. However the DDoS attack on DNS provider Dyn on October 21, 2016 shows that DNS is critically important. Knowing when/if your DNS is being impacted allows you to make changes and recover faster. It also allows you to understand if you are investing in the right partner for providing DNS.

The Network Latency tile (top right) is a measure of how healthy your network connections are. This data can be used to understand if you have peering issues with your network providers, or if your network infrastructure (load balancer) is under pressure.

The Server Response time tile (bottom, second from the right) is a measure of how fast the server can respond to a request. This allows you to understand from an end point of view if the server applications are causing a performance bottleneck, we will come back this later.

The last tile on the bottom right, shows Client Impression time.  This allows you to understand how long does it take for the browser/mobile browser to display something for the end user.  Understanding what is happening in the user’s browser is the final link in the chain.

Digital Experience Monitoring and top-line revenue

News Media organizations primarily generate revenue through displaying advertisements to readers/viewers and subscriptions. When it comes to driving revenue from ad impressions, keeping the user on the site is key. This is what the industry calls “stickiness”. Performance is a key contributor to end-user behavior. One of the ways Dynatrace tracks this is by executing a Bounce Rate analysis. In the graph below you can see that as performance worsens (the time along the bottom) the higher the bounce rate becomes. These are readers/viewers navigating away (bouncing) from the site because the page takes too long to load.

If performance is poor, users will not remain on the site and the number of ad impressions will drop. This directly impacts the top-line revenue generation for a news media website.

Also, code on the page can cause issues which prevent an ad from loading or being seen.  Below is an example of how Dynatrace discovers a Javascript Error on a page. You can see a screenshot showing a blank region of a page where an ad should be located.

When we look at the Javascript error we can see it is an issue with code coming from an ad provider which is failing and causing the ad to not display.

These JavaScript errors also impact top-line revenue for a news media outlet because there is no ad displaying for a reader/viewer when the error occurs.

Digital Experience Monitoring needs insight into the back end

We mentioned that performance can impact top-line revenue when readers/viewers bounce off a news site. One of the contributing factors to poor performance comes from the “back end”. The “back end” in this case refers to servers which respond to page requests, are hosted by the news site or cloud-based servers and services.

Below is a comparison of ten news media companies which provide the fastest server-side “back end” response times, and ten news media companies providing the slowest response times. The fastest sites provide response times faster than 200 milliseconds from their servers, and the slower outlets can take over a half a second.

While these response times might sound fast, the slower server-side response times can be expensive for the news outlet (and not just for the reader/viewers bouncing off slow pages). When you add up the processing required to service millions of visits, the sites providing the slower response times are paying more to service the same number of viewers as the sites providing the faster response times. This is all about computational capacity. The slower a transaction is on the server side the more compute resources it consumes. Compute resources, whether or not you host then yourself or use them from the cloud, cost money.

The applications which run these news websites and mobile apps are exceedingly complex. The complexity is so great that effective DEM data needs to be augmented with Artificial Intelligence based analyses to understand all of the dependencies which exist. Below is an example of a Dynatrace Smartscape automatically discovering all of the compute resources that would exist for any of the news media organizations we looked at today.

What’s going on behind the curtain?

While everyone is watching the political theater today, what interests me is happening behind the scenes. Events like this drive traffic to news outlets, however, depending on how that news site is being delivered, there can be a substantial impact on digital experience, which can lead to frustrated readers/viewers bouncing off the site. Poor digital experience impacts the ability to generate revenue from ad impressions for news sites. The news is a highly-competitive market, and the technology driving it is increasingly complex. To remain competitive news sites need to look for new ways to managing their digital experience.

OK, back to watching some political theater.

Stay updated