It is a fact that end user response time is critical for business success. The faster web pages are perceived, the longer users tend to stay on the page and therefore spend more money and drive business.In order to ensure that end user response times are acceptable at all times it is necessary to measure the time in the way the end user perceives performance. Measuring and monitoring your live system is important to identify problems early on before it affects too many end users. In order to make sure that web pages are fast from the start it is very important to constantly and continuously measure web page performance throughout the development phase and in testing. There are two questions that need to be answered
- What is the time the user actually perceives as web response time?
- How to measure it accurately and in an automated way?
What time to measure? Technical Response Time vs. Perceived Response Time
From this analysis it’s hard to tell what the perceived end user response time really is. Is it 1.6 seconds because that is the time when the browser could already start rendering the initial content of the HTML document? Or is it roughly 5 seconds when the first batch of embedded objects was fully downloaded? Or might it be 8 seconds – because that is the time till the last request was completed? Or is the truth somewhere in between?
There is more than meets the “HTTP Traffic” Eye
So what is the perceived end user performance?
I believe there are different stages of perceived performance and perceived response time.
The First Impression of speed is the time it takes to see something in the browsers window (Time To First Visual). We can measure that by looking at the first Rendering (Drawing) activity. Get a detailed description about Browser Rendering and the inner workings the Rendering Engine at Alois’s blog entry about Understanding Browser Rendering.
The Second Impression is when the initial page is fully loaded (Time To OnLoad). This can be measured by looking at the onLoad event which is triggered by the browser when the DOM is fully loaded meaning that the initial document and all embedded objects are loaded.
Let’s look at a second example and identify the different impression stages. The following image shows a page request to a product page on a very popular online retail store:
How to measure? Stop Watch Measuring vs. Tool Supported Measuring
The idea for this blog post came from talking with performance testing engineers at on of our clients. I introduced them to the Dynatrace AJAX Edition and was wondering about a small little gadget they had on their table: a Stop-Watch.
Their task was to measure end-user response time for every build of their new web-site in order to verify if the times are within defined performance thresholds and in order to identify regressions from build to build. They used the Stop-Watch to actually measure the time it took to load each single page and to measure the time till the page was responsive. The “manually” measured numbers were put into a spreadsheet which allowed them to verify their performance values.
Do you see the problems in this approach?
Not only is this method of measuring time very inaccurate – especially when we talk about measuring precise timings in tenths of seconds. Every performance engineer also has a slightly different perception of what it means for the site to be interactive. It also involves additional manual effort as the timing can only be taken during manual tests.
Automate measuring and measure accurately
When using the Dynatrace AJAX Edition as seen on the examples above all performance-relevant browser activities are automatically captured and enable us to determine the time of the 3 Impression Stages. The blog article “Automate Testing with Watir” shows how to use Dynatrace AJAX Edition in combination with automated testing tools. The tool also provides the ability to export captured data to XML or spreadsheet applications like Excel – supporting the use case of automated regression analysis across different web site versions/builds.
Using tools like Dynatrace AJAX Edition for Internet Explorer, YSlow or PageSpeed for FireFox or DevTools for Chrome enables automating web site performance measuring in manual and automated test environments. Continuously measuring web site performance in the browser allows you to always focus on end user performance which in the end determines how successful your website will be.