Testing and Optimizing of what I call “traditional” page-based web applications is not too hard to do. Take CNN as an example. You have the home page www.cnn.com. From there you can click through the news sections such as U.S, World, Politics, … – each click loading a new page with a unique URL. Testing this site is rather easy. Use Selenium, WebDriver or even a HTTP Based Testing tool to model your test cases. Navigate to all these URLs and – if you are serious about performance and load time – use tools such as YSlow, PageSpeed or dynaTrace AJAX Edition (these tools obviously only work when testing is done through a real browser). These tools analyze page performance based on common Best Practices for every individually tested URL.
Let me explain the differences between these traditional vs. modern web applications and let me give you some ideas on how to solve these new challenges.
Optimizing Individual Pages has become straight forward – everybody can do this
Single Page Web 2.0 Applications – that’s the next challenge!
I am sure you use Google on a regular basis, whether it is Google Search, Docs or Gmail. Work with these Web Apps and pay attention to the URL. It hardly ever changes even though you interact with the application by clicking on different links. Not all of these actions actually cause a new page with a different URL to be loaded. Let’s look at a simple example. I open my local Google Search (in my case it is Google Austria) and get to see the search field in the middle of the page. Once I start typing a keyword two things happen:
- the search field moves to the top and Google shows me an Instant Results based on the currently entered fraction of the keyword
- a drop down box with keyword suggestions pops up
But – the URL stays the same – I am still on www.google.at. Check out the following screenshot:
When we now look at the dynaTrace timeline we see all these Actions and the browser activities corresponding to these actions:
The question that comes up is – well – is the page performance good? Does Google follow their own Best Practices?
Analyzing Page Load Time is not enough
If we want to analyze this scenario with PageSpeed in Firefox we run into the problem that PageSpeed as well as YSlow are focused on optimizing Page Load Time. These Tools “just” analyze loading of a URL. In our Google Scenario it is just the loading of the Google Home Page – which with no surprise – gets a really good score:
Why this doesn’t work? In this scenario we miss all the activities that happen on that page after the initial page was loaded.
Analyzing All Activities on a page delivers wrong results
How to test and optimize Single Page Web 2.0 Applications?
For my Google Search example I will use WebDriver. It works well across Firefox and Internet Explorer. It gives me access to all DOM elements on a page which is essential for me to verify if certain elements are on the page, whether they are visible or not (e.g.: verifying if a the suggestion drop down box becomes visible after entering a key) and what values certain controls have (e.g.: what are the suggested values in a Google Search).
Object Pattern for Action based Testing
The following is the test case that I implemented using WebDriver. It is really straight forward. I open the Google Home Page. I then need to make sure we are logged in because Instant Search only works when you are logged in. I then go back to the main Google Search Page and start entering a keyword. Instead of taking the search result of my entered keyword I pick a randomly suggested keyword:
The script is really straight forward and fairly easy to read. As you can see I implemented classes called GoogleHomePage, GoogleSuggestion or GoogleResultPage. These classes implement the actions that I want to execute in my test case. Let’s look at the suggestionsFor method implemented on the GoogleHomePage class returning a GoogleSuggestion object:
The code of this method is again not that complicated. What you notice though is that I added calls to a dynaTrace helper class. dynaTrace AJAX Edition allows me to set Markers that will show up in the Timeline View (for details on this see the blog post Advanced Timing and Argument Capturing). The addTimerName method is a method exposed by a premium version of dynaTrace which we will discuss in a little bit.
Timestamp based Performance Analysis of Web 2.0 Actions
When I execute my test and instruct WebDriver to launch the browser with the dynaTrace-specific environment variables, dynaTrace AJAX Edition will automatically capture all browser activities executed by my WebDriver script. Read more on these environment variables on our Forum Post Automation with dynaTrace AJAX Edition.
Let’s have a look at the recorded dynaTrace AJAX Session that we get from executing this script and let’s have a look at the Timeline that shows all these actions that got executed to get the suggestions as well as clicking on a random link:
Action/Timer based analysis that allows automation
Instead of looking at these metrics manually dynaTrace supports us with automatically detecting regressions on individual metrics per Timer Name (=Action). If I run this test multiple times dynaTrace will learn the “expected” values for a set of metrics. If metrics fall outside the expected value range I get an automated notification. The following screenshot shows how over time an expected value range will be calculated for us. If values fall out of this range we get a notification:
Conclusion: Best Practices only work on Page Load Time but not on Web 2.0 Action Based Applications