Advantages of Load Testing with Real Browsers

With Web load you have the ability to load test your applications using real browsers (Internet Explorer or Firefox).  There some distinct advantages to load testing with a real browser versus testing with a traditional HTTP load testing tool.  In this post we will be discussing the differences between testing with real browsers vs. HTTP agents and the advantages of both.

What are HTTP Agents and Why Do We Use Them?

Most traditional load testing tools utilize a HTTP proxy to record scripts. The scripts will record every HTTP/HTTPS request that’s made. Scripts are then played back in a custom agent that replicates each HTTP request that was recorded in the script.  During playback when a response is returned from a request typically no DOM or JavaScript is executed in the agent, and the body of each response is just interpreted as plain text.

The primary reason for using this type of script and playback agent is scalability.  Browsers consume a great deal of memory and typically do not scale well.  HTTP agents are designed to use a lot less memory than a regular browser, usually by eliminating memory intensive processes such as DOM parsing and JavaScript execution.  Typically you can simulate 10-20 times as many virtual users on a machine using an HTTP agent vs. a browser agent.  This is important when balancing limited financial and physical resources against the requirement to execute large tests that require tens of thousands of users.

Using Real Browsers to Load Test

While HTTP agents are an effective means for simulating HTTP traffic in a browser they are still not the real deal.  There are few distinct advantages to using a real browser which HTTP agents cannot replicate:

  • Ease of Scripting: In a HTTP agent, because no DOM or JavaScript is executed each individual request must be captured and replicated.  Given the extremely dynamic nature of today’s web applications this can be difficult.  In order to make requests, dynamic variables must be extracted or correlated from the response using pattern matching typically with regular expressions.  This can quickly become cumbersome and complex especially with pages that contain a large number of objects and dynamic values.Browser agents on the other hand are full browsers and are able parse the DOM and execute JavaScript just like the browsers we use every day.  This means pages are constructed as they would be in a regular browser window.  As a result scripts are based upon browser actions such as clicking, typing, or selecting from a dropdown.  This simplifies the script significantly because of instead of chaining raw HTTP requests together and extracting dynamic values with regular expressions you are just determining where on the page to click.
  • Parallel Connections: One of the key performance features of a browser is its ability to open multiple TCP/IP connections in parallel.  With more TCP/IP connections, more content can be downloaded simultaneously. This is one of the primary contributing factors to the difference in response time performance between the different browsers.  This also means that more connections will be opened on the web servers that the browser is requesting information from.  As the number of parallel connections in browsers continues to increase the more concurrent connections applications will need to be able to support.Most HTTP agents have difficulties accurately replicating how a browser manages parallel connections.  As a result a much lower number of connections will be opened during a load test than what may be experienced during a real life scenario.  This is significant when testing the ability of an application to support concurrency, especially when more costly HTTPS/SSL connections are involved.
  • Object Download Accuracy: In today’s Web 2.0 world the content on a page is becoming more and more complex, which means more and more objects need to be downloaded on each page.  Many of these objects are coming from CDN’s or third-party sources, and are very dynamic.  When you record a script using a proxy you are capturing the HTTP requests made at the time of the recording.  If objects on a page change or the order in which they should be downloaded changes, this will not be reflected in the script unless the script is re-recorded or manually edited.  Also, while HTTP scripts can do a good job of identifying dynamic variables and generating dynamic requests, they are not perfect and many dynamic requests can easily be not requested properly during playback.In a real browser this is not an issue.  Once the base page is downloaded the browser has all the instructions it needs to compile the page and the paths to the objects it needs to download.  This means that when using a browser agent you are much more confident that all the correct objects will be downloaded and in the order they should be downloaded in.

While there are some important advantages with using a real browser, HTTP agents still have their place. Web load utilizes both browser and HTTP agents for load testing.  In special situations, or when an extremely high volume of load is required, an HTTP may be the preferred option.  When testing web services, HTTP agents may present a simpler and more efficient process for scripting and testing.  However as a general best practice, use a browser based agent first when possible and then use a HTTP type agent when necessary.