A while back I hosted a Webinar with Ron Woody, Director of Performance at GSI Commerce (now part of eBay). Ron and his team are users of dynaTrace – both AJAX and Test Center Edition. During the Webinar we discussed the advantages and challenges that Web 2.0 offers – with a big focus on eCommerce.

This blog is a summary of what we have discussed including Ron’s Tips and Tricks and Best Practices. The screenshots are taken from the original webinar slide deck. If you want to watch the full webinar you can go ahead and access it online.

Web 2.0 – A chance for eCommerce

Web 2.0 is a great chance to make the web more interactive. Especially for eCommerce sites it brings many benefits. In order to leverage the benefits we have to understand how to manage the complexity that comes with this new technology.

The Benefits of Web 2.0

JavaScript, CSS, XHR, … – that’s what makes interactive web sites possible – that’s what many of us consider Web 2.0.

When navigating through an online shop users can use dynamic menus or search suggestions to easier find what they are looking for. Web 2.0 also eliminates the need of full page reloads for certain user interactions, e.g: display additional production information when hovering the mouse over the product image. This allows the user to become more “productive” by reducing the time it takes to find and purchase a product.

Web 2.0 allows us to build more interactive web sites that support the user in finding the right information faster
Web 2.0 allows us to build more interactive web sites that support the user in finding the right information faster

The Challenges

The challenge is that you are not alone with what you have to offer to your users. Your competition leverages Web 2.0 to attract more users and is there for those users that are not happy with the experience on your own site. If your pages are slow or don’t work as expected online shoppers will go to your competitor. You may only lose them for this one shopping experience – but you may loose them forever if the competitor satisfies their needs. Worse than that – frustrated users share their experience with their friends and impacting your reputation.

Performance, Reliability and Compatibilty keep your users happy. Otherwise you loose money and your reputation
Performance, Reliability and Compatibility keep your users happy. Otherwise you lose money and your reputation

The Complexity of Web 2.0

Performance Optimization “was easier” before we had powerful browsers supporting JavaScript, CSS, DOM, AJAX, …

When we take a look at a Web 2.0 Application we have to deal with an application that not only lives on the Application Server whose generated content gets rendered by the browser. We have an application that spawns both server and client (browser). Application and navigation logic got moved into the browser to provide better end-user experience. Web 2.0 Apps leverage JavaScript frameworks that make building these apps easier. But – just because an app can be built faster doesn’t mean it operates faster and without problems. The challenge is that we have to understand all the moving pieces in a Web 2.0 Application as outlines in the following illustration:

Web 2.0 Applications run on both Server and Client (Browser) using a set of new components (JS, DOM, CSS, AJAX, ...)
Web 2.0 Applications run on both Server and Client (Browser) using a set of new components (JS, DOM, CSS, AJAX, …)

Performance in Web 2.0

With more application logic sitting in the Client (Browser) it becomes more important to measure performance for the actual end-user. We need to split page load times into time spent on the Server vs. time spent on the Client (Browser). The more JavaScript libraries we use, the more fancy UI effects we add to the page and the more dynamic content we load from the server the higher the chance that we end up with a performance problem on the Client. The following illustration shows a performance analysis of a typical eCommerce user scenario. It splits the end-user’s perceived performance into Time spent in Browser and Time spent to download (Server-Time):

Up to 6 seconds spent in the browser for Shipping, Payment and Confirm
Up to 6 seconds spent in the browser for Shipping, Payment and Confirm

Users get frustrated when waiting too long for a page. Optimizing on the Server-Side is one aspect and will speed up page load time. The more logic that gets moved to the browser the more you need to focus on optimizing the browser side as well. The challenge here is that you cannot guarantee the environment as you can on the server-side. You have users browsing with the latest version of Chrome or Firefox, but you will also have users that browser with an older version of Internet Explorer. Why does that make a difference? Because JavaScript engines in older browsers are slower and impact the time spent in the Browser. Older browsers also have a limited set of core features such as looking up elements by class name. JavaScript frameworks such as jQuery work around this problem by implementing the missing features in JavaScript – which is much slower than native implementations. It is therefore important to test your applications on a broad range of browsers and optimize your pages if necessary.

How GSI Commerce tames the Web 2.0 Beast

GSI Commerce (now eBay) powers sites such as NFL, ToysRUs, ACE, Adidas, …

In order to make sure these sites attract new and keep existing online users it is important to test and optimize these applications before new versions get released.

Business Impact of Performance

Ron discussed the fact that Performance indeed has a direct impact on business. We’ve already heard about the impact when Google and Bing announced the results on their performance studies. GSI confirms these results where poor performance has a direct impact on sales. Here is why:

  • Our clients’ competitors are only a click away
  • Poor performance increases site abandonment risk
  • Slow performance may impact brand

Client and Server-Side Testing

GSI does traditional server-side load testing using HP LoadRunner in combination with dynaTrace Test Center Edition. They execute tests against real-life servers hosting their applications. On the other side they also execute client-side tests using HP Quick Test Pro with dynaTrace AJAX Edition to test and analyze performance in the browser. They also leverage YSlow and WebPageTest.org.

GSI Browser Lab

Online Users of Web Sites powered by GSI use all different types of browsers. Therefore GSI built their own Browser Lab including all common versions of Internet Explorer and Firefox. Since dynaTrace AJAX also supports Firefox they run dynaTrace AJAX Edition on all of their test machines as it gives them full visibility into Rendering, JavaScript, DOM and AJAX. They use HP Quick Test Pro for their test automation:

GSI Browser Lab is powered by dynaTrace AJAX and includes multiple versions of Internet Explorer and Firefox
GSI Browser Lab is powered by dynaTrace AJAX and includes multiple versions of Internet Explorer and Firefox

How GSI uses dynaTrace

While HP Quick Test Pro drives the browser to test the individual use cases, dynaTrace AJAX captures performance relevant information. GSI uses a public API to extract this data and pushes it into a web based reporting solution that helps them to easily analyze performance across browsers and builds. In case there are problems on the browser side the recorded dynaTrace AJAX sessions contains all necessary information to diagnose and fix JavaScript, Rendering, DOM or AJAX Problems. This allows developers to see what really happened in the browser when the error happened without having them trying to reproduce the problem. In case it turns out that certain requests took too long on the application server GSI can drill into the Server-Side PurePath’s as they also run dynaTrace on their Application Servers.

Using dynaTrace in Testing bridges the gap between Testers and Developers. Capturing this rich information and sharing it with a mouse click makes collaboration between these departments much easier. Developers also start using dynaTrace prior to shipping their code to testing. Ron actually specified an acceptance criteria. Every new feature must at least have a certain dynaTrace AJAX Page Rank before it can go into Testing.

Besides running dynaTrace on the Desktops of Developers and Testers, GSI is moving towards automating dynaTrace in Continuous Integration to identify more problems in an automated way during development.

Analyze Performance and Validate Architectural Rules by letting dynaTrace analyze Unit and Functional Tests in CI
Analyze Performance and Validate Architectural Rules by letting dynaTrace analyze Unit and Functional Tests in CI

Saving Time and Money

With dynaTrace’s in-depth visibility and the ability to automate many tasks both on the Client and Server-Side it was possible to

  • Reduce Test Time from 20 hours to 2 hours
  • Find more problems faster
  • Shorten Project Time

The fact that developers actually see what happened improves collaboration with the testers. It eliminates the constant back and forth. The deep visibility allows identifying problems that were difficult/impossible to see before. Especially Rendering, JavaScript and DOM analysis have been really helpful to optimize page load time.

Tips and Tricks

Here is a list of Tips and Tricks that Ron shared with the audience:

  • Clear Browser Cache when Testing Load Time
    • Different load behavior depending on Network Time
  • Multiple Page Testing
    • Simulates consumer behavior with regard to caching
  • Test in different browsers
    • IE 6, 7 & 8 have different behavior
    • Compare Firefox & IE – understand cross-browser behavior
    • Use weighted averages based on browser traffic
  • Test from different Locations
    • www.webpagetest.org to test sites from around the world (including dynaTrace Session Tracking)
      Commercial solutions for network latency, cloud testing, etc…

Best Practices

Ron concluded with the following Best Practices

  • Performance Matters
  • Define Performance Targets
  • Test client-side performance
    • Cross-browser testing
    • Add server-side testing
    • Tie everything together
  • Automate!
  • Get Test and Development on the same page
  • Get Proactive
  • Benchmark your Site against Competition


If you are interested in listening to the full webinar that also includes an extended Q&A Session at the end go ahead and listen to the recorded version.