We recently participated in the STAR EAST conference in Florida and conducted a survey of the software test professionals in attendance to find out how they test web applications and what challenges they face.
The results were interesting – we learned that many testers are using tools that are not able to meet their testing challenges. Key highlights from the survey include:
- Most testers surveyed use QA performance tools behind the firewall to test web applications
From the group surveyed, only 18 percent go outside the firewall to test on a staging environment, and only 9 percent test on a production system outside the firewall.
This means that most testers are not testing the full application delivery chain, but only their own infrastructure. Problems caused by components outside the firewall, such as CDN’s, payment or shopping cart systems, or advertising software cannot be detected. Furthermore, there is no way to see what problems a customer may experience based on geography or device use.
- According to the survey participants, the primary goals for testing web applications are to ensure that the applications scale under load; deliver quality web experiences to all users; ensure that the site can handle peak traffic; and rapid resolution of performance problems prior to launch.
Earlier this year, we commissioned an independent study to examine the performance of websites experiencing peak traffic volumes and the impact on consumer behavior and business results. The study asked respondents during the peak time of the current holiday shopping season, if they had a poor experience on a retail website (the site was very slow, encountered web errors, online transaction failed) – more than 61 percent had a poor online experience.
If testers do not test outside the firewall, there is no way to meet the primary goals of delivering quality web experiences to all users. You may be able to ensure that your infrastructure can stand up to traffic, but you will have no method to detect performance problems prior to launch.
- The biggest load testing challenges identified were: rapidly identifying root cause of problems and generating load sufficient to fully stress the site.
It’s no surprise that these are the biggest challenges because it is very difficult to generate the amount of traffic needed to stress the system from inside the firewall. It’s also impossible to identify the root cause of problems unless you look at the transactions coming from the outside in and can connect the transactions to the problems.
Web applications face the customer, the world outside the firewall and outside the datacenter. If the goal of the business is to provide quality web experiences to all users, then you have to test the customer experience.
You have to use tools that are outside the firewall that will detect problems all along the application delivery chain, across geographies, devices, all the way back to your application. Once the problems are detected, the tools need to pinpoint the problem and facilitate a fix with engineering and QA using the same set of tools to look at a problem.
For example, our Gomez 360o Web Load Testing Solution allows you to pinpoint scalability and performance problems across the entire application delivery chain — down to the specific line of code and provides a lifecycle platform for collaboration between development, testing and operations which reduces friction and test iterations
If you find yourself in a similar situation as our survey participants, you may want to take a few minutes and learn more about our solution, or take our free Web Load Test to see what happens when your website traffic increases.