In part one, I discussed the importance of understanding the benefits ramping load can have on web load test results. In this post, I’ll discuss best practices for determining how long to ramp at the beginning of a web load test.
The general the rule of thumb for load test ramping is the slower the ramp the better. Of course, given time and cost restraints, the goal of most testers is to run the most effective test possible in the least amount of time. With this in mind, here are some best practices for determining how long to ramp at the beginning of a load test:
Determine the goal of the test – If the purpose of the test is to determine at what point the application’s performance bottleneck is, then the ramp should be slower and more gradual. If the purpose is to measure performance at a specific pre-defined load level, then a shorter ramp and a longer hold period will suffice.
All tests should have a minimum 20 minute ramp period – To ensure that load is consistently and evenly applied during the test, a minimum of 20 minutes of ramping is recommended for each test.
The longer it takes to complete a transaction, the longer the ramp period – Determine the total time it takes to complete a transaction (make sure to include any think time). If the average transaction takes over 2 minutes to complete, add an additional 5 minutes to the ramp for each additional minute. For example, if the average transaction takes 4 minutes to complete, add 10 minutes to your ramp.
When Ramping Is Not Necessary
An example of a test where a slow gradual ramp of load is not necessarily useful or desired is in a flash test. The purpose of a flash test is to mimic scenarios where a high number of visitors visit a website in a very short period of time.
This is common in situations that involve sales promotions or coupons or special events that are time sensitive. In this scenario it’s required to apply as much traffic and load to the application as quickly as possible. The quickest way to accomplish this is to have a very short ramp or to remove the ramp all together.
A common practice used during many load tests is to ramp down at the end of the test. The purpose of this is to slowly lower load and traffic levels on a system to allow the system to gracefully recover from higher load levels.
When using web load, the benefits of ramping down are minimal and the tests themselves will naturally ramp down as virtual users complete their last transaction at different times before shutting down. Removing the ramp down phase reduces the amount of time and cost of each test.
However, this type of ramp is not recommended for any throughput style tests or any tests where the goals are to determine when and where performance bottlenecks or break points occur on the system.
Adding steps to a ramp increases the total test time and runs counter to the purpose of throughput tests which are meant to be short and simple. Also, having a stepped ramp usually means that the throughput results will follow a similar stepped pattern making it harder to compare response time results with throughput results and interpret where exactly performance bottlenecks and issues have occurred.
If you choose to use a stepped ramp approach, it’s necessary to allow for sufficient time for load to level off during a step’s hold periods. If the steps are too close together, the ramp and hold periods will not be well defined in the results and will look almost as if a stepped approach was not used at all. This makes it more difficult to interpret the results and defeats the purpose of a stepped ramp.
A minimum of 10 minutes per hold period should be used in a stepped ramp approach. It is also important to consider how long the transactions take to complete and make sure that the hold periods allow enough time for a transaction to complete multiple iterations during the hold period.