Update: February 16th, 2016 5:30pm: Valentine’s Day Weekend Wrap Up
Some people will say that Valentines Day is for lovers, while that is true for some, Valentines Day is also for specialty retailers and services. As you saw leading into Valentines Day, the team here at Dynatrace had been watching a large selection of specialty retailers. Over the weekend these retailers all looked to be providing good availability to end users. But in this day and age it is not enough to just have the lights on. The end user experience has to be exceptional or retailers will lose potential customer engagements.
For example here we see the Online Florist Category. Below we are ranking by complete page download. This is from the moment someone makes a request for the page until the page has completely loaded. Some site owners feel that this metric is too restrictive and point to other metrics like DOM interactive or DOM Complete. These are internal browser metrics which denote when a user could potentially start interacting with a page or when the browser considers the structure of the page to be complete even if something is loading.
Using these DOM metrics is fine (it’s better to measure something than nothing at all), however many of these lower level metrics can vary greatly from one browser version to the next. If you are going to manage based on these lower level metrics, site owners need to consider a User Experience Management tool (Dynatrace UEM shown below) to understand all of the possible end user permutations that might be seen.
Over the weekend we watched some of the specialty services which we thought would see an uptick in traffic for Valentines Day. For example, below we are comparing Lyft to Uber. Lyft was responding faster than Uber however we noticed some small success issues over the past 5 days.
Our testing displays Availability, however it is best to think of this as success. How successful were we at making a request. Making sure the lights are on is no easy task given the complexity of these applications. The view below is a sample illustration of how a request transaction is responded to from the server side to the end user.
Below is another example (for the past 2 weeks) looking at server side request time for Online Confectionary (Candies and Chocolates) vendors. Here we see that Harryanddavid.com had the fastest server side response times, almost three times faster than the slowest company.
Specialty Retailers need to make sure their sites are not only running fast from an end user perspective but are also optimized on the server side. Optimization on the server side allows code to run more efficiently in terms of hardware resources, servers to respond more quickly and servers to handle additional incoming load.
More often than not, site owners are not aware of all of applications, services, processes and hosts that are running on the server side. Tools like the Dynatrace SmartScape below show how modern Application Performance Monitoring tools can be used to auto discover what is running on the server side, auto baseline performance and auto detect problems and their root causes.
Specialty Retailers had a relatively uneventful Valentines Day this year. To continue this next year, site owners should follow these three steps…
1/. Have a plan. Everyone needs a playbook. Retailers need to make sure they have plan in place when things go bad. The plan should not just be a reactive disaster recovery plan, that alone would be a bad plan. The Retailer’s plan has to look beyond disaster recovery and make sure that their sites and applications have been thoroughly tested well before any holiday event. The plans have to include ALL parts of the organization, it should not just be limited to operations, it has to include digital business owners, development, test and operations.
2/. Have the right tools. “Everyone has a plan until you get punched in the nose”. Having a site go down during a holiday event is a traumatic experience for a retailer. It’s not enough to have a plan, but you also have to have the right tools in place. When things go bad for a retailer we typically see them go into a “war room” scenario. Every team has their own set of tools which typically tell them that what they are responsible for is green, but still the problem persists. Retailers need tools that bring teams together, they need a common tool that everyone in the organization can use including operations, development, test and digital business owners.
3/. Understand the data. Having the right tools in place is a step in the right direction, having the expertise to understand what the data is telling you is another thing. Most retailers have teams of individuals who all understand their part of the application, and have a few “rock stars” who have the big picture in how it all works together. Modern retail applications are incredibly complex, with applications having hundreds if not thousands of dependencies. Sifting through log files (which is what we see most often) is the old (and most costly) way of solving a problem. Retailers need to have an understanding of the relationship between application components and external internet factors. While most retailers have a few rock stars what they need is a next generation set of tools to automatically notify that event is occurring, analyze all the dependencies and discover the root cause of the issue.
Retailers can rely on these three things to see them through when things go bad. The better they are at each of these the less likely an event will occur (an ounce of prevention is worth a pound of cure), the fewer war rooms they will have to convene, and faster they will be able to recover from an issue.
Update: February 10th, 2016 8:00am: Valentine’s Day Niche Retail Report
Time for some performance hugs and kisses from Dynatrace, just in time for Valentine’s Day. Valentine’s Day represents a niche retail event, not as big as the BlackFriday/CyberMonday, but still very important to select retailers and niche services.
Dynatrace did a study with a major retailer and found that when customer sessions started to have performance issues that online conversion would drop.
For these specialty and niche retailers, Valentine’s Day is a highly competitive time of the year. Consumers will not tolerate poor performance and will happily abandon sessions which are slow to look for similar products from competitors.
Performance matters even when it comes to chocolates and roses.
This year the team at Dynatrace is looking at a variety of niche categories for Valentine’s Day.
- Beauty and Wellness
- Dinner Reservations
- Greeting Cards
- On-Demand Ride Services
- Specialty Services
- Travel / Hotel Reservations
Let’s look at the past two weeks performance and availability for the companies we are tracking in each category.
Beauty and Wellness
Here we see Massage Envy leading the way at 3.6 seconds.
Russel Stover has the fastest Chocolate/Confectionary web site at 2.8 seconds.
Yelp is the fastest intros category at 3.8 seconds.
Looks like staying in with Netflix is leading the Entertainment category.
Aptly named Flower.com is the fastest in this category.
American Greetings is leading the field for cards.
Tiffany is providing the fastest response time at 3.5 seconds in this category.
OnDemand Ride Services
While Lyft is providing the fastest response time here we noticed some issues which impacted the success rate of our tests for the past week.
Victoria Secrets is leading the Specialty gifts.
Travel / Hotel Reservations
Hoteltonight.com is leading this category.
Keep checking back here, we will be following the performance of these companies leading into Valentine’s Day. Our Holiday Survey this year puts this into perspective….
81% of Millennials will abandon buggy sites/apps and shop elsewhere
51% of Millennials will complain on social media
49% of Millennials abandon sites/apps if they fails to load in 3 seconds
Also make sure to have a look at our Holiday Round Up as there is more detail as to why performance is so important for retailers.