“I’m looking to drop some serious cash on cookware, but I can’t add items to my cart” – this is a comment posted by a frustrated visitor of one of the Bon-Ton Online Stores. The question is what you do with that type of comment when you are responsible for End User Experience? Is it a problem with a single user who simply likes to complain in public – or – is it really a bigger problem impacting many users who won’t be able to do their online shopping? If it is the latter, it impacts business and reputation and it should get all the possible attention. In the case discussed in this blog it turned out to be a real problem potentially impacting all Safari online shoppers.
In a recent webinar, Bon-Ton explained how they deal with the fact that they simply can’t test every single use case on all browser versions and devices of their web application and therefore end up with problems that impact certain end users. They came up with a really interesting approach to look into the problems that their users report allowing them quickly identify (within minutes instead of hours) whether this is a problem to worry about or whether it is safe to ignore.
In this blog I will walk you through their 4-step approach which for instance allowed them to find the Safari-related cookie problem that prevented online shoppers from putting products in the shopping cart. It turned out that Safari has a bug when dealing with special characters in cookies that they used for some of the products in their product catalog. As problems like this only happen on specific browsers and specific use cases, it is even more important to listen to the “community feedback” because you simply can’t test all of these scenarios.
Step 1: Listening to User Feedback / Complaints
In order to get feedback from users you need to provide an option for users to do so. They provide a feedback form with email address, name and comment. They use Compuware APM and the Dynatrace Business Transactions feature to extract email and comment for each posting that is done so that new feedback will show up in their dashboards showing which users have posted which comment. The email is especially interesting because this also allows them to identify and track all Visits of that particular user which is important for Step 2.
Step 2: Identify the Click Actions of the User
Bon-Ton uses Compuware Dynatrace User Experience Management (UEM) to track the user experience of all their visitors on all of their online stores. They have set it up to capture the users email address when they logon to their site. This allows them to easily identify all Visits of a particular customer. In this case they needed to look up the Visits of the user that provided the feedback about not being able to add items to the cart. The following shows the list of actions this user did prior to clicking on the Contact Us link to provide the valuable feedback. The problem seems to be related to an HTTP 404 response when adding the item to the cart:
Step 3: Analyze the Problem
A deeper drill down into the request that responded with a 404 reveals the real reason for this error. An invalid cookie that was sent by the browser caused an exception in the IBM WebSphere eCommerce stack:
The actually cookies sent for each individual request were automatically captured by Dynatrace UEM allowing them to see the actual cookie value that caused this exception. It turns out that the special character they had in the product name “(r)” was not correctly escaped causing this exception on the application server. This means that this problem is likely going to also affect other customers who try to add products with this special character in the name.
Step 4: Identify the Impact of this problem
First, Bon-Ton analyzed how many of these “Invalid Cookie” exceptions actually happened throughout the day. Bon-Ton relied on Dynatrace UEM’s ability that captures all visits and any exceptions for every visitor, 24×7. It turned out there were many of these exceptions. The exceptions led them to the actual end-user actions that caused these exceptions, which showed them that this problem ONLY happens on Safari browsers- both Desktop and Mobile version:
A little research on Google turned out that this was a well-known issue with Safari that they could work around enabling their customers using Safari to buy products with names having special characters.
Conclusion: Listen To Your Customers and fix Problems That Weren’t Found In Testing
Just like other web companies, Bon-Ton used the feedback from customers to improve their web site and also to dig into problems that their visitors report. Having the right tools to quickly verify the problems reported allows Bon-Ton rapidly analyze the root cause of a problem and correctly prioritize fixes based on the impact to customers and the business. If you want to hear the full story feel free to listen into the recorded webinar.