”Our customers expect the same user experience throughout all our channels no matter if it is the website, mobile site or the real physical store!”
So, how can you measure the user experience across the various channels your customers use to interact with you? Let’s see what we can do to make the user experience quantifiable so we can make informed data-driven decisions.
If we look at the different channel types, they have all two things in common – a user and actions that the user executes. To make this more visual I put the graphic below together. It depicts a common Visit of a user with what their common User Actions might look like across each of the channels (Mobile App, Mobile Web & Desktop Web). Although the screens look different the expectations are always the same.
Why you Should Look at User Actions?
User actions are the important element as it exists on the web as well as on the mobile app. It is therefore something we can compare between the different channels. In the above graphic the second user action is “Search for… ” The user expects that the time it takes from the touch/click on the search button till the result list is displayed is similar. This timing is now our first indicator for the User Experience; the User Action Duration. We can now compare the user action duration over all the different channels and can work on getting the experience to the industry best practice “3 second promise”.
Looking at the user action duration and the number of errors, we now have two good metrics to quantify the user experience for each user action. But we still have more to investigate to get the overall user experience.
What Additional Information do we get by Looking at a Visit?
The collection of experiences across all user actions is what ultimately defines whether the user experience was satisfied, tolerated or frustrated. If we look at a “visit” as a collection of all user actions in the order the user executed them, we can get four very important data points: the first action, the last action, did the visit bounce and if the visit converted. These all help us to identify the User Experience Index. The screenshot below shows us an example of a single visit with every User Action completed. The user started the app, searched for a vacation, tried to book the trip and then the app crashed.
I have seen numerous customer examples like the one above where visitors had perfect experiences until they hit the final steps. Typically, if the last action fails or takes too long for the user, it will cause the user to bounce off to your competition. Therefore we have to consider a user that had bad performance or was facing an error on the last action, as frustrated.
These metrics are also very valuable when it comes to A/B or multivariate testing. Sometimes changing a picture or color does not help because errors and bad performance are already destroying the user experience. If you want to know more about it I wrote a blog focusing on why you’re A/B testing efforts might not have any chance to succeed.
How to Collect the Data
The collection mechanisms for the different technologies might vary, but there are Real User Monitoring solutions available that allow you to collect the real user monitoring data from:
- Mobile Native Apps
- Mobile Hybrid Apps
- Desktop Web Browsers
- Mobile Web Browsers
Do what you can to present the data gathered in a way that presents the most value to you. Feel free to use the image below as an example, when attempting to correlate the metrics we talked about here to the overall user experience. If you want to try it out on your own then checkout our Dynatrace free trial.
Delivering the same user experience across all the channels (Mobile App/Mobile Web/Web) is critical to the success of your business. By looking at the satisfaction of a visit, you’ll get a better understanding of what the user’s real experience was. Don’t just look at the performance and faults that occur on a single action but also at the critical entry- and exit actions. Considering these metrics, you can easily compare the overall experience of your users using one or multiple channels.