…This was one of the questions that we discussed on stage at Mobile Apps World in San Francisco last week. It reminded me that I never wrote a blog about this topic. These are my key findings:

– Looking only at the number of crashes as key performance metrics is a thing of the past
– Users expect  the same or faster performance from a mobile app compared to the desktop web
– Don’t let a third party provider or your backend services ruin your mobile performance

Let’s go into the detail and find out what this exactly means.

Looking at the Number of Crashes as the Key Metric Alone is Over

In the early days of app development the number one quality criterion was the number of crashes compared to the number of visits. This metric is still valid, but mobile app development has changed in the last 3 years. Nowadays the apps are developed in a more defensive way and global exception handling prevents lots of apps from crashing, but brings up another issue: Whenever something is wrong the user gets a message telling him more or less vague what the problem is, but he can still not do what he wanted. As an additional Key Performance Indicator (KPI) the number of errors has to be monitored. In the dashboard below you can see how such a minimum monitoring dashboard can look like.

Traffic Crashes and Exception Comparison
First Step Monitoring: Traffic Crashes and Exception Comparison

This high-level overview is good but when trying to fix the problem, the developer will ask you for more details. So in addition to this high-level information, Mobile APM has to provide very low-level data like crashes reports, exception stack traces, jailbroken attributes, memory or battery level.

Detailed error and crash report data for development analysis
Detailed error and crash report data for development analysis

The User Expects from Mobile Apps to Have the Same or Faster Performance than a Desktop Web Site

Web Performance Optimization (WPO) was and still is a trending topic for web development. The expectations of users regarding the performance of the desktop web are very high. What does this have to do with mobile?  Isn’t it something completely different? No, it’s not! From a user’s perspective, the mobile app is nothing more than another touch point with the company-like the website or the physical store. The user wants a similar experience no matter which touch point he uses. Therefore the bar for mobile app performance is already set by your colleagues developing the website. Additionally, the advancement of WPO over that last 10 years has the expectations high.

So you need a possibility to compare your mobile and web presence. The 4 KPI’s to look at are:

– Visit/Action Volume – You want to know how much traffic you have on each channel
– APDEX – As the unified overview metric that makes comparison easy
– Response Time/Duration of particular Actions – Measure the performance of your key actions
– Failure rate – You want to know the percentage of users impacted by crashes, errors…

The below screen shot shows how such a comparison looks like at a customer I work together with. For this eCommerce store, the “Search” is one of the key actions and they wanted to compare their US/European- against their Asian offering.

User Action Performance Mobile vs Web Comparison Example
User Action Performance Mobile vs Web Comparison Example

The user action is the object that allows you to compare mobile native with web. A user executes e.g. a search and he can do this with web and mobile so the expectations regarding the result are the same. The user action can be different, though. On the web, the click on the button can trigger 50+ web requests and the experience depends on the performance of all of them. On the other hand, we have the touch in the mobile app which might just trigger 2 requests. For the user, it does not matter how many request are executed at the end he wants the search results as fast as possible. This is a typical example of an action and why you need to compare on this level and not on per Web request.

Don’t let a Third Party Provider or your Backend Services Ruin your Mobile Users Experience

The integration of Third Party Content (TPC) on mobile is common and the issues and impacts are already well-known from the web. There is hardly an app out there that does not include a social media integration, ad plugin, payment service, et cetera. If everything’s fine with these services they provide a lot of value, but in case they are facing outages or slow performance they directly impact your user’s experience. Therefore, monitoring of TPC is absolutely necessary as part of a mobile application performance management  strategy.

In the transaction flow below, we can see another example how to ruin the performance of a mobile app – the mobile app backend services! This is an example of an app that needed more than 30 seconds to start and the reason for that could not be found on the mobile side. 99% of the user action duration were spent on the backend tiers. The issue could be solved together with the backend team as the end to end visibility allowed them to work together and identify the performance hotspots.

Slow Backend and Thrid Party Provider exposed
Slow Backend and Thrid Party Provider exposed

Hybrid Apps – When Mobile Meets Web

I was talking about web and mobile as two separate approaches, but a lot of native apps embed web views or use a framework like PhoneGap or Cordova, which rely on HTML5, JavaScript and CSS. Consequently, a JavaScript error in a hybrid app can be as bad as a crash in a purely native app. Your mobile app monitoring needs to be able to capture and report those errors as well so that you have the data the developer needs to fix the issue.

JavaScript Error in a Mobile Webview
JavaScript Error in a Mobile Webview

Conclusion

Mobile APM  must allow you to improve your mobile users experience. In order to do so, it has to deliver the data needed to reduce the number of errors in the app and provide an end-to-end view comprising mobile code, third party content providers as well as backend services to enable you to identify performance hotspots. Because of the required end-to-end view it is clear that mobile APM can be the starting point of your global APM journey or complement your existing APM strategy.