I just heard an interesting story from Runtastic, who is one of our Mobile Real User Monitoring customers. Runtastic’s main mobile app allows their users to track their sport activities such as running, biking, hiking or skiing. They had a user complaining about too high speed. He did not actually complain about the performance of the app but instead complained about his average speed during his workout as calculated by the app. After his one hour biking session, the app showed the he covered a distance of 490 km – that would probably be a world record and give him the yellow shirt in a small French bicycling event .

map

What went wrong on the mobile device?
After a short brainstorming session, the developers came up with the following three potential root causes for this bug:

  1. Bad GPS coordinates
  2. Problems with the calculation of the distance in the app code
  3. In correct values delivered to the app from the operating system

Looking at the route traced on the map (see picture above), the recorded GPS coordinates could not be the problem because it matched the route the user took. So, how did the Runtastic team go about finding out which of the remaining two possibilities was the real culprit?

The developers decided to build a unit test into the app where they could check the result of the device’s calculation for a well-known distance between two coordinates. Whenever the result was different than the expected, they used the capabilities of Dynatrace to report the error. This allowed the team to check whether the issue was a single user problem or not and which devices/operating systems were impacted.

After a weekend the team had enough visits showing an error like this:

visit with error

The first guess was that the device has an impact on the calculation. By grouping all the collected error events by device, the team got the following results:

errors grouped by device manufacturer

As the error events were evenly distributed across several manufacturers, a particular device could not be the reason. However, grouping them by operating system made the problem obvious.

errors grouped by operating system
It turns out that starting with Android 4.1, some of the devices have a bug within the calculation of the “distanceTo”-interface. For the Runtastic team, this unfortunately means that they cannot do anything about it as this interface is provided by the operating system. To limit the problem, they can give their users the advice to upgrade their operating system when they run into this problem.

Conclusion:
To improve the user’s experience for mobile native apps, you sometimes have to run the unit test in production. It is only there that you get the variety of devices and operating systems that allow you to accurately discover what the problem is. The Real User Monitoring solution leveraged must be capable of supporting such scenarios. Sometimes, being aware of a problem, even if you cannot fix it, can improve the user’s experience.

Further Links