Are you running an eCommerce site that relies on 3rd Party Services such as Address-Validation, Credit Card Authorization or Mapping Services? Do you know how fast, reliable and accurate these service calls (free or charged) are for your web site? Do you know whether it has an impact on your end users in case one of these services is not available or returns wrong data?

End User and Business Impact of 3rd Party Service Calls

In last week’s webinar Daniel Schrammel, IT-System Manager at Leder und Schuh (responsible for sites such as www.shoemanic.com or www.jelloshoecompany.com) told his story on the impact of 3rd party online services to their business. One specific problem they had was with a service that validates shipping address information. In case the entered shipping address is valid, users can opt for a “cash on delivery” option which is highly popular in the markets they sell to. If the address can’t be validated or the service is unreachable, this convenient way of payment is not available and users have to go with credit card payment. As the eCommerce platform used to run their online stores also comes from a 3rd party provider the Leder und Schuh IT team has no visibility into these 3rd party online service calls, whether they succeed and how that impacts end user behavior.

Monthly Report basically means No Visibility

As stated before, Leder & Schuh uses an eCommerce Solution that was not developed in-house. Therefore they had no option to monitor the service calls from within the application as this was not supported by the platform (no visibility into application code). They had to rely on a monthly report generated by the Address Validation Service telling them how many requests they had last month, how many succeeded, partially succeeded (e.g: street number incorrect) or completely failed. With that aggregated data it was

  • Impossible to tell which queries actually caused the verification to fail (was it really the user entering a wrong address or is the service not using an up-to-date address database?)
  • Hard to tell whether a failing address validation has an impact on users decision to actually buy shoes (is there a correlation between address validation and bounce rates?)

Live-Monitoring of Service Quality

In order to solve this problem they had to get visibility into the eCommerce Solution to monitor the calls to the 3rd party address validation service. They were interested in

  1. the call count (to validate the service fee they had to pay)
  2. the response code of the service (to see the impact the response had on users bouncing off the site)
  3. the actual input parameters that caused the service to return an address validation error (to verify whether addresses were really bogus or would have been valid)

Using dynaTrace allowed them to accomplish these and other general Application Performance Management goals without needing to modify the 3rd Party eCommerce Platform and without any help of the 3rd party address validation service. The following dashboard shows the calls to the address validation service. On the top line we see green which represents the calls that return with a success, orange which represents validations with partial success and red that indicates those calls that failed. The bottom left chart shows an aggregation of these 3 return states showing spikes where up to 30% of the validation calls don’t return a success:

Monitoring 3rd Party Service Calls, the response code and impact on End Users
Monitoring 3rd Party Service Calls, the response code and impact on End Users

Monitoring the service like this allows Leder & Schuh to

  • Get Live Data on Service Invocations -> don’t have to wait until the end of the month
  • Can look at those addresses that failed -> to verify if the data was really invalid of whether the validation service uses an out-of-data database
  • Can verify the number of calls made to the service -> verify if they don’t get charged for more calls matches what they get charged
  • Can monitor availability of the service -> in case the service is not reachable this breaches the SLA

Impact of Service Quality to User Experience and Business

As indicated in the beginning, the option “cash on delivery” is much more popular than paying by credit card. In case the address validation service returns that the address is invalid or in case the service is down (not reachable) the user only gets the option to pay with credit card. Correlating the status and the response time of the service call with the actual orders that come in allows Leder & Schuh to see the actual Business Impact. It turns out that more users bounce off the site in case the only payment option they are given is paying per credit card (Functional Impact) or in case the validation service takes too long to respond (Performance Impact). The following dashboard shows how business can be impacted by the quality of service calls:

Quality of Service Calls (Performance or Functional) have a direct impact to Orders and Revenue
Quality of Service Calls (Performance or Functional) has a direct impact to Orders and Revenue

Want to learn?

During the webinar we also talked about general Response Time, Service Level and System Monitoring they are now doing on their eCommerce Platform. With the visibility they got they achieved some significant application performance improvements and boosted overall business. Here are some of the numbers he presented:

  • 50% less database queries
  • 30% faster Landing Pages
  • 100% visibility into all transactions and 3rd party calls

You can watch the recorded webinar on the dynaTrace recorded webinar site.