How to block web monitoring agents (aka bots) from Google Analytics

This is my second post on how to block web monitoring agent traffic from infiltrating your Google Analytics data. My last post about blocking IP addresses of agents is still valid, but I’ve recently come upon a much better way to make sure you block web monitoring traffic from the major web monitoring vendors in one method.

The solution involves using a tag management system. On www.dynatrace.com, we use Google Tag Manager. While these instructions are specific to Google Tag Manager, I’m sure you can do this with other tag management systems, like Tealium.

Step 0: Use Google Tag Manager

Step 1: Create a custom JavaScript macro

This macro will fire to detect the user agent string and then return a value. True=Agent/Bot; False=Not an Agent/Bot. You can name it anything you want, we called ours Agentbot.

Here’s the code:

function() {
  var useragent = navigator.userAgent.toLowerCase();
  var regex = /(gomezagent|apm synthetic agent|ruxitsynthetic|ktxn|khte|ktht|dynatrace|rigor|datadogsynthetics)/g;
  if (useragent.match(regex)) {
    return true;
  } else {
   return false;
 }
}

Step 2: Create a rule using the macro to return a value

Next, we want to grab the value returned by the macro and use it to create a firing rule.

The macro you created should appear in the drop-down menu. In this case, we want to know when a visitor is an Agent/Bot, so we tell the rule to be considered matched when the macro returns a true value.

Give your rule a name. We called ours, surprisingly, Agentbot.

Step 3: Use the rule to exclude the traffic from firing your Google Analytics tag

Now, all you need to do is add the rule as a Blocking Rule to any tag you want to NOT fire when an agent hits your site. Keep in mind that to truly measure your end-user experience, you want to have as many tags fire for agents as would for an actual site visitor. As always, it’s a tradeoff – do you want clean analytics or a detailed understanding of your customer experience?

One workaround would be to create another analytics tag that would ONLY fire for bot traffic. To do this you’d follow the same directions with the new analytics tag, but use the rule as a firing rule, not a blocking rule.

Step 4: Publish your tag manager container

Done! It’s just that easy. Now, most of the agent traffic will be gone from your analytics.

Tag managers are an extremely powerful tool, but they are a little more complicated than simply deploying analytics tags on your website.

If you have any thoughts or improvements, please comment away!