I recently began research on Search Engine Optimization to further understand what we can learn from Performance Monitoring solutions. As an IT nerd, I started the investigation on the technical side of the house, speaking with operators and CTO’s about robots. Interestingly, all of them expressed the same concern: “They are an issue! We run more than half of our data center for them!”
At first I found this to be a bit difficult to believe, but then they shared real statistics to support their concerns.
BVA Auctions Real vs Search Engine Traffic
PrepSportswear Real vs Search Engine Traffic
Have you ever checked the amount of robot traffic on your website? After I heard about it I checked out our own blog and, although the blog content doesn’t change that often, there is quite a bit of bot traffic — around 17%.
Are you experiencing the same surge in your data centers? If you want to share your “Search Engine vs Real User Traffic” chart feel free to reach out to me via Twitter, LinkedIn, and I will add it to this blog. To generate this data/chart sign up for Dynatrace Personal Free.
In my next blog I will illustrate how BVA Auctions addressed this challenge and reduced the traffic by 75% without harming its search rank. Also, here’s my Velocity Conference presentation about the topic on youtube.com and the corresponding slide deck.