Is it possible to show different server to search engine bots and different server to users in order to improve page speed of the site, specially for a large E commerce website. Idea is to show same content, everything same to both users and bots, so its not cloaking but it will help in the improvement of your site speed. So 2 questions -

  1. Is this possible from development point of view.
  2. What will be the effects of doing this.
  • Spiders are unlikely to cause major speed issues on your site. If a bot like Googlebot is actually causing performance issues, you can use Search Console to limit its crawl rate. You can most likely improve performance much more significantly by optimizing your CSS, JS, and database queries, and experimenting with caching. If you're still having performance issues it might be best to look into load balancing - where all traffic (not just spiders vs. humans) is routed through a network and may hit one of several different servers to improve load times. – WebElaine Jul 6 '18 at 15:06
  • This would probably be flagged as cloaking – jrtapsell Jul 6 '18 at 15:59
  • If your site is slow for Google it'll be slow for users which potentially is a worse outcome because conversion is reduced. If you want an ultra-fast server for your customers and for Google, then have Google host your website. Having administrated hundreds of sites on Rackspace, AWS and Azure, I've found Google is by far the quickest, they simply have the best-built networks. tools.pingdom.com/#!/8Osg9/https://www.bybe.net with hardly any optimisation. – Simon Hayter Jul 6 '18 at 18:37

Showing bots better performance won't help SEO. If that is what you hope to gain from it, it won't work.

Google's algorithms for slow sites only punish really slow sites. Ones that take 7+ seconds to serve the initial page request. As long as your site is even decently fast, you won't have any problems with Google's site speed algorithms that punish slow sites as measured by Googlebot.

Improving your site speed such that everything above the fold is usable in under 3 seconds can improve your SEO, but you need to make that performance available to both users and bots. For SEO in this realm Google notices users positive reactions to better performing sites rather than measuring the speed directly itself.

It is possible to put load balancer rules in place to serve the Googlebot user agent from its own set of faster servers. I've tried it. It does nothing to improve SEO. Plus it is dangerous. If that set of servers goes down, it is hard to notice because you won't see any problems with your site. However, Googlebot will start getting errors and start kicking your pages out of the index.

| improve this answer | |
  • Thank you all for the answers, I was looking to understand if this helps SEO or not. Thanks! – NRose Jul 7 '18 at 22:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.