Google Webmaster Tools crawl errors reported after using Geo IP delivery

Question

I am using Magento for my site.

Recently I blocked my website for the U.S. using the PHP Geo IP Location country database.

This means that when people visit from the U.S., the site will show just a restricted access message.

But the next day when I logged into my Webmaster Tools account, I see errors for 300 pages.

It shows 500 errors for the links, however the pages are working for me.

Here is the error reported by Google:

http://example.com/: Increase in server errors May 4, 2014 Google detected a significant increase in the number of URLs we could not access. It is likely that your server had an internal error or was busy when attempting to process these requests.

Answer 1

Sounds like you have not taken into consideration that Google crawls sites all across the globe but primary these crawls do occur more often in .US data centers. You could override this by using a user agent detect script that allows the Googlebot regardless of its GEO location, however you should also take into note that Google doesn't view this solution as ideal and could harm your rankings, Google wants the same treatment as every other user.

More practical solutions widely seen on the net:

1. Serve a notice that you don't deliver products or services based on GEO Location.
2. Block content, not the pages... i.e you block the inner container but again this is risky as Google won't be able to see it.