Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

I noticed tons of bandwidth being used by non-USA search engine bots, so I began blocking them in an effort to save bandwidth and cpu cycles for actual users and the search engines they come from (Google, Bing, Yahoo, Ask, etc.).

Other than potentially losing some international traffic (which isn't really important to us since all of our content is very USA-centric), what additional dangers should I be concerned about?

I'm using a modified version of Jeff Starr's User Agent Blocklist

share|improve this question
add comment

1 Answer

Love it, Dangers...

Anyway, if you depend on overseas sales from international customers, you eliminate their ability to find your site through their preferred search. Eliminating Yandex, for example will mean that Russian Federation and Eastern European countries will be finding results elsewhere as you are now invisible to any searches through that SE.

If you aren't doing overseas sales to the countries that search engine represents, you're saving (as you've noted) your server a lot of wear and tear for no good reason at all. The only harm I noticed when bringing a new site on line and blocking certain foreign search engines from the reindex flurry due to their ability to overload the website was reduced access from the language pools/countries that they represented. Once Google/Bing/Yahoo got through battering the site and we'd improved its performance to handle the load, we added Yandex, et. al. as we do sales in these countries. Yandex in particular was very quick on getting the site back in and fully indexed.

So in short, no SEO danger was noted, except what was expected, loss of visibility on that search engine.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.