Sign up ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

Are there any search engines that do not care about robots.txt?

share|improve this question
Any reason you want to know? –  John Conde May 21 '11 at 16:27

1 Answer 1

up vote 3 down vote accepted

All of the major, and minor, search engines respect robots.txt. Generally speaking only robots with bad intentions, or are poorly written, ignore robots.txt. You can find a list of bad bots here.

share|improve this answer
Thanks! The list is pretty old though, no entries later than 2007.. –  Kim May 21 '11 at 16:44

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.