6

Are there any search engines that do not care about robots.txt?

1
  • Any reason you want to know?
    – John Conde
    Commented May 21, 2011 at 16:27

1 Answer 1

5

All of the major, and minor, search engines respect robots.txt. Generally speaking only robots with bad intentions, or are poorly written, ignore robots.txt. You can find a list of bad bots here.

2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.