Are there any search engines that do not care about robots.txt?
1
-
Any reason you want to know?– John Conde ♦May 21, 2011 at 16:27
Add a comment
|
1 Answer
All of the major, and minor, search engines respect robots.txt. Generally speaking only robots with bad intentions, or are poorly written, ignore robots.txt. You can find a list of bad bots here.
answered May 21, 2011 at 16:26
-
1
-
There are others that seem to be more current you can check out.– John Conde ♦May 21, 2011 at 16:47