Even Google will occasionally include a site in its search index, despite the fact that the site disallows all crawling with robots.txt. Google views robots.txt rules as preventing crawling, but not necessarily inclusion in the search index.
When Googlebot can't crawl a site, but the site has lots of external links, Google may include pages from the site in the index and use the anchor text of the links to determine what the page is about.
Google's Matt Cutts blogged about this. He gave an example of the Department of Motor Vehicles in California. Its website disallowed crawling, but it would be ridiculous if people couldn't find it on Google. So they came up with a way of including sites like that without being able to crawl them.