I had got done a SEO Audit by Alexa for my website. It reported that all the crawlers are blocked from my website because of my robots.txt. My robots.txt looks something like
User-agent: * Disallow: / User-Agent: GoogleBot Allow: / User-Agent: Bingbot Allow: / User-Agent: Slurp Allow: / User-agent: ia_archiver Allow: / Sitemap : [Sitemap URL]
I have also checked these search engines and they do show indexed results from my website. Also before I had added permission for Alexa's bot
User-agent: ia_archiver Allow: /
without which Alexa was not able to do the audit. I am puzzled as to why Alexa is still reporting so, that all crawlers are blocked even when Alexa itself used that robots.txt permission to crawl my website.
I am using Allow operator as a way of having only selective bots be able to crawl my website. I would also like to highlight the fact that Alexa's own bot was not able to crawl my website for audit till I had added it to website's robots.txt using the same selective allow method.