Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.
I use robots.txt to block all robots.txt compliant bots from a directory like this:
How can I override this so to allow Mediapartners-Google (AdSense) to /sub/?
Adding this at the top of the robots.txt works:
N.B. you may need AdSense to download your robots.txt again, which I believe they do on a 24hr basis.
Required, but never shown