Webmasters Stack Exchange is a question and answer site for pro webmasters. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

When I initially setup my blog I put the restrictions on robots.txt way too high. They were so stick that googlebot can't even access the udated less strict robots file. I've been at this problem for days, I'm in a bind. I will be extremely grateful if someone could help me out.

share|improve this question

migrated from wordpress.stackexchange.com Sep 8 '12 at 12:06

This question came from our site for WordPress developers and administrators.

How many days have been passed? – Aurelio De Rosa Sep 8 '12 at 20:47

I'm not sure I 100% understand your question (you should add a snippet of your robots.txt rules to get a definitive answer)

Sill, from what I read here you must have a "Disallow:\" rule inside which you'll need to remove. I`m saying this because you robot.txt is always should be in root directory and this is the only rule that can prevent access to that (not including wrong syn-text of course)

Also, before doing anything, go into Google Webmaster Tools and use "Fetch as Googlebot" (under "Health") to find out if indeed your robots.txt is as inaccessible, as you think it is.


share|improve this answer
I'm wondering if OP set a massive crawl-delay? – DisgruntledGoat Nov 25 '12 at 17:17

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.