1

When I initially setup my blog I put the restrictions on robots.txt way too high. They were so stick that googlebot can't even access the udated less strict robots file. I've been at this problem for days, I'm in a bind. I will be extremely grateful if someone could help me out.

1
  • How many days have been passed? Commented Sep 8, 2012 at 20:47

1 Answer 1

1

I'm not sure I 100% understand your question (you should add a snippet of your robots.txt rules to get a definitive answer)

Sill, from what I read here you must have a "Disallow:\" rule inside which you'll need to remove. I`m saying this because you robot.txt is always should be in root directory and this is the only rule that can prevent access to that (not including wrong syn-text of course)

Also, before doing anything, go into Google Webmaster Tools and use "Fetch as Googlebot" (under "Health") to find out if indeed your robots.txt is as inaccessible, as you think it is.

GL

1
  • I'm wondering if OP set a massive crawl-delay? Commented Nov 25, 2012 at 17:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.