Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

On the robots.txt fetch tab there is a yellow triangle when hovered over. I am getting the following message. "Crawl postponed because robots.txt was inaccessible" However I am showing 0/151 Errors/attempts.

I use the same robots.txt on other sites and there is no problem.

www.dtnmgt.com/robots.txt
share|improve this question
    
When did Google WMT last fetch your robots.txt and when was it last modified. Google may take 24-48 hours or so to update it's last fetch of your robots.txt. –  zigojacko Feb 20 at 16:50
    
The errors popped up on the 13th but we have had 3 successful fetches since then. The most recent on the 20th –  Ryan Feb 20 at 16:59
    
I don't think it's an issue, your robots.txt is accessible fine - I think it's just a warning and that there was probably a disruption to the last crawl for whatever reason. –  zigojacko Feb 20 at 17:19
    
Try fetching your robots.txt as Googlebot within Google WMT as well just to make sure it is able to access it without issues. –  zigojacko Feb 20 at 17:22
    
Yep. Everything works fine. I will let it sit for a couple days –  Ryan Feb 20 at 21:26

1 Answer 1

Checked out the robots.txt and there were no issues. Your robots.txt is accessible fine - I think it's just a warning and that there was probably a disruption to the last crawl for whatever reason.

Try fetching your robots.txt as Googlebot within Google WMT as well just to make sure it is able to access it without issues.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.