Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

My site's URL is: www.hackinguniverse.org

From some day in Google Webmaster tool a message is showing that says "Some important page is blocked by robots.txt".

My robots.txt is:

User-agent: Mediapartners-Google
Disallow:

User-agent: *
Disallow: /search
Allow: /

Sitemap: http://www.hackinguniverse.org/feeds/posts/default?orderby=updated

Now for other information - I hosted this website on blogger. I have some other sitemaps too where I had only included "/Search" in Disallow like in this robots.txt file. But those sites are ok. I mean no message is showing on those. So why am I getting that message telling that I have blocked some important page via robot.txt?

share|improve this question
3  
"Allow" directives are generally redundant, unless they are replacing previous directives. They don't help and they can make the code confusing. The purpose of robots.txt is to exclude url's. By design everything that is not excluded can be crawled. –  Osvaldo Jan 20 '12 at 23:19
    
@Osvaldo Yah... But can I change robot.txt for Google Bloger.. Because i am not able to change it.. –  Rounak Jan 21 '12 at 4:08
add comment

1 Answer

Check the download date on the robots file, it could be referring to an old version of the file.

share|improve this answer
    
Oh... I dont know but Google Automaticly fixes the problem !! –  Rounak Jan 21 '12 at 20:34
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.