I am working on this site dealsin.us and the Google webmasters tool shows that there are about 9800+ urls blocked by robots.txt. You can view the robots.txt here I have blocked some directories which are not meant for the users and our only to the staff behind the website. I am really confused and would appreciate any help on this.
migrated from wordpress.stackexchange.com Apr 25 '12 at 15:44
This question came from our site for WordPress developers and administrators.
Webmaster Tools used to show you all the URLs that you have blocked with
If your pages are appearing in search results without problems (a quick
However, looking at the robots.txt I do notice a few things.
Second, the 'sitemap' line should be separate from the rest, i.e. have a blank line after it. And the