Tagged Questions

4
votes
1answer
2k views

Google Webmaster Central tells me that robots is blocking access to the sitemap

This is my robots.txt User-agent: * Disallow: /wp-admin/ Disallow: /wp-includes/ Sitemap: http://www.mydomain.org/sitemap.xml.gz But Google Webmaster Central tells me that robots is blocking ...
2
votes
2answers
4k views

Googlebot crawler: URL restricted by robots.txt - how to fix the entries in robots.txt?

I noticed recently that Google is not caching all of the pages on my website. Upon using the Google webmaster diagnostic tool, I realized that some of my pages were being restricted by entries in my ...