2

I added a tube (with AVS script) in Google Webmaster Tools, but the issue is with the robots.txt file which is blocking everything.

Here is my AVS tube robots.txt:

User-agent: *
Allow: /
Disallow: /siteadmin/
Disallow: /cache/
Disallow: /include/
Disallow: /install/
Disallow: /language/
Disallow: /media/
Disallow: /tmp/
Disallow: /classes/
Disallow: /templates/
Disallow: /scripts/
Disallow: /addons/
Disallow: /cron/
Disallow: /contrib/

Have to mention that the site is with non-www version. When someone types in the www. version of the site it will be redirected to non-www version.

This is the message from Webmaster Tools:

Url blocked by robots.txt.

Sitemap contains urls which are blocked by robots.txt.

  • Having just launched a new site I'm having exactly the same problem. Never seen it before. I'm wondering if Webmaster Tools (Search Console) has a cached version of my robots file (initial launch had Disallow: /* from when we were testing). I can see the new robots.txt in the testing system... but it is blocking every page, and none of the pages sit in blocked folders. – user60913 Feb 18 '16 at 21:57
  • Can you upload a copy of your sitemap file as the error being displayed indicates that there is a URL in your sitemap file which is blocked by the robots.txt, Need to see the sitemap.xml file to check what is happening. – Chris Rutherfurd Apr 15 '16 at 3:16
  • How can I PM you to show you the sitemap? – El Patron Apr 16 '16 at 17:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.