Robots.txt never downloaded but some blocked URLs in GWT

Question

There is something I don't understand in Google Webmaster Tools (GWT) for my Wordpress site.

In menu "Blocked URLs", it mention that my robots.txt has never been downloaded but there are some blocked URLs. It's kind of weird and not logical. Am i missing something?

User-agent : *
Disallow: /*?
Disallow: /wp-login.php
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content
Allow: /wp-content/uploads
Disallow: */trackback
Disallow: /*/feed
Disallow: /*/comments
Disallow: /cgi-bin
Disallow: /*.php$
Disallow: /*.inc$
Disallow: /*.gz$
Disallow: /*.cgi$
Disallow: /author/*

I'm afraid my robots.txt doesn't block several URLs I want to block.

Edit (screenshot):

Answer 1

Google will sometimes say it's been blocked by robots.txt when in fact it is a noindex tag preventing Google from indexing the page. If you have any plugins for maintenance mode or others which can possibly set noindex on your home page or other pages I would suggest you go to the page view the source in your browser and check that there is no noindex being rendered by a plugin.

Answer 2

It may just be a glitch – they do happen from time to time.

One other possibility occurs though: your screenshot is from GWT on a non-www version of your domain. Do you have the www version authorised as well (e.g., so that you can set preferred domain)? If so, and the www version is canonical, it could explain why Google would know how many URLs are blocked without technically having downloaded the file (they downloaded the www version… which is the same file really, but isn't in a pedantic techie sense).

Bit of a long shot, but that's about the only explanation, other than a glitch, I can come up with.

Answer 3

Problem comes from encoding of robots.txt file. UTF-8 encoding is recommended by Google.