Why do many valid links in my blog show error 500 in Google WebMaster Tools?

Question

Majority of the links in the screenshot below are valid! Such as:

And so on... Even if I tried "Fetch as Googlebot" the result is 200 OK.

enter image description here

Please note: I do not block any content for google bot in my robots.txt, neither in my HTML meta tags.

Look at your server logs and see if the Googlebot visited on that date/time and if your server really issued a 500 error. At least then you'll know it's legitimate and not some weird Webmaster Tools problem.
I guess NOT, actually ALL my real blogs whose link format is similar to this: wayneye.com/Blog/[Blog-Title] Are not indexed by Google, you could search google "site:wayneye.com", you will see it never indexed my real blog links... BTW, all those links will return HTTP 301 to redirect to wayneye.com/ViewBlog.aspx?BlogID=xx But many of my links that indexed by google also return HTTP 301, while they were indexed. I was really confused, please help!
What does any of that have to do with my suggestion that you should look at your server logs?

DisgruntledGoat · Answer 1 · 2011-07-29 23:33:25Z

Possible causes for the 500 error you're seeing:

Your robots.txt file starts with an illegal character. When validated here, the file contains a character that's hidden when viewed as UTF-8 in a text editor such as TextMate, but visible when viewed as ISO-8859-1 (see image below). To fix this, delete the first line, retype it, save the file, and revalidate it.
Your site may have simply returned a 500 error when Google last crawled it. (i.e. The server was down.) If this is the case, the errors you should not be repeated when Google next crawls your site, assuming it stays up.

Thanks a lot for the answer! I fixed robots.txt, however, my service provider does not allow me to view the IIS log.... Since ALL my blog posts that URL like wayneye.com/Blog/[Blog_Title] does NOT indexed by google, so I don't think it is because of "server down", instead, I strongly suspect there are something wrong however I couldn't find out...
Even I tried "Fetch as google bot" in Webmaster tool, it returned OK for wayneye.com/Blog/[Blog_Title]

migrated from stackoverflow.com Jul 28 '11 at 17:28