3

In my Google Webmaster Tools, I received an error which I am not understanding:

When we tested a sample of the URLs from your Sitemap, we found that some of the URLs were unreachable. Please check your webserver for possible misconfiguration, as these errors may be caused by a server error (such as a 5xx error) or a network error between Googlebot and your server. All reachable URLs will still be submitted.

I received this warning for a URL that is currently working perfectly. Can you please tell me if this a serious warning, or what it means?

Improve this question
2
  • 1
    Have you tried the "fetch as Google" feature in Webmaster Tools with that URL? It can give you more information if Google can't get to a page.
    – Stephen Ostermiller ♦
    Jun 24, 2014 at 19:04
  • 1
    and you get this error again and again ? google is buggy today
    – john Smith
    Jun 24, 2014 at 21:44

1 Answer 1

Reset to default
1

This simply means Googlebot could not search (crawl) the URLs defined in the provided sitemap. Failure can be caused by a number of reasons.

Some that I have found include:

  • Sitemap is syntactically incorrect. Use the W3 Validator to check.
  • Sitemap is incorrectly listing URLs.
  • Site was down/offline.
  • Site was slow and their crawler timed out.
  • Security settings prevented their bot from reaching specific pages. HTTP Authentication is a common issue.
  • Pages may no longer exist (which they should if they are in a sitemap.)
  • Corrupted HTTPS configuration.
  • Mobile sites that restrict access to mobile only clients.

You will want to examine your site's access and error logs for Googlebot queries. You can easily identify this by looking at the User-Agent string in your logs.

Here is a list of Google crawler agents.

Improve this answer
2
  • 1
    DNS's down/unreachable, redirect loop, might be other reasons
    – Binarysurf
    Jun 24, 2014 at 19:58
  • I forgot about the redirect loops. I think by default, Google will only follow about 5 redirects.
    – jeffatrackaid
    Jun 24, 2014 at 19:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.