1

For the last couple months, I've been unable to fetch any pages of my site in Google Search Console.

(The error message says "Temporarily Unreachable")

Here are the possible causes I've come across and the ways I've tried to test them.

*The page doesn't exist. (I don't think this is it because the page is visible to any non-bot visitor.)

*The Googlebot is being blocked by my robots.txt file. (I don't think this is it because the only entry on this file is User-agent: * )

*The Googlebot is being blocked by my hosting company. (I don't think this is it because I viewed the bot traffic stats and Googlebot is successfully crawling the site.)

*The page takes too long to load and creates a timeout error. (I don't think this is it. Some of my pages are genuinely slow, but--as a test--I also uploaded some small, mostly blank html pages which load in half a second. And, they won't fetch either.)

Although a variation of this question have been asked elsewhere on the site, the suggested answer--that this is a temporary issue in Google Search Console--doesn't apply in this case as the issue has been persistent for eight weeks in my case.

If you can see an error in my testing logic above, please let me know. Alternatively, if you can think of another angle from which I should approach this, I'd be curious. Thanks!

|improve this question
  • Other than that google console is working properly? – John Could Oct 27 '18 at 23:42
  • I'm not sure what you're asking. – user2258740 Oct 28 '18 at 0:23
  • 1
    Check your headers to make sure your server isn't accidentally serving a x-robots-tag header that's blocking the bots. – Maximillian Laumeister Oct 28 '18 at 2:51
  • I just tested this, but it doesn't appear to be the issue either. Thanks for suggestion, though. – user2258740 Oct 28 '18 at 19:37
  • Have you tried using the url inspection tool then doing a live check. It's also worth trying the mobile friendly testing tool as that requests as Googlebot. Another test is using the Structured Data Testing Tool. It's from Google but as a different user agent. – Tony McCreath Oct 28 '18 at 23:17
2

What is pretty clear is that this really only relates to the GSC Fetch tool.

Google's help page tells us that the "Temporarily unreachable" error occurs when:

  1. Fetch as Google can't currently fetch your URL because the server took too long to reply.
  2. Or Fetch as Google cancelled your fetch because too many consecutive requests were made to the server for different URLs.

So this can be when Googlebot is not able to fetch your given URL due to maximum timeout, IP blocking, the page is blocked in the robots.txt file, the page is blocked with noindex attribute or hosting server may be down.

As per No.2 it is often that Google itself may be limiting the requests to your server, so use it less. If none of the above reasons are in play, give it some time before using the Fetch tool again and don't use it too much when you do, and I would expect this would resolve the issue.

Also be mindful not to include the preceding / in your URLs as this is already included by Google and it may be causing a double request.

|improve this answer
  • Thanks for weighing in. Of the five possibilities you listed in your note above, four were covered in my original post. The last--a no-index attribute--isn't the issue in this case either. As for giving Fetch more time, I hadn't used it in weeks when I ran the tests above, so that shouldn't be the issue either. If you have other suggestions, please send them on--as I'd be happy to test those as well. – user2258740 Oct 28 '18 at 19:40
  • The thoughts of other possible reasons are the type of pages that are being loaded (many asynch scripts etc. or convoluted coding making lots of calls to the server), or it may be DNS request issues, or you are on shared hosting which is not appreciating the number of requests. Have you asked your hosting co. what they are seeing in the logs at these precise times? Maybe also test your DNS requests through MXtoolbox or similar. – garth Oct 31 '18 at 1:22
  • Thanks for following up. After your note, I tested removing all but one of the posts on my site to see if the issue was with the framework of the site or the size of the site. With just one post up, Fetch worked properly. To that end, I don't have a solution in place yet, but it appears that the issue is specific to site size/database size or the resource usage and site speed issues that result from the site/database size. – user2258740 Oct 31 '18 at 17:26
  • @user2258740 Although you stated in your question that you "also uploaded some small, mostly blank html pages which load in half a second. And, they won't fetch either." - These small HTML pages presumably bypassed any server-side processing? – DocRoot Oct 31 '18 at 20:03
  • @DocRoot This is an excellent point, and I don't know how to account for it. If anyone reading this has an idea, I'd welcome any input. – user2258740 Nov 1 '18 at 2:58

Not the answer you're looking for? Browse other questions tagged or ask your own question.