Fetch as Google goes infinite loop (again)

Question

A couple of days ago, I faced an issue with Google Webmaster's Fetch as Google functionality. The page was going into an infinite refresh loop. I started a thread on Google's forum. The issue disappeared by itself.

Unfortunately, this issue is happening again. Google still does not have the latest copy of my main page in its cache. The Fetch as Google page is going into an infinite loop again.

Anyone knows why? Any idea/suggestion is welcome... Thanks!

Answer 1

This is a known problem with Webmaster Tools as you have first hand experience when you noticed it before. You can test your own site using firebug and reading the response you get.

Test Your Own Pages

While Webmaster Tools is good it should be noted its not perfect and more than often out of date as it's not interlinked in real time with their indexer or caching service. You can test your own pages if you have Google doubts, simply install firebug and your be able to see from the header messages what your site is returning.

I've just tested your site and can confirm that its work as intended I get 200 with no 301 redirects meaning the infinity loop doesn't exist and something that is clearly wrong with the Google Fetch Bot. You can review the response I got from below.

GET yourcurrencyconverters-tostaky.rhcloud.com 200 STATUS OK

Chances are Google will fix the problem in a few days time so test it then but in the mean time just test it yourself using Firebug.

Additional Reading

I've read your post on http://productforums.google.com/forum/#!topic/webmasters/Voqv-_Ts2AE and I can confirm that the index.html also returns a valid response

GET index.html 200 OK

Webmaster Tools is often bugged, outdated, or just plain annoying

As mentioned in the previous question and adding to this one, Google Webmasters, Googles Rankings and Indexing and Even Google cache service are all independent services when one gets updated it can often take many days afterwards to update in Webmaster Tools and in other services.

Trailing Slashes

One small possiblity is that your site is causing a redirect on the trailing end slash, its unlikely but make sure you test the fetch with and without the ending slash to see if theres any different.

http://yourdomain.com/
http://yourdomain.com 

Things to Do

As mentioned Google takes time and rather concerning yourselve too much maybe and I hope you don't take offence but maybe you should focus more time on getting your site out of "This site is still under construction! Come visit us again later!" and into a live site, since watching it on Google won't do any good. Simply get on with more construction things and care little about Google.

Answer 2

Just like bybe, I tested your site and found no problems. I use a command line tool called "curl" to do so. It can send a user agent like Googlebot

curl -D - -s -A 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)' http://yourcurrencyconverters-tostaky.rhcloud.com/

when I run that, I get the correct response

HTTP/1.1 200 OK

I would also suggest installed in plugin for your browser that can switch your user agent. With such a plugin you could test sending requests much like Googlebot does.

---

Even with this testing, there still could be problems with your site and it might not be webmaster tools fault.

• You recently changed your DNS. In such a case Googlebot may still be using an old IP address for your site and hitting the wrong server. If this is the case, you will need to wait until the time-to-live (TTL) has expired on the DNS entry that Googlebot has.
• You or your ISP has a rule in place to filter or redirect traffic coming from the IP address that Googlebot is using.

If you have access to them, look at your server logs while you use the fetch as Googlebot tool. Make sure the request from Googlebot actually makes it to your server. See what response code your server logged that it sent. See if there are any error messages logged about that request.