We have a website with a RSS feed. When I use Google Reader to subscribe to the feed, it is only updated once in a while - something like every 4-5 days. In Google Reader the latest 2 or 3 posts appear at the same time.

I have setup an quick test script that downloads the RSS feed every hour, so that I could monitor, that the feed is updated as expected. And everything looks good, when I just download the feed.

I'm beginning to believe that the cause for this behavior in Google Reader is to do with Googles RSS crawler, feedfetcher. When I look at Date Received and Date Published - Google Reader differentiate between Date Received and Date Published - Date Received is the same for the 2 or 3 posts, and Date Published matchtes the pubDate in the feed.

This leads me to believe that Google's RSS crawler, feedfecther, doesn't crawl the RSS feed more that something like every 4-5 days.

What are your experiences with Google feedfetcher? Is it expected to have this kind of latency?

(I haven't looked at the server logs yet, but that would be a logically next step.)

  • 1
    Do you have a ttl set in the feed? I can't confirm whether or not Google pays any attention to it, a quick search produced conflicting reports, but it may be worth investigating.
    – robertc
    Jan 6 '12 at 19:14
  • How does it work if you ping? pingomatic.com My experience is that GR updates immediately after a post in my unknown Wordpress blog.
    – Osvaldo
    Jan 6 '12 at 20:04
  • Try adding your feed to feedburner.google.com and ensure your publishing system/CMS is pinging services like Pingomatic Jan 7 '12 at 10:45
  • @robertc No TTL defined anywhere. Jan 8 '12 at 12:48
  • @Osvaldo I will look at Pingomatic - thanks for the tip. Jan 8 '12 at 20:55

After monitoring Google Reader for a couple of weeks, it seems that Google's feedfetcher bot now visits the RSS feed on a regular schedule. The conclusion therefore is that there are some degree of latency between the first subscription to a RSS feed from a Google service till the RSS feed are regularly indexed.


Feeds are checked "about once an hour" but that schedule "may" be reduced if there aren't many subscribers. (Note the usual vagueness you get from Google about pretty much everything. This was one of the reasons I stopped using Reader for almost all feed reading myself; there's no real way of knowing unless you actually publish the feed yourself and check.)
I've never encountered a lag as bad as you're describing, though.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.