1

We have a website with a RSS feed. When I use Google Reader to subscribe to the feed, it is only updated once in a while - something like every 4-5 days. In Google Reader the latest 2 or 3 posts appear at the same time.

I have setup an quick test script that downloads the RSS feed every hour, so that I could monitor, that the feed is updated as expected. And everything looks good, when I just download the feed.

I'm beginning to believe that the cause for this behavior in Google Reader is to do with Googles RSS crawler, feedfetcher. When I look at Date Received and Date Published - Google Reader differentiate between Date Received and Date Published - Date Received is the same for the 2 or 3 posts, and Date Published matchtes the pubDate in the feed.

This leads me to believe that Google's RSS crawler, feedfecther, doesn't crawl the RSS feed more that something like every 4-5 days.

What are your experiences with Google feedfetcher? Is it expected to have this kind of latency?

(I haven't looked at the server logs yet, but that would be a logically next step.)

Improve this question
8
  • 1
    Do you have a ttl set in the feed? I can't confirm whether or not Google pays any attention to it, a quick search produced conflicting reports, but it may be worth investigating.
    – robertc
    Jan 6, 2012 at 19:14
  • How does it work if you ping? pingomatic.com My experience is that GR updates immediately after a post in my unknown Wordpress blog.
    – Osvaldo
    Jan 6, 2012 at 20:04
  • Try adding your feed to feedburner.google.com and ensure your publishing system/CMS is pinging services like Pingomatic
    – Mike Hudson
    Jan 7, 2012 at 10:45
  • @robertc No TTL defined anywhere.
    – Emil Rasmussen
    Jan 8, 2012 at 12:48
  • @Osvaldo I will look at Pingomatic - thanks for the tip.
    – Emil Rasmussen
    Jan 8, 2012 at 20:55

2 Answers 2

Reset to default
1

After monitoring Google Reader for a couple of weeks, it seems that Google's feedfetcher bot now visits the RSS feed on a regular schedule. The conclusion therefore is that there are some degree of latency between the first subscription to a RSS feed from a Google service till the RSS feed are regularly indexed.

Improve this answer
1

Feeds are checked "about once an hour" but that schedule "may" be reduced if there aren't many subscribers. (Note the usual vagueness you get from Google about pretty much everything. This was one of the reasons I stopped using Reader for almost all feed reading myself; there's no real way of knowing unless you actually publish the feed yourself and check.)
I've never encountered a lag as bad as you're describing, though.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.