Take the 2-minute tour ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.

We have a website with a RSS feed. When I use Google Reader to subscribe to the feed, it is only updated once in a while - something like every 4-5 days. In Google Reader the latest 2 or 3 posts appear at the same time.

I have setup an quick test script that downloads the RSS feed every hour, so that I could monitor, that the feed is updated as expected. And everything looks good, when I just download the feed.

I'm beginning to believe that the cause for this behavior in Google Reader is to do with Googles RSS crawler, feedfetcher. When I look at Date Received and Date Published - Google Reader differentiate between Date Received and Date Published - Date Received is the same for the 2 or 3 posts, and Date Published matchtes the pubDate in the feed.

This leads me to believe that Google's RSS crawler, feedfecther, doesn't crawl the RSS feed more that something like every 4-5 days.

What are your experiences with Google feedfetcher? Is it expected to have this kind of latency?

(I haven't looked at the server logs yet, but that would be a logically next step.)

share|improve this question
1  
Do you have a ttl set in the feed? I can't confirm whether or not Google pays any attention to it, a quick search produced conflicting reports, but it may be worth investigating. –  robertc Jan 6 '12 at 19:14
    
How does it work if you ping? pingomatic.com My experience is that GR updates immediately after a post in my unknown Wordpress blog. –  Osvaldo Jan 6 '12 at 20:04
    
Try adding your feed to feedburner.google.com and ensure your publishing system/CMS is pinging services like Pingomatic –  Mike Hudson Jan 7 '12 at 10:45
    
@robertc No TTL defined anywhere. –  Emil Rasmussen Jan 8 '12 at 12:48
    
@Osvaldo I will look at Pingomatic - thanks for the tip. –  Emil Rasmussen Jan 8 '12 at 20:55
show 3 more comments

2 Answers

Feeds are checked "about once an hour" but that schedule "may" be reduced if there aren't many subscribers. (Note the usual vagueness you get from Google about pretty much everything. This was one of the reasons I stopped using Reader for almost all feed reading myself; there's no real way of knowing unless you actually publish the feed yourself and check.)
I've never encountered a lag as bad as you're describing, though.

share|improve this answer
add comment
up vote 1 down vote accepted

After monitoring Google Reader for a couple of weeks, it seems that Google's feedfetcher bot now visits the RSS feed on a regular schedule. The conclusion therefore is that there are some degree of latency between the first subscription to a RSS feed from a Google service till the RSS feed are regularly indexed.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.