The default rss size for drupal.org appears to be 10 items. This can cause problems for feedreaders such as the google reader (currently the most popular client) for feeds which are frequently updated like the cvs feed. The result is that subscribers to these feeds end up not seeing items that are in the feed because by the time the google reader comes back to the site some items have moved through the feed and are no longer available. This is particularly the case for the cvs feed at http://drupal.org/cvs?rss=true

I suggest we increase the rss size from 10 items to 30 http://drupal.org/admin/content/rss-publishing

I haven't tested yet and there is a chance that cvs.module doesn't respect this setting. If that is the case rest assured that I'll patch it ;)

Comments

sepeck’s picture

sepeck commented

It seems more a limitation of using Google reader then anything else. What sorts of performance issues are there potentially with the large number of feeds we currently have?

moshe weitzman’s picture

moshe weitzman commented

this has nothing to do with google reader limitations. they are an consumer like any other ... this is so easy to change and to revert, that i think we should just change it and watch for DB or network badness.

greggles’s picture

greggles
he/him
Primary language English
Location Denver, Colorado, USA
commented

@Sepeck - I tried going that route first. There is no configuration that I have control over in the google reader application to be able to fetch the feed more often. I submitted a request for ideas or help to the reader group (which is monitored by google, but apparently not closely monitored) http://groups.google.com/group/google-reader-howdoi/browse_frm/thread/b7... That's what gave me the idea to increase this number.

For another example of this problem, take a look at the hoops that Gabor jumped through to get yahoo pipes

A possible problem here is that the refresh interval can only be set as short as 15 minutes, which could be too long, given the frequency of commits at drupal.org. If commits run out of the ten commits long window showed in the RSS feed, we don't see them in Yahoo! Pipes and as a result, we don't aggregate them at drupal.hu. Thankfully aggregator.module can be form_alter()-ed, so we can set a shorter interval if need be.

From http://hojtsy.hu/blog/2007-mar-20/yahoo-pipes-wizardry-drupal-translatio...

So, google reader and yahoo pipes show this problem. Most feed readers are going to be fetching the feeds as anonymous and therefore squid-cached users. I have a hard time imagining this creating a performance problem.

I agree this is a limitation of most web-based feed readers that should be fixed in the readers and not on drupal.org but, much like the html standards incompatibilities from IE6, when the majority of users are using a broken tool we sometimes have to make small adjustments to get our systems to work with them. This is a really small tweak on our side that is easy to flip back.

greggles’s picture

greggles
he/him
Primary language English
Location Denver, Colorado, USA
commented

Pardon, on re-reading I see that was a problem with our aggregator and not yahoo pipes. All the same I think it helps show that 10 commits is not enough.

sepeck’s picture

sepeck commented

My comment on the limitations of the client check in was more observationally random then anything else though it seems to have been interpreted differently by the comments.

I would coordinate an increase with killes. He is in a position to know what needs to be done to both monitor the effect of the change and when doing the change would not be done at the same time as other work/changes which would skew the measurements.

greggles’s picture

greggles
he/him
Primary language English
Location Denver, Colorado, USA
commented
Status: Active » Fixed

I asked killes, he said fine, so I changed the setting.

Turns out cvslog doesn't care about that setting - so, here's the issue to fix that http://drupal.org/node/221295

Anonymous’s picture

Anonymous (not verified) commented
Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.