problems with using link element as identifier

I never used an item ID due to inheriting the way the Drupal core aggregator worked.

Adding a new field to the aggregator_item table should not be an issue with an update() for the module. If there's no item ID available in the dbase for an item, i'll fall back to checking the link + title are different.

i'll be honest and say i've never aggregated an ATOM feed so don't know anything about their special features.

Looking at my first ATOM feed (http://www.tbray.org/ongoing/ongoing.atom) would it not be suitable to use the get_id() to obtain the full story link for both ATOM and RSS?

Seems to work on my test feeds anyway. The RSS feeds i've checked have a guid element in them, which the get_id() uses.
If there's no get_id() value, feedparser could then fall back to the standard get_link() i guess?

It seems the guid is simply the full url to the story anyway - so no new database fields need to be created after all?

noooooooo try http://baheyya.blogspot.com/atom.xml

ids are not links, ids are unique identifiers, some systems use links cause each post has a page of it's own, some don't cause they support giving posts an # anchor instead of a full page.

It's a shame the parser library has no defined method for generating consistent GUID's for me. As such, i've filed a feature request but will look in to generating ATOM-like GUIDs for all feed items using the methods discussed in http://diveintomark.org/archives/2004/05/28/howto-atom-id#other

using get_id() isn't a viable option. id's aren't required to be a link. some blogs use the url as an id, but a ton do not. it doesn't work with blogger atom feeds (arguably one of the most common feeds on the intarweb). nor does it work with drupal feeds... and it's a sad day when we can't even use the aggregator with drupal feeds...

eh budda said

It seems the guid is simply the full url to the story anyway - so no new database fields need to be created after all?

whichs sounds like replacing links with guids, we do need a new database field there is no avoiding it.

when i downloaded the module (a day and a half ago) it was using get_id() to get the links. by default. i had to change it back to get_link() so it would actually do something useful.

the change from get_link to get_id makes the module useless for many (most?) feeds, changing priority to critical, module as it stands is unusable, it's been 4 days since we reported this already.

There wont be any updates for a while either. I don't do this as my day job or for money, and I'm going on holiday.

understood and you deserve your vacation, for us following the issue it's trivial to patch anyway, I'm just worried about people downloading the module in it's current state.

That's what you get from bleeding edge dev snapshots I guess. No warranties.

Fixed/old file is back in CVS now.

As an update, I'm awaiting an update to the simplepie parser engine to provide a consistent GUID for feed items. Follow my request at http://simplepie.org/support/viewtopic.php?id=677

Having spoken with a SimplePie developer it seems updates are going to be a long time coming due to university commitments. So, at least for now, i'm going down the road of generating my own GUID for any feed - even if an ATOM feed is providing one.

The code doesn't rely on an items title or description/body - so they can still be edited at a later date. The method is based on the post over at diveintomark.org.

function feedmanager_generate_guid($feed_url, $item_date, $item_url = '/') {
  $feed_url_info = parse_url($feed_url);
  $item_url_info = parse_url($item_url);
  return "tag:{$feed_url_info['host']}," . $item_date . ':' . str_replace('#', '/', $item_url_info['path']);
}

Are there any potential problems with replacing a feed provided GUID with our own internal one?

I'll update CVS soon, once I have an upgrade in place for any existing feed items in your site dbase.

CVS now has a new duplicate item system in place, and an upgrade path to generate GUIDs for existing aggregator_item nodes.

You'll need to run update.php to sort your site out. backup your database first! The upgrade worked okay for two of my sites, but I make no promises.