When consuming an Atom feed such as http://www.php.net/releases.atom - I get the status message saying "1 new item(s) were saved. 1 existing item(s) were updated." even on a brand new Drupal install. Looking at the nodes that were created, there are definitely two, and looking in the XML, both items have different <id> values...

Comments

alex_b’s picture

Confirmed that feed is valid: http://validator.w3.org/feed/check.cgi?url=http%3A%2F%2Fdrupal.org%2Ffil...

Looking at this now.

alex_b’s picture

I can't confirm this problem on a clean latest 6.x install of feedapi+common syndication parser: http://skitch.com/alexbarth/bqbsw/php.net-releases-localhost

I'm guessing there's some interaction with previous actions on the feed. May be already one node existed before refreshing?

edsko’s picture

For what it's worth, I had the exact same problem: a refresh on the feed would download 1 of the entries in the feed and "update" the others, even though none of the items were yet in the database.

After a lot of debugging I realized that the UIDs from the feed were not consistent from one download of the feed to the next (the feed in question was http://www.kravmagaeu.com/index.php/en/upcoming-events?view=simplecalend... , generated I think by a Joomla module called "SimpleCalendar").

When I hacked feedapi_node.module to always return TRUE from _feedapi_node_unique all the nodes were properly added; of course, now when I refresh the feed I have make sure to delete all nodes first. Might be useful to add an option to ignore UIDs, always assumes that nodes are unique, but always delete nodes before refreshing (of course, this is doable for small streams only).

lyricnz’s picture

StatusFileSize
new36.03 KB
new17.89 KB
new79.08 KB

I just retested this:
- emptied database from existing D6 install, reinstall drupal
- current feedapi CVS
- parser_simplepie/simplepie.inc 1.1.3
- enable feedapi, feedapi_node, parser_simplepie
- create feed node with http://www.php.net/releases.atom and default options (see first attachment)
- click "refresh", issue is observed as reported! (second attachment)
- open /admin/content/node and observe two items of content (one is feed, one is content)

lyricnz’s picture

I just tried this yet again, with an utterly empty site (from scratch, empty filesystem and database). Same results as above. Attached is a complete log of my drupal installation, and new screenshots (same as above).

Notice, in particular, that only one of the two nodes has the "new" tag.

Next, I installed devel module (fourth attachment), so I can look at those created nodes more carefully. The new feed item looks fine (fifth attachment). Time to add some debugging to feedapi to figure out why it thinks it's updating an item...

lyricnz’s picture

StatusFileSize
new110.84 KB

Adding some debug output to _feedapi_invoke_refresh() shows that the feed looks basically okay, and it's processing both nodes, per first attachment. It's calling _feedapi_node_unique() to determine whether the items are distinct, and it thinks that they're the same! This appears to be because of:

  // Feed item is duplicate, if URL or GUID are duplicate or if they are both missing.
  if (isset($feed_item->options->original_url)) {
    $nid = db_result(db_query("SELECT fni.nid FROM {feedapi_node_item} fni JOIN {feedapi_node_item_feed} ff ON ff.feed_item_nid = fni.nid WHERE fni.url = '%s' AND ff.feed_nid = %d", $feed_item->options->original_url, $feed_nid));
    if ($nid !== FALSE) {
      return $nid;
    }
  }

When $feed_item->options->original_url is an empty string (as opposed to unset), like it is in this case, it will keep thinking nodes are duplicates.

So... the problem is really that the feed item original_url should not be set, rather than being empty. Or, this check should treat the two the same.

lyricnz’s picture

Status: Active » Needs review
StatusFileSize
new1.8 KB

It seems this was broken by this change which added html_entity_decode() to the output of simplepie. However html_entity_decode(NULL) ==> '', which is what's causing the problem downstream.

Suggest we change this to check for NULL first, see attached patch.

lyricnz’s picture

(FWIW, my original report was confused by the two nodes - one of which was the _feed_ and one of which was the created+updated feed item)

aron novak’s picture

In #7, the patch looks good, does not break anything.

However html_entity_decode(NULL) ==> ''

Which version of PHP? For me, NULL, simply returns an empty string:

dprint_r(serialize(html_entity_decode(NULL)));
s:0:&quot;&quot;;
lyricnz’s picture

That's what I mean by '' - an empty string (which is what's breaking _feedapi_node_unique)

From #6: "When $feed_item->options->original_url is an empty string (as opposed to unset), like it is in this case, it will keep thinking nodes are duplicates."

aron novak’s picture

Status: Needs review » Fixed

Thanks, commited.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.