Feeds from PubMed not getting parsed

interesting - i can confirm this - just tried to add your feed to my own install and got nothing in the display BUT it said "created 15 records" when it was added

this same thing happened to me with an rss for most recent by tag from smugmug.com- it said that it created 100 records, but only shows 1 item...

is their something odd going on with parsing? my other feeds are rolling in just fine...

The problem is that the feed items that PubMed is generating don't have a publication date, so they have a timestamp of 0 and won't show up in Managing News with the default view, which only displays items newer than three months (items with no publication date are dated to 1970). You can change this behavior by eliminating that filter, overriding the view 'mn_news'.

Hi tmcw,

Thanks for the information, and your suggestions. I also find that PubMed feeds are not displayed. However, I have somewhat different experience with other feeds than does saltcod. For me, two RSS feeds from the journal Science also do not display, but two RSS feeds from the journal Nature do display. Also different that saltcod, one of the feeds that displays has records that are stored in the 'feeds_data_syndication' table, but several other feeds - with distinct nids - that work fine do not seem to be stored in that table.

I think there are some differences in the data structures of the feeds that are underlying these differences. I have seen such differences in feeds from different sources using other aggregators.

Also, I tried your suggestion of overriding the mn_news view filter for Syndication: timestamp. I also changed the Data Processor settings at:

http://localhost/managingnews10b5/admin/build/feeds/edit/syndication/set...

from '12 weeks' to 'Never', so that PubMed articles will be displayed instead of hidden, as suggested by your post above. However, neither of these changes fixed the problem, and the MN app behaved exactly the same as before. Is there some other way of doing this that you had in mind?

Any suggestions about how to proceed?

#6 @saltcod: I think #7 is your answer.
#8 @mcaudy: can you review the database and see whether the items actually exist? The table is feeds_data_syndication?

Not having looked at pubmed myself, the actual way to fix this would be on the Feeds module level: Invalid publication dates should be replaced by the current timestamp when they're saved. We may also want to do more advanced checking for available dates on the parser level in Feeds.

Moving issue to Feeds.

@alex_b: I have looked in the feeds_data_syndication table, and the data for the feeds is there, both for the feeds which are displayed, and the feeds which are not. As I said in my previous post, the feeds from Nature are displayed, but the feeds from Science and most of the PubMed feeds are not, and all of these feeds have many, many items that fall within the time period which should be displayed.

** Thus, it seems that there is some kind of feed data discrepancies that determine whether specific feeds display or not.

I therefore have included as much info as possible about the various results, so that the problem can be worked out.

I have made a table (below, at bottom) which shows the results.

You can access the site itself at:

http://www.mcaudy.net/managingnews10b5/tracker

(I will give you and any other Dev Seed people partial admin privileges for this site, if you want.)

You also can download a sql export dump of the feeds_data_syndication table at:

http://www.mcaudy.net/managingnews/mcaudy_feeds_data_syndication_v1.sql

An Excel file showing the same feeds_data_syndication table data can be downloaded at:

http://www.mcaudy.net/managingnews/mcaudy_feeds_data_syndication_v1.xls

in case anyone would rather see the data in tab format. Sheet 2 of this excel file also contains a copy of the info in the table below, but in a much clearer format.

The table below shows which of several PubMed and journal feeds displayed in MN, and which did not, as well as the title and Feed NID, in case you want to look through the feeds_data_syndication table. I also included the URLs for the feed data, so you can look at most of the data directly, if you want.

The HubMed feed (Feed NID 7) shown below is odd in that it originally did not display at all in MN. However, when I changed the syndication timestamp display criteria as per tmcw's suggestions, some data did display, but only for a single item, although there are many, many more items in that feed that are even more recent than the single item displayed.

Thanks for your help in getting this sorted out. And thanks very much for creating Managing News, and the new Feeds modules that will take Drupal feed aggregation to a new level.

Mike Caudy

Displays? FEED NID Feed Title URL
yes 3 Drupal Blogs, News, Modules and Themes new http://pipes.yahoo.com/pipes/pipe.run?_id=LFtDHHMs3BG_Un2LJhOy0Q&_render...
yes 4 DrupalFlexPG Pages (Nodes) as an RSS feed new http://www.mcaudy.net/drupalflexPG/page_rss_feed
yes 8 Drupal-KM Features Demo new http://www.mcaudy.net/acquiadrupal100/rss.xml
no/yes* 7 HubMed - transcription regulation new http://www.hubmed.org/feeds/rss.cgi?q=transcription%20regulation
yes 15 Mashable! new http://feeds2.feedburner.com/Mashable
yes 17 Nature - Current Issue new http://feeds.nature.com/nature/rss/current?format=xml
yes 16 Nature AOP (Advance Online Publishing) new http://www.nature.com/nature/journal/vaop/ncurrent/rss.rdf
yes 5 PubMed RSS Feed via AcquiaDrupal:8083 new http://192.168.1.104/acquia-drupal/rss.xml
no 6 pubmed: notch proneural new http://eutils.ncbi.nlm.nih.gov/entrez/eutils/erss.cgi?rss_guid=1-mNKjUkf...
no 14 Science: Current Issue new http://www.sciencemag.org/rss/current.xml

saltcod, tmcw, mcaudy: these were helpful reports.

I am pretty sure that the feeds that don't work all suffer from the same problem: missing publication dates. As a result, the publication date falls back to UNIXTIME 0.

I created a new issue focussing on the fix: #618018: If pubDate is 0 or empty, use current time

Setting this issue to duplicate.