Performance - max number of feeds

That's a very difficult question to answer globally.

Overall, aggregation performance depends on:

- Your server's CPU and storage I/O performance
- Your server's network connection
- The content type being created (complex CCK content type? simple Data record?).
- The activity of your feeds being processed (many new items per run?).
- The number of feeds being processed.
- The parser being used (not as critical as other factors).

Usually, as performance degrades you will see feeds that appear to be stale (no new items present although original feed has been updated a while ago). The staleness will increase with the number of feeds you add. A good measure of overall aggregation performance is the time difference between the most recently updated feed and the last updated feed:

# my_importer_id is the id of the importer to be examined (can be looked up in feeds_importer table).
SELECT MAX(last_executed_time) - MIN(last_executed_time) FROM feeds_schedule WHERE id = 'my_importer_id';

I have added #1 to the docs: http://drupal.org/node/622698

Can we consider your question answered?

Hang on, you shouldn't get time outs with 15-20 feeds. You shouldn't get timeouts at all.

- What else are you running on cron (search?)
- Any chance to share the feeds here?
- Are you running on a shared server?
- Have you tried using drush to run cron?
- How do the timeouts manifest themselves? How do you notice that there are timeouts?

Also wondering about performance.

I'm running Feeds on a 4GB server and often get timeouts during the initial import of nodes when I create a new feed.

I'm importing 10-12 youtube feeds. Am using pubsubhub and telling it to wait at least 30 mins before updating again. When I import a node, I'm filling in the embedded media field to create a "video" content type. I'm also shipping the nodes out to Open Calais for automated tagging.

The setup works, but I've noticed my server crapping out on me 10-12 times a day since I started doing this. Not good, obviously.

Any tips on ways to identify where the problem might lie?

Would I improve performance by telling pubsubhub to wait longer before importing stuff?

This is a great module, but like all Drupal modules I've seen that rely on Drupal to import stuff, the "black box" nature of the system makes it hard to ever trust it.

Simplifying title for better legibility of list view of issue queue.

If all of your feeds support pubsubhubbub you can lower your refresh period from 30 minutes to somewhere like 2 days or 7 days.

Yes, PuSH feeds should be slowed down automatically, but we're waiting on #721428: Make scheduler next scheduled time based to make this happen.

Also: could you explain better what "crapping out" means in this context?

Crapping out = Apache dies and needs to be restarted.

In truth, we're still looking to pinpoint the problem, so it can't necessarily be laid at Feeds feet. That's why I'm trying to get a sense of how resource intensive the module is.

I see - 10-12 feed subscriptions doesn't sound bad at all.

I added this question to the ongoing FAQ collection #836618: Compile FAQs

Thanks for your input guys.