Aggregator: many feeds cause cron time out
alex_b - March 10, 2009 - 22:31
| Project: | Drupal |
| Version: | 7.x-dev |
| Component: | aggregator.module |
| Category: | bug report |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | closed |
| Issue tags: | Novice |
Description
function aggregator_cron() {
$result = db_query('SELECT * FROM {aggregator_feed} WHERE checked + refresh < :time', array(':time' => REQUEST_TIME));
foreach ($result as $feed) {
aggregator_refresh($feed);
}
}Aggregator tries to update all expired feeds on cron. This results in cron timeouts when the number of expired feeds exceeds the available time on cron (max_execution_time).
1) Add a variable of what % of max_execution_time aggregator should use. Let's use a sane default (50 %) and no setting on the UI. This is an advanced setting that can be tuned by experienced site developers and should not bother site builders.
2) Let's use this variable in aggregator_cron() to cancel feed refreshing when aggregator has used 50 % of max_execution_time.
3) The query that loads the feeds to be refreshed shouldn't try to load all expired feeds but only 100 at a time.
#1
This is a novice task.
#2
#3
This should be fixed, IMO, by a proper job queue.
#4
#391340: Job queue API marking postponed.
#5
I think this is closed(in a pretty awesome but non-novice way) by #578676: Use queue for cron now.