function aggregator_cron() {
  $result = db_query('SELECT * FROM {aggregator_feed} WHERE checked + refresh < :time', array(':time' => REQUEST_TIME));
  foreach ($result as $feed) {
    aggregator_refresh($feed);
  }
}

Aggregator tries to update all expired feeds on cron. This results in cron timeouts when the number of expired feeds exceeds the available time on cron (max_execution_time).

1) Add a variable of what % of max_execution_time aggregator should use. Let's use a sane default (50 %) and no setting on the UI. This is an advanced setting that can be tuned by experienced site developers and should not bother site builders.
2) Let's use this variable in aggregator_cron() to cancel feed refreshing when aggregator has used 50 % of max_execution_time.
3) The query that loads the feeds to be refreshed shouldn't try to load all expired feeds but only 100 at a time.

CommentFileSizeAuthor
#2 397872_aggregator_timeouts.patch1.39 KBaron novak

Comments

alex_b’s picture

Issue tags: +Novice

This is a novice task.

aron novak’s picture

Status: Active » Needs review
StatusFileSize
new1.39 KB
dries’s picture

This should be fixed, IMO, by a proper job queue.

catch’s picture

Status: Needs review » Postponed

#391340: Job queue API marking postponed.

neclimdul’s picture

Status: Postponed » Closed (fixed)

I think this is closed(in a pretty awesome but non-novice way) by #578676: Use queue for cron now.