Dequeue all jobs at cron

Steven Jones - November 4, 2009 - 15:01
Project:Job queue
Version:6.x-3.x-dev
Component:Code
Category:feature request
Priority:normal
Assigned:Unassigned
Status:active
Description

When job queue is invoked from cron, there's a quick count of the number of rows in the job queue table made, and then only that number of jobs are dequeued and run. This prevents jobs adding other jobs and seems a little unnecessary I think.

<?php


function job_queue_cron() {
 
$job_count = db_result(db_query('SELECT count(*) FROM {job_queue}'));
  while (
$job_count > 0 && job_queue_dequeue()) {
    if ((
timer_read('page') / 1000) > ((ini_get('max_execution_time') ? ini_get('max_execution_time') : 240) / 2)) {
      break;
// Stop once we have used over half of the maximum execution time or exceeds the original number of jobs.
   
}
   
$job_count -= 1;
  }
}
?>

could change to:

<?php
function job_queue_cron() {
 
$job_count = db_result(db_query('SELECT count(*) FROM {job_queue}'));
  while (
$job_count > 0 && job_queue_dequeue()) {
    if ((
timer_read('page') / 1000) > ((ini_get('max_execution_time') ? ini_get('max_execution_time') : 240) / 2)) {
      break;
// Stop once we have used over half of the maximum execution time or exceeds the original number of jobs.
   
}
  }
}
?>

#1

hass - November 10, 2009 - 00:34

This is 100% what I've asked for in #373050: Loop job_queue jobs until half cron time has been used and got a by design... :-(

#2

Steven Jones - November 17, 2009 - 10:11

I totally understand this is a duplicate of #373050: Loop job_queue jobs until half cron time has been used but I've got working code that adds the desired feature.

From the other issue:

The best practice for handling failed jobs is for the job to re-queue itself. Often this happens because a 3rd-party resource is offline. This makes a pointless semi-infinite loop if the queue is relatively empty.

Given priority of jobs effects the order in which they are dequeued you can quite easily end up with something looping anyway, so I'm not sure this is a valid defence, I would expect that if there are jobs in the queue job_queue would try to run them if it had time to.

 
 

Drupal is a registered trademark of Dries Buytaert.