Posted by Steven Jones on November 4, 2009 at 3:01pm
| Project: | Job queue |
| Version: | 6.x-3.x-dev |
| Component: | Code |
| Category: | feature request |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | needs review |
Issue Summary
When job queue is invoked from cron, there's a quick count of the number of rows in the job queue table made, and then only that number of jobs are dequeued and run. This prevents jobs adding other jobs and seems a little unnecessary I think.
<?php
function job_queue_cron() {
$job_count = db_result(db_query('SELECT count(*) FROM {job_queue}'));
while ($job_count > 0 && job_queue_dequeue()) {
if ((timer_read('page') / 1000) > ((ini_get('max_execution_time') ? ini_get('max_execution_time') : 240) / 2)) {
break; // Stop once we have used over half of the maximum execution time or exceeds the original number of jobs.
}
$job_count -= 1;
}
}
?>could change to:
<?php
function job_queue_cron() {
$job_count = db_result(db_query('SELECT count(*) FROM {job_queue}'));
while ($job_count > 0 && job_queue_dequeue()) {
if ((timer_read('page') / 1000) > ((ini_get('max_execution_time') ? ini_get('max_execution_time') : 240) / 2)) {
break; // Stop once we have used over half of the maximum execution time or exceeds the original number of jobs.
}
}
}
?>
Comments
#1
This is 100% what I've asked for in #373050: Loop job_queue jobs until half cron time has been used and got a by design... :-(
#2
I totally understand this is a duplicate of #373050: Loop job_queue jobs until half cron time has been used but I've got working code that adds the desired feature.
From the other issue:
Given priority of jobs effects the order in which they are dequeued you can quite easily end up with something looping anyway, so I'm not sure this is a valid defence, I would expect that if there are jobs in the queue job_queue would try to run them if it had time to.
#3
+1 for this feature. My use case involves fetching an XML resultset from a resource, and recursively queueing a number of additional new jobs based on the contents of the resultset (the results of these jobs in turn spawn new jobs and so on)...
Steven Jones' looks sound to me, attached is a patch against HEAD that incorporates this change in case anyone wants to test-drive this change.
#4
Since $count not used function could be
<?phpfunction job_queue_cron() {
while (job_queue_dequeue()) {
if ((timer_read('page') / 1000) > ((ini_get('max_execution_time') ? ini_get('max_execution_time') : 240) / 2)) {
break; // Stop once we have used over half of the maximum execution time or exceeds the original number of jobs.
}
}
}
?>