hook_queue_info or hook_cron_queue_info - Meta queue information [#1233682]

Queue UI defines its own hook for queues: hook_queue_info()

Sadly I didn't get that hook documented in the code very well but here's an example:

function queue_ui_queue_info() {
  return array(
    'queue_ui_test' => array(
      'title' => t('Test queue'),
      'batch' => array(
        'operations' => array(array('queue_ui_batch_test', array())),
        'finished' => 'queue_ui_batch_finished',
        'title' => t('Processing test queue'),
      ),
      'cron' => array(
        'callback' =>'queue_ui_test_process',
      ),
    ),
  );
}

Core defines hook_cron_queue_info which has some overlap

Let's discuss a path forward.

If you use hook_cron_queue_info() what are you using it for? Do you want to continue using it for cron workers?

Comments

Comment #1

Dinesh Kumar Sarangapani commented 16 July 2012 at 05:41

Hi coltrane,

So queue manager uses its own hook_queue_info() to define the queue as you have mentioned. But the hook is for Drupal 8 and not Drupal 7.

Drupal 7 uses "hook_cron_queue_info", let us define the the batch and other cron in the same drupal hook which was defined for.

Comment #2

tsphethean commented 10 February 2013 at 11:43

I think the advantage of hook_queue_info is that it doesn't care how you're going to process the queue. Defining queues by hook_cron_queue_info makes the assumption that queue processing is going to be done by cron, which IMO isn't always desirable.

A majority of queues that I've implemented have had custom drush commands written to expose the queue processing to a command which can then be scheduled via crontab or other schedulers. In Drupal cron world (out of the box), to get a queue processed at regular, short intervals would involve the cron interval to be set very short, which could be a performance hit on busy sites running the entirity of hook_cron. With drush commands, or similar, the queue processing doesnt tie up web server threads as it is executing PHP directly, and jobs can be split between servers or run on a server which doesnt receive any web traffic - minimising the impact on the end user.

What would be truly awesome, would be if Queue UI exposed a standard queue processing drush command, which used hook_queue_info for queue discovery and processing. Custom arguments could be defined in the hook for passing into the queue processing callback. This would allow really simple separation of processing with very little custom code to be written by queue implementers.

Comment #3

coltrane

he/him

commented 11 February 2013 at 16:47

"a standard queue processing drush command, which used hook_queue_info for queue discovery and processing"

Drush 5 has provided the queue-run and queue-list commands which take an item from the command line argument and passes it to the hook_queue_info() (and even hook_cron_queue_info()) callback function. The drush queue functions are at http://api.drush.ws/api/drush/commands!core!queue.drush.inc/5.x.

One problem of drush's queue processing command is you don't have control over claiming the item or deleting it, which may be a problem for some.

Comment #4

tsphethean commented 11 February 2013 at 21:53

Ah I'd not seen that before, thanks for the link.

I guess this could be extended if we exposed more callbacks in hook_queue_info. Thinking through the scenarios we'd need:

Callback for claiming, as you say
The item processing callback (existing worker callback) - this should return true or false
Success callback - when worker returns true, if not defined default to deleting queue item
Failure callback - when worker returns false, if not defined default to releasing queue item (or moving to error queue?)

Comment #5

damienmckenna

TN, USA

commented 13 April 2013 at 03:46

I think the project page should specifically state "this does not integrate with the core cron queue, only queues built to work with the new hook_queue_info."