Cron jobs fall into a couple of main categories:

- things that have to be run periodically and don't care about site events - mainly garbage collection like session and cache.
- batch processing of things - search indexing of new nodes, purging of deleted stuff.

For the latter case, these are increasingly moving to the queue API, although it's not 100% consistent in core.

Issues like #943772: field_delete_field() and others fail for inactive fields, and the one I can't find about indexing nodes for searching immediately, might be helped by a poor mans queue runner.

Drupal 7 has a poor mans cron. Currently the implementation is very basic - 1px gif/xhr requests were causing Drupal to be bootstrapped twice each request, and at one point there was a proposal to do ob_flush() during a shutdown function but this didn't take on, so we ended up just running cron inline instead, which is sucky but I argued for that in the end.

With the queue runner, it'd be more a case of setting $_SESSION['run_queues'] after a form submit, check that on the next page, if it's set, add the 1px gif or whatever to that page, which hits /queue_run with a token. This would only ever be triggered by form submissions so it'd not have the page caching issues of cron runs.

Things it could be useful for:

- field deletion
- mass deletes of other stuff
- operations that trigger menu rebuilds or similar expensive operations, that don't necessarily have to happen inline with the request - just very shortly afterwards.
- indexing nodes in the core search module immediately after posting instead of waiting for cron.

Comments

Issue tags:+Performance

We don't need to use 1px gifs, between these two we can issue http requests from within the request out to the Drupal site then bail without checking the result, which takes around 1ms.

https://github.com/biznickman/PHP-Async

http://drupal.org/project/httprl

So I've started having a think about this, but for the async in Guzzle we'll need to add the Async plugin to core (http://guzzlephp.org/guide/plugins.html#async-plugin) - will that be a problem, and is too late in the cycle to get this in before feature freeze?

Was also thinking this queue runner would need to be able to be disabled (in the performance section or elsewhere?) as some use cases may want to prevent anything running "randomly"?

For implementation, I was thinking of extending hook_cron_queue_info() to add a flag for whether the queue specified should be executed by this queue runner. There is hook_queue_info which has been exposed by the contrib Queue UI module (http://drupal.org/project/queue_ui) but I guess getting that into core is pushing it?

I don't think that is a problem at all, the issue linked above in fact already adds that plugin.

Ok great, so for working on it I'll apply the patch from #1599622: Run poormanscron via a non-blocking HTTP request and we can make that a dependency.

I think I was editing my post as you replied, do you think identifying the queues to be run in queue runner should be configured in hook_cron_queue_info()?

One further thought on this, why would we only want to spawn these queue runner processes from a form submit? Looking at how the poormanscron will work (and works in D7) we'll be spawning a request to /cron/%process every time we reach the cron processing threshold, is there any reason why we wouldn't want to do the same for queues?