| Project: | Drupal core |
| Version: | 8.x-dev |
| Component: | cron system |
| Category: | feature request |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | needs review |
Issue Summary
Automatic cron is invoked during a page request, and for as long as cron is running, the page does not get sent to the browser. This can cause a significant delay in the page being rendered. This can be seen by putting a call to sleep(25) in hook_cron.
Also, if an error is encountered during cron invocation, it can output error text at the bottom of the rendered page. This can be seen by simply throwing an exception in hook_cron.
Both of these issues can be addressed by running cron in a separate process, forked from the process servicing the request. Not all environments are configured to make this possible; the Process Control extension must be installed in order for php to make use of fork.
For environments that do not support forking, we can gracefully fall back to the current behavior.
See the attached image for an example of an error message being appended to the display.
| Attachment | Size | Status | Test result | Operations |
|---|---|---|---|---|
| ExceptionThrownInCron.png | 28.68 KB | Ignored | None | None |
Comments
#1
This patch uses pcntl_fork to accomplish running cron in a separate process. If the pcntl_fork function is not available, drupal_cron_run is executed in the same process. This ensures a basic cron functionality is always available.
The test had to be modified slightly to grab the value of the 'cron_test' variable directly from the database because the value is updated in the child process and the value has already been cached in the parent process.
Using this patch, the requested page returns immediately, and any errors encountered during cron execution have no way of affecting the rendered page.
#2
This is nifty and superior to doing cron as part of the same page request.
This would be a great solution for Drupal Gardens as we don't have to setup 25,000 crontabs. :P
#3
I think the error message being appended to the display is its own bug that needs a more general solution. I created one here: #963206: Errors thrown during automated cron result in an extra error page appended at the bottom of the normal one
As for this patch, one big question... Do you know why the PCNTL manual (http://www.php.net/manual/en/intro.pcntl.php) says this?:
That is very vague, but certainly sounds ominous :)
***
I remember with some fondness our attempt around a year ago to do this via the "Connection: close" method:
http://drupal.org/node/566494#comment-2115892
That did not work in every server configuration, but it had the advantage of working in many common ones (for example, it worked just fine on my laptop's default Apache installation). However, we did not test it extensively enough to know if had any undesired side effects.
#4
The comments on
pcntl_fork()shed a little more light on that ominous message:There are a few suggested workarounds for MySQL, but it doesn't seem to be as cut-and-dry as merely checking to see if
pcntl_fork()exists (this patch does look like it's attempting to handle that, though).#5
That's a feature request, bumping to D8. We cannot change this so late in the cycle, and this only affect a page once in a blue moon.
#6
A few notes regarding this, fyi as I was experimenting with pcntl_fork in cron as well:
pcntl_fork runs only for CGI or CLI:
http://php.net/manual/en/function.pcntl-fork.php#49949
although this is handled in the patch by calling function_exists, the patch won't have the desired effect in a lot of cases.
I'm not sure, if it's enough to close the database within the child process only and leaving it open in the parent process. If the parent want's to use the database afterwards and the connection was closed by the child... couldn't this lead to race conditions which could be avoided by closing the connection just right before forking and therefore force parent and child to open a new database connection? (That's how I'm handling it)
#7
In my experience forking can be dangerous or at least cause side effects (expected or unexecpted). Nevertheless, when using fork, there's always a lot of things to consider, in order to remedy these side effects, and you might end up with much more bug-fixing than you bargained for. Besides that, there's also the platform issue, not only *nix/windows but also cgi/cli only as already mentioned.
Ultimate Cron implements a background process driven poormans cron, by issuing a new http request without waiting for the response, thereby detaching it from the users request.
I would like to see something like this in Drupal 8.
#8
Cross posting #1189464: Add a 'poor mans queue runner' to core.
Not sure what ultimate cron uses, but the approach sounds similar to
https://github.com/biznickman/PHP-Async
http://drupal.org/project/httprl
linked from that issue too. Given the constraints of having to trigger cron from http requests in the first place (i.e. not from cron or jenkins or anywhere like that), then the non-blocking http request seems like a good way to handle this, and is not something I remember being discussed in the original issue.
#9
Ultimate Cron uses the same technique (through Background Brocess which is somewhat similar to httprl). I've heard of ideas using a 1px gif for this, and it doesn't sit well with me.
Another technique I've considered for poormans cron, is doing a combination of:
* Connection: Close
* a specified Content-Length
* user_ignore_abort(TRUE)
* set_time_limit(very-high)
in order for the client to detach and for the cron stuff to commence.
This could save an extra http request while still not affecting the users.
#10
Both the 1px gif and Connection: Close with specified content length in one of the original poormanscron core issues and rejected. Core was originally doing 1px gif but that caused an additional http request and full bootstrap on every single request to Drupal including cached ones so was removed in favour of what we have now (which makes the request that runs cron very slow but doesn't mess everything else up). The content length stuff sounded great but I remember David Rothstein had trouble getting it actually working reliably.