Cron Multi-Threaded

News

Core patch inspired by Cron MT committed to D7 #434350: Add Database::closeConnection() to explicitly close a connection.

Overview

Cron MT is a pseudo multi-threaded implementation of cron. This module utilizes the PCNTL extension to run cron hooks in parallel by forking the parent process that Drupal is running in. By invoking the hook_cron() implementations in separate processes, the system can take advantage of it's multiple CPUs as opposed to calling the hooks one at a time in a single build-up, tear-down process. In addition to increasing efficiency, Cron MT adds fault tolerance as the other cron processes are able to continue in the event that one hangs. Running Cron MT in verbose mode allows for easy identification of the offending function by logging the start and stop times of each hook_cron() implementation via watchdog.

Maintainer

Cron Multi-Threaded is developed and maintained by Chris Pliakas.

Requirements

Cron MT requires the PHP PCNTL and POSIX extensions. The PCNTL extension currently works only on Unix and Unix-like platforms. Additionally, a very small core patch bundled with the module must be applied to the Drupal installation. Cron MT is designed to run via command line interface and will not work otherwise.

Installation

• Download and extract the files like any other standard contributed module.
• Apply the patch corresponding to the version of Drupal you are using. If there isn't an exact match, use an earlier patch closest to your version of core. For example, the DRUPAL-6-6.patch file is applicable to core versions 6.6 through 6.9.
• Make sure both the bin/cron_mt_pcntl_check.php and bin/cron_mt.php files are executable.
• If running Cron MT via the the bin/cron_mt.php script, copy the etc/cron_mt.example.conf file so it is named etc/cron_mt.conf and edit the path directive to reflect your Drupal installation's root.
• Navigate to Administer > Site building > Modules and enable Cron MT.
• Visit Administer > Site configuration > Cron Multi-Threaded to configure how many processes are allowed to run concurrently.
• Configure cron to call your preferred method of Cron MT, for example via Drush or the bin/cron_mt.php script. Note that both methods should be called as standalone scripts and not through wget, lynx, or curl. If you choose to use the bin/cron_mt.php script, the cron entry should be similar to the following line:
  0 * * * * /path/to/module/bin/cron_mt.php

Why The Patch Is Required

I believe that patching Drupal core should be done as a last resort when no other options are available. A patch is required to run Cron MT because Durpal does not allow us to re-establish database connections as they are stored as a static variables within the db_set_active() function. Since Cron MT forks the parent process multiple times during it's run, the database connection is automatically closed when the the first child process exits. Because the processes share the same resource ID for the database connection, it is closed in the parent process and subsequent child processes as well. In order to solve this problem, the patch adds a second parameter to the db_set_active() function which allows us to re-establish a database connection by forcing the static variable to be overwritten even if it already exists.

Downloads