Cron support has already been partially implemented in the CiviCRM drush module, however, we need a way in Aegir to call our cron at some interval (for example, the civimail cron should run every 15 minutes or so).

At first, we might want to try some "one size fits all" solution, and eventually have a way to customize the delay betwen cron runs (for bigger sites).

Without this functionality, people must configure the various CiviCRM crons manually.

Comments

joe.murray’s picture

Just wondering if there is some overlap with http://civicrm.org/civicrm/contribute/transact?reset=1&id=24

Could you clarify what the partial cron support in the CiviCRM drush provides?

If you could dump the status of your work in terms of desired approach and roadblocks if any that would be great. I take it that we would need to have a means to provide the user, password and sitekey PUT params to the Aegir cron jobs, and that we want to support all of the scripts in civicrm/bin, including those that need to be renamed to remove the .txt extension.

bgm’s picture

There might be overlap with the MiH. I'm not sure which approach they will take, and I would be curious. One problem with automating cron runs is scheduling them at the right moment. Probably once the MiH is done, we can piggy-back on their work (assuming the scheduling will be more flexible than what I am planning to do, which will be a bit bare metal, one size fits all).

This is what happens at the moment:

* at installation, provision_civicrm automatically creates a civicron user and role, and grants it some Drupal permissions.
* the password for the civicron user and the site key are saved in the drushrc file

When (for example) "drush civicrm-civimail" is run, it fetches the username, key and password from the drushrc file, and runs the task. (you could also pass the user/password/key arguments on the command line, but it is not necessary, since they are saved in the drushrc.php)

There are only two cron jobs supported right now, and not necessary well tested: civicrm-civimail and civicrm-member-records (UpdateMembershipRecords).

The next step in Aegir is to schedule the execution of those crons, say every 15 minutes for civimail, once per day for civicrm-member-records, etc.

jmcclelland’s picture

StatusFileSize
new4.25 KB

I'm uploading a patch to ensure the site key and civicron user are created on verify. I know it's a bit controversial because it will break external cron jobs. On the other hand, without it, sites moved to aegir will never have their cron jobs properly run because the civicron user won't be created.

I've added a hook_cron function to one of our custom modules on our Drupal sites that call the civicrm cron functions every time the site cron runs (once an hour by default), which seems fine to me.

I'm not sure if there's a better way to execute the cron jobs though - or how we would want to do it since not everyone will want the same cron jobs to run.

jamie

bgm’s picture

@jamie: thanks for the patch!

For the cron user: I would not reset the password on verify, only detect if the user exists, and if not, create it.

An interesting way to run crons (although not all are available) is using Drush. For example, the CiviMail cronjob, which can be very CPU-intensive, can be run this way:

nice drush @www.example.org civicrm-civimail

This way you can also set more tolerant PHP timeouts for the CLI, than for Apache (and "nice" the task, so that it doesn't kill too much the server).

(for now, I setup the cron manually, but eventually it would be nice to have an Aegir scheduler task for that)

sfyn’s picture

I am unsure of where to post this issue, I am trying to debug a civicrm mail issue on aegir and the platform is unfamiliar to me, so here goes.

The problem: A mailing scheduled for May 31st has still not started processing.

Steps taken:
1. I verified the site key was present in both drushrc.php and in civicrm.settings.php. It was.
2. I attempted to run drush civicrm-civimail - producing this error message:

Usage: $cd /your/civicrm/root; $php5 bin/index.php -u user -p password -s yoursite.org (or default)
Drush command terminated abnormally due to an unrecoverable error.                 [error]

3. Running this command with a --debug flag produced the html code for a civicrm page informing me that --debug is not a supported option.

So, um, help?

sfyn’s picture

I temporarily "solved" my problem by visiting:

http://drupalsite.url/sites/all/modules/civicrm/bin/civimail.cronjob.php?name=USERNAME&pass=PASSWORD&key=KEY

Where USERNAME was my username on drupalsite.url and PASSWORD was my password on that site, while KEY was the civicrm site key.

mrfelton’s picture

Status: Active » Needs review
StatusFileSize
new3.16 KB

Attached patch applies to the latest in git. It's essentially the patch in #3 reworked to not regenerate a password for the civicron user every time a site is verified, and instead, just check if the user exists and create it if not. I also changed it to use the provision_password() function to generate the password, which is a duplication of Drupal's user_password function.

bgm’s picture

Title: Cron support for civimail, email processing, update membership record » Cron support for civimail
Status: Needs review » Active

Applied the patch for provision_password() and merged the work from the master branch to 6.x-1.x branch

(Jamie's patch had already been committed, but on the master branch.. now switched everything to the 6.x-1.x branch).

However, the core issue here is how to schedule those crons automatically in Aegir.

bgm’s picture

Title: Cron support for civimail » Cron support for civimail - hosting_civicrm_cron

ergonlogic wrote a small module to handle the cron queue in civicrm >= 4.1: hosting_civicrm_cron.

If anyone has tested it, please post your feedback :)

jmcclelland’s picture

I think the hosting_civicrm_cron seems like the right approach.

I think it needs to be updated to call the unified cron job (rather than just the mailing cron job).

Also - looks like it calls via an http request - might be nice to have the option to call via a civicrm api call instead.

Woops... looks like both of these issues are already in the issue queue

For the record, I currently run cron via a custom module through that module's _cron hook. I think controlling this via hostmaster is much better.

joe.murray’s picture

Status: Active » Closed (duplicate)

Based on #10 I am closing this. Bam, pls reopen if appropriate.

  • bgm committed a7b9f29 on 6.x-2.x
    Issue #1097554: README.txt: removing from TODO about civimail/membership...

  • bgm committed 694c9d6 on drop_v1 authored by mrfelton
    - #1097554 by mrfelton, use provision_password() instead of ad-hoc...
  • bgm committed a7b9f29 on drop_v1
    Issue #1097554: README.txt: removing from TODO about civimail/membership...

  • bgm committed 694c9d6 on 6.x-2.x-20140828-cleanup authored by mrfelton
    - #1097554 by mrfelton, use provision_password() instead of ad-hoc...
  • bgm committed a7b9f29 on 6.x-2.x-20140828-cleanup
    Issue #1097554: README.txt: removing from TODO about civimail/membership...