We need a way to classify (taxonomy?) a site and then execute some shell tasks.

Right now we have some developer/themer Aegir hosted sites under git revision control. We need a way to automate their git push so the changes reach the Aegir hosted sites. Some of these need to follow the git branch when getting cloned.

Based on the experiment below we think this needs a hosting module producing task nodes based on shell tasks available. Is this a possible solution? Producing task nodes every 5 minutes? How could we accomplish this?

The small example below generates a cronjob task list based on taxonomy terms added to the site nodes. For this example to work the taxonomy module must be enabled within the hostmaster site, the array keys from $tasks must match a taxonomy term. This has some limitations ...
- using hosting_cron_get_sites which has nothing to do with these tasks (used as a convinience now)
- generating crontab items is not aware of new or deleted sites.
- not using hosting cron

$tasks = array(
  'git pull 5m' => array(
    'description' => 'Run a git pull every 5 minutes',
    'command' => 'cd %publish_path%/sites/%hosting_name%; git pull',
    'interval' => '*/5 * * * *',
    'args' => array('publish_path', 'hosting_name'),
  ),
  //'git follow master branch' => tbd
  //'git follow hosting_name branch' => tbd
);

$sites = hosting_cron_get_sites($limit = 500);

foreach ($sites as $s => $site) {
  $platform = node_load($site->platform);

  $vars = array(
    '%publish_path%' => $platform->publish_path,
    '%hosting_name%' => $site->hosting_name,
  );
  //print_r($platform);
  //print_r($site);
  foreach ($site->taxonomy as $term) {
    if (isset($tasks[$term->name])) {
      $task = $tasks[$term->name];
      $command = $task['command'];
      $command = str_replace(array_keys($vars), array_values($vars), $command);
      echo $task['interval'] . " " . $command . "\n";
    }
  }
}

The output looks like

drush @hostmaster php-script taxonomy_runner.php 
*/5 * * * * cd /var/aegir/platforms/drupal-6.22/sites/test.example.com; git pull

Comments

clemens.tolboom’s picture

It maybe better to use server tasks then add these to provision?

- Add a task within profiles/hostmaster/modules/hosting/site/hosting_site.module
- extend provision drush commands
?

helmo’s picture

subscribe

helmo’s picture

Status: Active » Needs review
StatusFileSize
new1.41 KB
new1.12 KB

A task as suggested in #1 would be more direct and predictable for a developer.
Also pulling many sites every 5 minutes does not scale very well.

I've cooked up two little patches to create a Git pull task in the hostmaster and let provision do the work.

This does open up the question which other git (or other vcs) related tasks we could support.
Not sure if the versioncontrol_api could help us here but who knows.

* setting up a git repo for a new site
* Switch to a different branch

clemens.tolboom’s picture

Status: Needs review » Needs work
+    'description' => t('Update sites dir from git'), ¶

Could be more informative.

"Runs a git pull on the current site when applicable."

+  $tasks['site']['git_pull'] = array(
...
+    'title' => t('Pull git'), ¶
...
+    'description' => t('Update sites dir from git'), ¶
...
+  );

whitespaces

+++ platform/git_pull.provision.inc
@@ -0,0 +1,27 @@
+  if (drush_shell_cd_and_exec($site_path, 'git pull')) {

This could use some more code ie. a dir exists for '.git'

This way our users get more informatie about the git status of the site.

+++ platform/git_pull.provision.inc
@@ -0,0 +1,27 @@
+    drush_log(dt('Pulled from git on !path', array('!path' => $site_path)), 'ok');
+   } ¶
+  else { ¶

contains whitespaces

Powered by Dreditor.

helmo’s picture

Status: Needs work » Needs review
StatusFileSize
new1.15 KB
new1.7 KB

I fixed the whitespace errors, added an is_dir check for the .git directory and changed the description string.

clemens.tolboom’s picture

+++ platform/git_pull.provision.inc
@@ -0,0 +1,31 @@
+  if (!is_dir($site_path . '/.git')) {
+    return drush_set_error('DRUSH_PROVISION_GIT_PULL_FAILED', dt("The site does not seem to ge a valid Git clone"));
+  }

Typo: "does not seem to ge a"

The error text should have the site_path in it. This way an Aegir user could ask easier for a fix?

What about: "The site has no .git repository located in !site" ?

I have no way of testing this atm :-(

anarcat’s picture

Status: Needs review » Closed (won't fix)

I think this belongs in contrib, sorry!

helmo’s picture

I can understand that.

I don't mind starting a sandbox for this, but would really hate it if I were to duplicate the work of someone else.

So, am I crazy to wanna store this site specific stuff in git and pull it using a taks from the hostmaster site?

helmo’s picture

Just for future reference, I've opened a sandbox for this http://drupal.org/sandbox/helmo/1284702