Make hosting (specifically hosting-dispatch) work with Drush 5

Agreed, moving to the Drush queue for more visibility, could you advise as to what the replacement for the drush_backend_fork function is? And then could you move it back into our Hostmaster queue please?

Thanks!

The changes quoted above were part of a series of very large changes to backend invoke to support Windows (spaces in filenames) and concurrent execution of multiple commands (drush_backend_invoke_concurrent).

I suggest that the first thing that you do is change all of your code that uses drush_backend_fork to instead call drush_process_invoke. This will give you spaces-in-filenames compatibility, but you will lose the "fork" capability, as the command will not return control to you until it has completed execution.

Once you get to that point, I suggest that we add a new item to $backend_options, 'fork' => TRUE. So, your call to drush_invoke_process would look like this:

 $values = drush_invoke_process($site, $command, $args, $options, array('fork' => TRUE));

n.b. previously, there were $hostname and $username parameters to the backend functions; now you're going to want to pass $site = array('remote-host' => $hostname, 'remote-user' => $username) instead.

Then we can re-add the missing fork capability to Drush. If we can work through this quickly enough, there is a plan to release Drush-5RC1 and Drush-4.6 sometime this week. It would be nifty to get this patch in both releases, as then you could have one call that worked on both versions of Drush. Maybe that won't turn out to be feasible, but it's good to have goals. :)

I will monitor this issue and help out with the Drush tasks.

Created a Drush issue for this: #1433556: Aegir wants the drush_backend_fork capability restored..

I think what greg meant here is that drush_invoke_process() needs to be modified so that it *can* fork, which is not currently the case. :)

Let's keep this opened. Drush 5 support is an explicit goal of the 2.x release and we should help Drush here, not just wait. :)

Yes, #7 is correct. If you could post the patch from #6, then perhaps others can participate in making Drush and Aegir work together correctly once I have a patch posted on the Drush issue.

Instead of:

drush_backend_fork('@self', "hosting-" . $queue, array(), array('items' => $count));

Try:

drush_invoke_process('@self', "hosting-" . $queue, array(), array(), array('items' => $count), array('fork' => TRUE));

This presumes you would like to run: drush hosting-$queue --items=$count &.

Instead of:

drush_invoke_process('@self', "hosting-task", array($task->nid));

Try:

drush_invoke_process('@self', "hosting-task", array($task->nid), array(), array('fork' => TRUE));

This presumes you would like to run: drush hosting-task $task->nid &.

Please forgive the blending of bash and php syntax above. Test with #1433556: Aegir wants the drush_backend_fork capability restored..

Yes, drush_backend_invoke_args was removed in favor of drush_invoke_process. Just post the old code here if you have any question about how to map the args from one function to the new one.

#15 looks right to me.

drush_invoke_process() replaces most of the old methods for calling drush commands from drush commands. Only drush_invoke() and drush_dispatch() remain, but those were used only in special cases. Most drush_invoke_* and drush_backend_* calls should become drush_invoke_process.

drush_backend_output works the same in Drush-5 as it did in Drush-4. drush_backend_invoke should be replaced with drush_invoke_process, but be careful with this one, as drush_backend_invoke allowed you to put the command name, options and args into a single string that was exploded on space. drush_invoke_process does not allow this.

I just wanted to say that you guys are doing amazing work and I'm really happy to see people undertake this important task! Keep it up, can't wait to see 2.x actually "compile" with Drush 5. :)

I started a page here: http://www.drush.org/drush-4-to-drush-5

When I run "$ drush update" I now see a notice telling me that I should upgrade my version of drush to 5.1 as it is the new "recommended" version. Considering that this instantly breaks Aegir and prevents it from running cron tasks with no visible notice or warning displayed to the user, the recommendation strikes me as kind of silly, but that's the way things are.

This issue is already flagged as critical, but I just wanted to re-iterate that the situation kind of sucks atm. I wish I could offer a patch, but I don't think I'll be able to for this one.

I agree with thedavidmeister.

I'd also like to say that every time I try my luck at installing Aegir (i.e. "let's see if I can get it to work this time"), it fails miserably.

The testing and release process of Aegir needs to be improved a great deal.

Also, because of the heavy dependency on Drush, the Aegir team should strive to publish new releases to be compatible with each new Drush release. For example, run automated testing for each new git commit in the Drush repository.

We are trying to fix this for 2.x

In the meantime, it looks like it is totally ok to put drush5 into a separate folder in your user's directory and alias it in your bashrc or bash_profile. That way you can at least get the latest features through the cli so even if Aegir doesn't get the benefit of caching, etc..

Something as simple as this (assuming you extracted drush5 into ~/drush5):

alias drush5='~/drush5/drush'

Hello,

I inadvertently upgraded a working drush 4.5 to 5.2 (by running pecl upgrade - and there was no opportunity to say "no"). Now aegir sees my dush version as 5.2 and debian as 4.5. Con no longer runs, and I am unable, believe it or not, to downgrade drush to 4.5. Are there any downgrade guides for aegir. Don't use version control yet - will have to start that now, I see.

I am contemplating modifying the aegir hostmaster upgrade.sh script and trying to upgrade from 6.19 to 6.19 and hopefully that process will give me an opportunity to choose drush 4.5 during that process. Any ideas on if this should work and if it might destroy aegir hosting platform. Unfortunately this is on a production server. I am an aegir / drush relative newbie.

Otherwise, do you recommend me applying the patch in comment # 22, above, and that should allow drush 5.2 to be properly functional, which might then allow me to downgrade it to 4.5.

OR

Any easy way to just downgrade drush (in aegir) to 4.5 without touching the aegir and hostmaster packages?

Thanks for any advice and help here.

Eddie

We would recommend restoring the previous version of Drush. If it came from packages, then removing the newer version of drush would be a good first step, then running something like:

sudo apt-get install --reinstall drush

Should re-install the old version of Drush.

In the future please raise support issues like this in new issues and don't hijack existing ones!

It seems that this patch was merged in, so I am not sure why this is still marked as opened. This just works now.

@tstoeckler & @greg.1.anderson - it would be useful if you could take a look at #1612042: hosting-dispatch throws loads of garbage when running in drush 5, where I see a lot of garbage coming out of hosting-dispatch, even without the -v or -d flags...

I am getting this error after installing aegir 1.9
I got a message in my logs
ran drush '@hostmaster' hosting-dispatch --debug

PHP Fatal error: Call to undefined function drush_backend_fork() in /var/aegir/hostmaster-6.x-1.9/profiles/hostmaster/modules/hosting/dispatch.hosting.inc on line 44

Fatal error: Call to undefined function drush_backend_fork() in /var/aegir/hostmaster-6.x-1.9/profiles/hostmaster/modules/hosting/dispatch.hosting.inc on line 44
Drush command terminated abnormally due to an unrecoverable error. [error]
Error: Call to undefined function drush_backend_fork() in
/var/aegir/hostmaster-6.x-1.9/profiles/hostmaster/modules/hosting/dispatch.hosting.inc,
line 44 [0.76 sec, 34.3 MB]

patch does not apply cleanly:

aegir@:~/hostmaster-6.x-1.9/profiles/hostmaster$ patch -p1 < 14*
patching file modules/hosting/dispatch.hosting.inc
patching file modules/hosting/hosting.api.php
Hunk #1 succeeded at 298 (offset 35 lines).
Hunk #2 succeeded at 316 (offset 35 lines).
patching file modules/hosting/task.hosting.inc
Hunk #1 succeeded at 101 (offset -3 lines).
Hunk #2 succeeded at 112 (offset -3 lines).
patching file modules/hosting/task/hosting_task.module
Hunk #1 FAILED at 498.
1 out of 1 hunk FAILED -- saving rejects to file modules/hosting/task/hosting_task.module.rej
aegir@:~/hostmaster-6.x-1.9/profiles/hostmaster$ modules/hosting/task/hosting_task.module.rej
-su: modules/hosting/task/hosting_task.module.rej: Permission denied

aegir@devmac:~/hostmaster-6.x-1.9/profiles/hostmaster$ more modules/hosting/task/hosting_task.module.rej
--- modules/hosting/task/hosting_task.module
+++ modules/hosting/task/hosting_task.module
@@ -498,7 +498,7 @@
drush_log(dt("Running tasks queue"));
$tasks = _hosting_get_new_tasks($count);
foreach ($tasks as $task) {
- drush_backend_fork("hosting-task", array($task->nid));
+ drush_invoke_process('@self', "hosting-task", array($task->nid), array(), a
rray('fork' => TRUE));
}
}

other files that call drupal_backend_fork
hosting/hosting.queues.inc:174: drush_backend_fork("hosting-task", array($task->nid));

Edit: Solved by downgrading to 4.6

Yeah, so at the moment the 1.x branch is compatible with Drush 4, and the 2.x branch is compatible with Drush 5, we are trying to get a stable release for 2.x done this year.