Aegir wants the drush_backend_fork capability restored.

Now getting fatal: Undefined function drush_backend_invoke_args()
It should be possible to replace that with drush_invoke_process(), too, though, right?

EDIT: Sorry, wrong issue.

Hmm.. this isn't working for me.
I also tried bringing the exact Drush 4 behavior back á la:

$exec_cmd = '(' . $exec_cmd . ' &) > /dev/null';
drush_op_system($exec_cmd);
return;

(Just like in http://api.drush.org/api/function/drush_backend_fork/4.x)
but that didn't work either.

That said, is there any more reliabe way to test this? Currently I'm just using Aegir with the patch over at #1433406: Make hosting (specifically hosting-dispatch) work with Drush 5, but, as before, what happens is that the tests run through, but the Task isn't marked as comleted, i.e. the spinner keeps spinning. That's probably not the most reliable way to test this, though.

The patch in #1 works fine for me running following code using the command "drush scr fork5.php". With the patch we get 10 simultaneous processes as expected.

$fork = drush_get_option('forktest');
$pid = getmypid();
$ourFileName = dirname(__FILE__) . "/fork-$fork-pid-$pid";
$ourFileHandle = fopen($ourFileName, 'w') or die("can't open file");
fclose($ourFileHandle);
$fork++;
if ($fork < 10) {
  drush_invoke_process('@none', 'scr', array(dirname(__FILE__) . '/fork5.php'), array('forktest' => $fork), array('fork' => TRUE));
}

I've tested this on both Centos 5.2 and Mac OS X Lion... it works as expected on both OSes. Interesting on Centos:

$drush version &
[1] 3430
drush version 5.0-dev

[1]+  Done                    ~/dev/drush5/drush version

On MacOS X it exhibits the behaviour Greg described above - stopping until the command fg is used.

... and I can confirm that #1437762: Backend drush command reparsed by escapeshellcmd() which causes issues if you're using a drush.ini file is fixed in drush5

Hi Greg,

This is not an issue for drush but occurs for all php commands run from the command line.

If you create a php script test.php as

  print 1;

Try running php test.php & it hangs too on Mac OS X

Run it using thisphp test.php < /dev/null & and it does not

Same for drush version < /dev/null &

Afaik there is one outstanding issue for the patch in #1. When running the test described in #5 under drush5 lots of output is written to stdout which is not the case with drush4.

Is this a desired effect of the changes to backend.inc in drush5?

This patch only prints backend output for forked processes if the verbose or debug options are set.

I think this patch is good to go.

@alexpott: Is there a specific way to test your patch?

Also, I cannot reproduce #8. It hangs for me either way...

I'm on Debian 6.
I tried #8 again, and it works for me if you redirect the output to /dev/null. Redirecting the input doesn't work. Hmm..

Greg #12 - how right you are that was meant to be a return not an exit.

Anyhow I think the patch attached is a "cleaner" solution using a helper function - drush_backend_check_output() - to determine if the backend process should print to stdout.

Working some tests...

Then I must press [Enter] to see the done message

Ahh, I didn't know that was the wanted behavior. (I told you I was a noob :)...)

Yes, I can reproduce exactly what you describe.

Patch now with added test goodness...

The test invokes a forked process that creates a file after a second in PHPUnit's sandbox folder. Meanwhile the test checks for the existence of this file then waits for 2 seconds and checks again.

No problem... didn't know about UNISH_TMP :)

Changed to use UNISH_SANDBOX as that seems more consistent will other tests that are creating files eg. testArchiveDump

Updating issue status...

We need to document the 'fork' backend option. I'd like to state why one would use it, in addition to stating what it does.

Good point. Patch now includes documentation...

How does fork help with 'Can be used to write commands that restart themselves before they run out of memory.'. We already do that with regular backend calls. See updatedb and migrate-import in migrate module. Furthermore, please state what happens with integrating results from forked calls. Return values are ignored?

CNW for docs

I tried to get this to work, but am getting a weird error:
When I verify a site, it re-saves the corresponding alias with the wrong path to the site. Namely it always uses the main hostmaster platform as a site path (i.e. /var/aegir/hostmaster-1.6/sites/foo instead of /var/aegir/platforms/bar/sites/foo).

So my question regarding drush_invoke_process is about the provision-save command. It takes an alias as an argument, so I don't know what to put in the $alias argument for drush_invoke_process.

// Is it this...
drush_invoke_process($alias, 'provision-save', array($alias), ...);

// ...or is it this?
drush_invoke_process('@self', 'provision-save', array($alias), ...);

I currently use '@self', because I thought it shouldn't really matter, but that does not seem to work. It may, of course, very well be, that my problem is caused by something else, but I thought I'd ask.

Thanks, I'll try that.
Yes, I first wanted to post this issue in the Hostmaster queue, but since this is specific to the Drush 5 port...
I'll still try to get this to work in the next few days, but for what its worth, the tasks now actually complete fully, so it seems the 'forking' part seems to work.
/me shuts up now to let the big guys fix this :)

Patch attached for another go at documentation. My primary use case for this functionality is http://drupal.org/project/drushd. This needs to be able to fork a process in a non-blocking way to allow daemon threads to refork safely.

The new documentation from the patch:

/**
 *      'fork'
 *        Overrides the backend invoke process to run non blocking commands in 
 *        the background. Forks a new process by adding a '&' at the end of the 
 *        command. The calling process does not receive any output from the child 
 *        process. The fork option is used to spawn a process that outlives its
 *        parent.
 */

Looks fine. Perhaps use http://php.net/manual/en/function.usleep.php in order to wait less.