Finish up command bootstrap levels and intelligent command listing

I'd love Adrian's patch review on this. he is the one who thought that some commands need control the bootstrap level (presumably installer and updater)

I'll review this on the plane tomorrow, but i need to review it in context of the .. err context patch.

Which needs to go in first ?

Personally, in my commands, i need to be able to delay bootstrapping, till the configurations are created correctly.
This involves moving the DRUSH_URI definition and drush_set_environment code (which correctly sets the headers), to a later point in execution (if bootstrap is not set).

ie: when you do 'drush.php install somesite.com --root=/some/drupal/path' , it needs to not bootstrap, until after i create the drupal config and the mysql database, and then bootstrap the additional levels.

Sounds like Install/Update needs are met, so feel free to proceed with commit of this and with the error context patch. I didn't get to test it but I'm sure we can iron out problems afterwards.

I should note that this commit breaks provision, I don't exactly understand why. From what I understand, now drush bootstraps as much as it can now, which will break provision since it sometimes needs to do some things (like creating the settings.php) *before* bootstrapping. In short it needs much better control over the bootstrap workflow than what's available here.

In fact, I think there's been a conceptual confusion about the 'bootstrap' flag in drush command declarations. This patch assumes it's "minimal bootstrap level necessary to run the command" whereas I think it should mean "do not bootstrap beyond that level". Maybe we need both?

Not sure where to go from here now. I'll test my luck at putting this back to 'needs work' since this was done for provision after all...

This was committed, and broke all the provision commands. http://drupal.org/cvs?commit=182490

We need the bootstrap to set the maximum level to boot to, and not the minimum required level to boot to.

http://drupal.org/node/400034

I also think it's wrong that all the application logic is in the shutdown code AND the documentation is also incorrect.

I am reverting this patch , and will look at it more closely on monday. http://drupal.org/cvs?commit=182904

I had to go back to 1.44 in order to fix this on my build.
1.48 killed my queue dispatcher --- 1.47 or #387214 killed my provision
Everythings happy again @ 1.44
^_^

Would be great if Owen and Adrian could get this sorted quickly. We really need to release Drush 2 ASAP. Our current situation is confusing for many potential drush users. I removed all releases from the project page except HEAD because I have no intention of supporting them.

Now that context is in, I'm busy working on a new version of this patch.

It's essentially modeled after how Drupal does it's bootstrapping, with the addition of some drush specific levels.

/**
 * Sequential Drush bootstrapping levels
 */
define('DRUSH_BOOTSTRAP_NONE', 0);
define('DRUSH_BOOTSTRAP_PLATFORM' 1);
define('DRUSH_BOOTSTRAP_SITE', 2);
define('DRUSH_BOOTSTRAP_DRUPAL_CONFIGURATION', 3);
define('DRUSH_BOOTSTRAP_DRUPAL_DATABASE', 4);
define('DRUSH_BOOTSTRAP_DRUPAL_FULL', 5);

This will allow any command to specify the level to bootstrap to, and allow the help command to sequentially boot to new levels.
This will also allow us to cleanly tie the error handling into the current bootstrap level, and reliably detect more errors (such as not having a proper drupal root, not having a proper site directory, the 'site not found' error)

I'm still trying to determine what the default bootstrap level is going to be.

So here's a weird little issue that I see cropping up.

So essentially when dispatching the command, it will start at bootstrap level 0, find the relevant commandfiles, boot to bootstrap level 1, do an additive find, etc.

However, when it finds a command with a lower bootstrap level, let's say for example 'status' (this is not an actual case, just a pertinent example), it should cease trying to find a command with a higher bootstrap level, and dispatch the command.

However, if you then add a command called 'status modules', which requires a higher bootstrap level than the 'status' command which is found and evaluated first, it will never reach that command.

I can't think of a way around this, and I think it's actually the way it's supposed to work. So it should be documented at least.

My first inclination is that we can just document this somewhere and accept the limitation.

I've been doing some more work on this patch, and i've found a pretty solid approach.

What i've been doing is taking the drush_bootstrap as it is now, and moving the code into their own functions.
This means the code that is used in drush_bootstrap to set the constants is now callable up to a specific level.

The levels are :

/**
 * @name Drush bootstrap phases
 * @{
 * Sequential Drush bootstrapping phases.
 */

/**
 * Only bootstrap Drush, without any Drupal specific code.
 *
 * Any code that operates on the Drush installation, and not specifically
 * any Drupal directory, should bootstrap to this level.
 */
define('DRUSH_BOOTSTRAP_DRUSH', 0);

/**
 * Set up and test for a valid drupal root, either through the -r/--root options,
 * or evaluated based on the current working directory.
 *
 * Any code that interacts with an entire Drupal installation, and not a specific
 * site on the Drupal installation should use this bootstrap phase.
 */
define('DRUSH_BOOTSTRAP_DRUPAL_ROOT' 1);

/**
 * Set up a Drupal site directory and the correct environment variables to
 * allow Drupal to find the configuration file.
 *
 * If no site is specified with the -u / --uri options, Drush will assume the
 * site is 'default', which mimics Drupal's behaviour.
 *
 * If you want to avoid this behaviour, it is recommended that you use the
 * DRUSH_BOOTSTRAP_DRUPAL_ROOT bootstrap phase instead.
 *
 * Any code that needs to modify or interact with a specific Drupal site's
 * settings.php file should bootstrap to this phase.
 */
define('DRUSH_BOOTSTRAP_DRUPAL_SITE', 2);

/**
 * Load the settings from the Drupal sites directory.
 *
 * This phase is analagous to the DRUPAL_BOOTSTRAP_CONFIGURATION bootstrap phase in Drupal
 * itself, and this is also the first step where Drupal specific code is included.
 *
 * This phase is commonly used for code that interacts with the Drupal install API,
 * as both install.php and update.php start at this phase.
 */
define('DRUSH_BOOTSTRAP_DRUPAL_CONFIGURATION', 3);

/**
 * Connect to the Drupal database using the database credentials loaded
 * during the previous bootstrap phase.
 *
 * This phase is analogous to the DRUPAL_BOOTSTRAP_DATABASE bootstrap phase in
 * Drupal.
 *
 * Any code that needs to interact with the Drupal database API needs to
 * be bootstrapped to at least this phase.
 */
define('DRUSH_BOOTSTRAP_DRUPAL_DATABASE', 4);

/**
 * Fully initialize Drupal.
 *
 * This is the default bootstrap phase all commands will try to reach,
 * unless otherwise specified.
 * This is analogous to the DRUPAL_BOOTSTRAP_FULL bootstrap phase in
 * Drupal.
 *
 * Any code that interacts with the general Drupal API should be
 * bootstrapped to this phase.
 */
define('DRUSH_BOOTSTRAP_DRUPAL_FULL', 5);

/**
 * @} End of Drush bootstrap phases.
 */

Any command that needs to bootstrap to a lower phase than full, will be able to further bootstrap at it's own convenience using drush_bootstrap(DRUSH_BOOTSTRAP_WHATEVER).

When drush is initialized now, it will sequentially try to bootstrap to the drupal_full level, and test whether there is a command that
should be executed at the current bootstrap level, if there is it will hand over control of bootstrapping to the command using drush_dispatch.

Here's the patch.

I tested this with hosting/ provision (where it's mostly working .. but i think the issues that remain are on my side), and on drush_extras 5.x (which i didn't modify at all).

I added another bootstrap level to require a valid login to the site.

here's a new version of the patch, with more logging (only visible with --verbose)

I reviewed this, and am comfortable with committing this once the dl command is working properly. Nice work, Adrian.

Here's my latest version of the patch, fully tested with hosting and provision (the companion patches are here : http://drupal.org/node/411822)

I'd like to get this committed ASAP, so i can have a solid base to make smaller patches on. I'm maintaining over 100k's worth of patches across 3 projects for this stuff.

Any further issues we find we can create new, simpler issues for.

I fixed up dl command and committed this. Thanks Adrian.

A typo went in at some point, here's a trivial fix.

Sorry, wrong issue.