Add support for commandfiles, alias files, and config files in DRUPAL_ROOT/sites/all/drush.

This is similar in concept to the "Grayside feature" (load drush config from the root of the git repository). I am in favor of this, but could we call the folder "drush" instead of ".drush"? I think it makes sense to hide this folder in $HOME, but not in /etc or DRUPAL_ROOT.

It is supported to put site alias files in drupal root so this makes sense. we really need to unify search paths for commandfiles, aliases, and drushrc files

Maybe have the additional Drush extensions in sites/all/drush? I get anxious about having a bunch of stuff in the Drupal root. Would also be cool if extensions could be installed via drush dl drush_shell too, but that's a separate matter.

+1 for sites/all/drush.

Having said that, we took a long time to convince people not to put drush or its commandfiles inside of a drupal site. I'm not eager to going backwards on this. sites/all/drush does not sound appetizing to me.

I agree that folks should be discouraged from keeping drush inside a Drupal site, but for projects where code is shared across a variety of environments, it may be necessary.

Since Drush is a utility, I think it makes sense semantically to put a drush directory in the drupal root rather than within sites/all, but sites/all has practical advantages like making core updates smoother and keeping all non-core pieces in one place.

If #5 is true (and it does seem compelling), then I think that DRUPAL_ROOT is not better than sites/all/drush in this respect, and ".drush" does not really fix this IMO. How about '../drush'? A lot of people put their DRUPAL_ROOT inside of a "project" for the site.

Rather not that the website make any assumptions about the structure above it. A number of projects do have a project directory there, but in another number of cases, that's the 'shared' space that a lot of different sites call home. Just look at an Aegir platform setup.

Referring to ../ is one of the first scary things that would cause nervousness in a security audit. Justified or not, it looks like a bad smell in the code.

Well, the reference to ".." only happens in Drush, not in the Drupal site, so security audits are not really applicable here.

I don't think it would be terrible to have a shared drush folder for a collection of Aegir sites, but you are correct, if that is your environment, then you wouldn't have a place to put one drush folder per site if we used "../drush". I think what this boils down to is that the sentiments of #9 are in opposition with #5. If we want the feature of one drush commandfile folder per site, then sites/all/drush is best, and if we don't -- well, I guess we can skip the feature. In Drush 5, we already load a drushrc.php file from DRUPAL_ROOT/drushrc.php during drush_bootstrap_drush. Inside this file, you can put in your own logic and call _drush_add_commandfiles($searchpath);, where $searchpath is any arbitrary array of paths where you would like to search for commandfiles in.

Perhaps we should just remove the _ from drush_add_commandfiles and fix this in documentation (examples/example.drushrc.php).

Well, with Aegir, i've got enough root anyway... it's just an example of an equally common architecture. I feel icky about upward leakage.
I don't actually mind {drupalroot}/.drush

But I think that it would be hard to get good consensus on everyones requirements, so I suggest adding an include path in drushrc - similar to the site_aliases search path array - would give folk who need it the appropriate control - or that _drush_add_commandfiles() as above works the same.
Thumbs up recommending to that approach - I can't see this bikeshed getting painted otherwise.

I'll also mention that #759906: Update core without moving core files out of the way. may be a factor here.

Here is a patch that "solves" this problem through documentation. Note that the technique isn't terribly clean; it works if placed in drushrc.php files at the Drupal root, but does not work for other drushrc.php locations (unless your Drupal site is selected via cwd instead of an alias or cli option...).

Maybe one of the compromises suggested above would be better.

Of the possible locations, I'm now leaning toward sites/all/drush. Sorry I have been waffling on this.

Yet I have a nagging feeling that we're asking for support requests. Commands that have to run before DRUSH_DRUPAL_ROOT phase will not be found. There are going to be a fair number of commandfiles that won't work when put inside of Drupal. For example, we can't auto-download extensions to this dir as suggested in #3.

If we proceed, the original patch adds code to the SITE bootstrap phase instead of ROOT. Needs docs changes as well.

Due to my recent changes to the Drush bootstrap (identify site earlier in bootstrap), I expect that DRUSH_BOOTSTRAP_DRUSH commands can be placed in sites/all/drush (if a patch similar to #0 is applied), and will be found, and will run at the specified bootstrap level (DRUSH_BOOTSTRAP_DRUSH). Haven't tested this exact configuration, but essentially that's what I was doing when I tested #15, and early-bootstrap commands work.

Great - that resolves one major hurdle.

The other is that alias and config files can be put in the root of a drupal site today, but commandfiles are proposed for sites/all/drush or ROOT/.drush. Shouldn't these all look in in the same place?

I'm in support of sites/all/drush. People already know how to use it (ie modules/themes/libraries), and if we document it correctly, I don't think we'll run into many support issues around the lack of multi-site. We could even add multi-site support in as a follow-up issue if it's desired...

Do you think drush pm-download drush_queue should download to sites/all/drush? I'm conflicted about that one.

Yes, currently alias files can be placed at DRUPAL_ROOT; in Drush 5, I think this location should move to sites/all/drush, same as for commandfiles.

pm-download of Drush extensions could go to /etc/drush if it exists, or sites/all/drush if it exists (and a site is bootstrapped), or $HOME/.drush otherwise.

I like the implications of this. I've had the site aliases in local sites folders, and had to copy or symlink them, or keep them somewhere global.
sites/all/drush will be a good alternative

c.f. related issue #1377824: Centralize logic for determining location of site-wide and user-specific configuration directories

Based on the discussion in #1377824: Centralize logic for determining location of site-wide and user-specific configuration directories, maybe commandfiles should go in sites/all/drush/commands.

Should alias and config files also be allowed in sites/all/drush? It would be strange if this were not supported; this seems to be moshe's point in #18. If we allow aliases and config files here, do we eliminate support for placing them at the Drupal root?

Would it be less confusing for folks to close this issue as a dup of #1377824?

This patch standardizes on sites/all/drush for alias files, config files, and commandfiles when searched for in a drupal site. alias and config files may no longer be placed in DRUPAL_ROOT. As suggested, the commandfiles are loaded during DRUSH_BOOTSTRAP_DRUSH so even early command should succeed.

I did not change the default destination for pm-download when it download commandfiles. I'm pretty neutral on that.

All tests are passing.

Yeah, that looks good to me.

There are pros and cons to changing the default for pm-download. I suppose that if you inserted a check for DRUPAL_ROOT/sites/all/modules (exists and is writable) in after the test for the site-wide file, but before the test for the user's $HOME/.drush folder, then it would usually do the right thing. I'm neutral on this as well, and feel the patch can go in as-is.

I do get the following error in the site upgrade test:

file_put_contents(/tmp/drush-sandbox/web/sites/all/drush/aliases.drushrc.php): failed to open stream: No such file or directory

Other than that, the patch looks good.

I forgot to mention that all cached environments from unit tests are invalid as they don't have sites/all/drush dir. Thats why you got that error. To delete cached environments, rm -rf $TMPDIR/drush-cache

Committed.

I will do the Change Notification soon.

Added Change record and removed tag.