Allow and encourage top-level directories instead of /sites/all/* directories

This makes me smile so hard. :)

Yay yay yay! Wow I can't believe all the pinnings are there already. Nice work on #562042: Search for install profiles in sites/[all|site]/profiles folders, and move core profiles into /core/profiles :D

I hope more justification isn't needed than sun has given already, but overall this is going to be HUGE win for new users to Drupal. Even today people are constantly putting modules and themes directly into the root /modules and /themes directories. Now that we've moved the root directories, people have no idea where modules and themes live at all (at least not without starting to dig through directories). Moving the end-user modules/themes to the root level let's people do what they've intuitively been wanting to do for years.

Awesome, thanks!

Tried to clean up references to sites/all.

So cool!

Possibly, but the last time we touched that file it was 3+ months and 80+ comments so I'd recommend deferring that to a follow-up.

No, unfortunately not because Git will not let you check in empty directories. :\ That's why they're there.

Oh, I'm sorry. /core/modules/README.txt. Yes, that's probably fine.

Actually, any file in a directory will allow you to check in/out "empty" directories with Git. Even if those are blank files. If the whole purpose of a readme file is to be able to check in an otherwise empty directory, it might make sense to place an empty dot prefixed file there instead. Good options might be .empty, .blank, or .placeholder... or of course an empty .gitignore would also do the trick. :)

Right, that's true, but it doesn't hurt to put docs in here too to explain how the directories are working, so we went with that. sun's actually talking about different README.txt files though, which are in the *core* modules/ and themes/ directory and say basically "hey, buster don't put stuff here." That's no longer relevant when the top-level directories are the ones you should put your custom stuff into, as it will be after this patch. :) So those files in the core/ sub-directories can/should be removed.

I'm totally in for this. :D

Can someone step through a D7 -> D8 upgrade path with some contributed modules/themes and see how that goes first?

This is going to be a huge pain for people managing multiple sites on different versions. The drush maintainers will probably kill you. I admit I'm not the "target audience" for this issue, but I've never really had a problem with /sites/ confusing me when building single-site installs.

(Although, disclaimer, I'm the one who added sites/all in the first place back in Drupal 5. :-) )

Quick reaction..

1. There is a simple beauty to mv sites elsewhere. Good for upgrades and backups.
2. When you discover you want multi site, the transition process is already tricky. Starting with the idea
   of multisite when possible to avoids the mini migrations of, say, later APC optimization.

This change seems good for the not so experienced.
Leaving sites/default/settings.php behind doesn't sound that fine though. Easiest to find for beginners would be in the base directory, but I don't know if that's possible (permissions, etc ?)

What other uses are left for the /sites dir ?

Does this mean the sites/*/ will die? I have some multisites running, and it works very well, it's great a benefit when you have limited memory space for APC and such things.

@pounard: Hell no, just sites/all/*! :) Everything is explained in issue summary. :P

IMHO, this issue could benefit from more visibility.

My 2 cents: after some thoughts, personally I'm against this. I think it's really nice that all core files are in core/ folder, and having all site files sitting in sites/[all|default|XYZ] folder is consistent and convenient.

I think it would be great to just add in README.txt file one extra paragraph:

• About Drupal
• Folders/files structure
• Configuration and features
• Appearance
• Developing for Drupal

explaining structure of folders and what goes where.

IMO with moving only sites/all/* folders without touching rest of sites/* will only bring inconsistency and bad UX (user = site builder).

Ok, thanks.

#562042: Search for install profiles in sites/[all|site]/profiles folders, and move core profiles into /core/profiles was recently fixed, but this issue here intents to move the /sites/[all|site]/* (including /sites/[all|site]/profiles) to the root of the directory structure.

I tried to understand from reading through this issue how the all vs. specific themes/modules/profiles will be separated once we implement this + #1055862: Require sites.php to opt-in for multi-site support/functionality but don't get it. Someone care to explain? Will it be:

/modules/all
/modules/specific-site
/themes/all
/themes/specific-site
/profiles/all
/profiles/specific-site

@klonos
As I understand it, the "all" folder concept is to be completely removed. So modules and themes should normally go in /modules or /themes directly. That way all sites can use it. If there is a specific site that wants to override a module it would go in sites/specific-site/modules.

...ok so:

1. top-level modules/themes/profiles folders from D7 (that were used for core modules/themes/profiles) were moved under /core for D8.
2. sites/all/* were moved to top level, thus recreated the top-level modules/themes/profiles folders the were (re)moved in step 1 - only this time they'll be used for non-core modules/themes/profiles.
3. for site-specific modules/themes/profiles in D8 users will have to create themselves a sites/[site-name] directory (or will the D8 folder structure already include a "sites" top-level directory?) and place them there.

Right?

@klonos Yes, that is how I understand it.

More work on moving also settings.php to the drupal root is going on over at #760992: Where should site specific modules, themes and settings be kept?.

Thanx ;)

i like it the way klonos described it.
a lot of documentation will have to be changed and some notices will have to be visible somewhere

don't forget Libraries dir during planing, as Libraries API might still get in time for core 8.
#667058: Add a libraries folder with a README.txt in it to DRUPAL_ROOT
#1167496: Libraries API in core

let me throw in some wood to the fire...
there has been in the past the discussion about using /contrib dir in place of /sites/all dir, as a solution that solves the problem underneath, but does not cause confusion (like the whole deal of /modules stated as "core alert, dont' hack" since old times) to current drupal users.

i'll revive it briefly, perhaps not with many arguments just yet, but instead with directory schematics:

current D8 scheme (/sites/all/*):

core/ * drupal core only
    includes/
    lib/
    misc/
    modules/
    profiles/
    scripts/
    themes/
    vendor/
sites/ * contrib files and site specific files
    all/ * all contrib files, not specific to a site
        libraries/
        modules/
        profiles/
        themes/
    default/
    www.example.com/

proposed change - plan A (/sites/all/* -> /*):

core/ * drupal core only
    includes/
    lib/
    misc/
    modules/
    profiles/
    scripts/
    themes/
    vendor/
libraries/ * all contrib libraries, not specific to a site
modules/ * all contrib modules, not specific to a site
profiles/ * all contrib profiles, not specific to a site
themes/ * all contrib themes, not specific to a site
sites/ * site specific files
    default/
    www.example.com/

proposed change - plan B (/sites/all/* -> /contrib/*):

core/ * drupal core only
    includes/
    lib/
    misc/
    modules/
    profiles/
    scripts/
    themes/
    vendor/
contrib/ * all contrib files, not specific to a site
    libraries/
    modules/
    profiles/
    themes/
sites/ * site specific files
    default/
    www.example.com/

as you can see, plan B, might create a little more organization, clarity, simplify documentation... added to less confusion caused to old drupal users _and_ a clear correlation to contrib vs core vs user space.

I'd rather kick the idea of a "contrib" directory to some other issue. That's a brand new idea, whereas this issue is about reshuffling what we already have.

the whole "contrib" name wasn't referring to whether modules/themes/profiles are contrib, custom, fell_from_the_sky or belong to a category like administration, but rather, they are non-core, and also, are not specific to sites (ie, /sites/all).

it's rather a placeholder name. could be something else, like "global", "non-core", etc. what i'm trying to sell is just a top-level directory that holds /modules, /profiles, /themes, etc in it, keeping things a little more organized and separate.
i don't think they belong in /sites, but since we are changing and perfecting things, right now i don't feel comfortable with having them in /, for the various reasons mentioned above, and for others...

like i might add, it's also easier and faster to do a
tar -zcvf contrib.tar.gz contrib
than doing a
tar -zcvf contrib.tar.gz libraries modules profiles themes
or a
tar -zcvf contrib.tar.gz --exclude=core --exclude=sites --exclude=index.php --exclude=robots.txt --exclude=.htaccess --exclude=web.config .
from an administrator/backup point of view

the hierarchy inside these directories is another whole issue that i wasn't trying to discuss here; i've done it before elsewhere ;)

sun wrote:

Can we move forward here?

If there are still concerns about removing support for sites/all/*, then I'd suggest to just simply not remove that discovery path from drupal_system_listing() - but still create the new top-level directories for freshly installed sites. In essence, we don't really care and if some people find sites/all/* more attractive than top-level directories for whatever reason in their custom setup, then they could happily use them.

If that's what it takes to get this patch in, then I will happily do that minimal adjustment/revert to the patch.

This seems a good, flexible compromise, so that site builders have more options to organize the file system the way they choose.

As suggested by others, other options like the recent proposal to group directories, could also be discussed and considered later in other issues for possible addition to drupal_system_listing.

Ok, let's do that then. Can we get an updated patch?

I guess that once we have #1439316: Provide means for module maintainers to collect heuristics on certain settings of their modules. in place and enough time has passed so to have adequate data collected, we'll be able to see the percentage of people still using /sites/all. If that amount is trivial, then perhaps we can revisit this and remove the support for /sites/all/*. I guess we'll only want to do this for performance reasons - if that is an issue with that specific feature.

I used the previous patch on a site for 2 weeks, and the new one still works as well, and is LESS controversial.

Ok great. I did a quick manual test to ensure that multisite was still working with this patch, and despite a derail based on my assumption that it picked up subdomains even though it does not... it still works. So yay!

Committed and pushed to 8.x. WOOHOO. This is going to need a change notice.

According to the last paragraph of the change notice:

In addition of making the directory structure much more intuitive for all users, this change also includes the separation of installation profiles provided by Drupal core in /core/profiles vs. contributed and custom installation profiles in /profiles, which helps with updating the code-base.

I think the D8 example is slightly misleading:

/core/modules        # Drupal core modules only. (!)
/core/themes         # Drupal core themes only. (!)
/core/profiles       # Drupal installation profiles. (mixed)

Should be something like:

/core/modules        # Drupal core modules only.
/core/themes         # Drupal core themes only.
/core/profiles       # Drupal core installation profiles only.

great =)

found an old related issue:
#1443420: About Drupal 8 directory structure

A question came up about this change on #1539940: Encourage best practices in /sites/README.txt, /modules/README.txt, /profiles/README.txt and /themes/README.txt.

Specifically, how does this affect the other sub-directories in sites/all that haven't been mentioned (such as translations, libraries, drush, etc).

great patch that i wish i was following at the time. i really think we should go all the way and remove sites/all from the discovery process. it does not simplify drupal to add new ways of doing stuff. you have to remove the old ones too. the only justification given here "so we can let site builders choose where to put their files". i think thats pretty weak relative to the added confusion of 'the modules could be here or there or there or ..."

i'm adding drush support for this patch and each directory scan takes time.

WTF, I did not remove those tags.

Regarding #49, I did a bit more research...
- libraries are contrib-only so I think we shouldn't bother making a top-level directory for them.
- translations apparently don't belong in sites/all at all. See #1802322: Translation instructions are incomplete

RE #50, true, too many more than one ways of doing the same thing confuses.

#50 And how about multisite? How will this work?

@pounard - the recommended place to put code thats shared across sites is now /modules, /themes, /profiles, etc. thats the whole point of this issue, which has been committed.

ho you mean that even multisite would share all, and there should be no way to provide different module versions on different sites? I never really used it except once, that's a very uncommon doing, but is that something we can do?

pounard: read the change notice on this issue (link is up in the summary section) -- it should answer all of your questions. sites/(site) is still available, as is sites/all.

I was asking because #50 was suggesting i really think we should go all the way and remove sites/all from the discovery process and that's disturbing regarding multisite. Aside of that, I'm not against this issue, it's a fine feature to allow both top level instead of all and sites/SITE/*

I read that as dropping sites/all, not dropping sites/{sitename}.

sites/all does allow for folks to easily create a standard recipe without going all the way into basic profile creation, but with tools like drush make I think that ground is reasonable well covered.

Oh my... ok I misunderstood what he said, sorry.

@jhodgdon, #52:

...libraries are contrib-only so I think we shouldn't bother making a top-level directory for them.

...well there's #667058: Add a libraries folder with a README.txt in it to DRUPAL_ROOT

Tagging, per #61.

Updated issue summary.