As recently discussed on the devel list, here's my first crack at supporting modules and themes in sites/all/modules and sites/all/themes, respectively, rather than putting contrib modules/themes in with core ones. It seems to be working for me, with one problem that I'd like some help on. :-)

The system table stores, among other things, the path to a given module/theme/whatever. If a module is moved from modules/ to sites/all/modules, then the system table doesn't update. The cause seems to be the way the table is repopulated. The filename is used as the key, and therefore to repopulate the table Drupal will pull out the old records, delete the table, tweak the records, and reinsert them all, with the old filename still intact. That means a module must always be in the first place it was seen or Drupal will never notice it again.

I don't know if that's a specific bug to this patch or a separate issue, but it was driving me batty. :-) Any assistance on that would be most welcome, as would reviews of this patch itself.

@Dries: If you want to make sure this goes into 4.7, please set to critical. If you'd prefer to hold off for 4.8 on it, please set to postponed and I'll come back to it later. I'm fine either way, but I wanted to get some action on this front. :-)

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Dries’s picture

I think this is 4.8 material. It's something we can commit as soon HEAD opens up.

Crell’s picture

Priority: Normal » Critical
Status: Needs review » Postponed

4.8 it is. Setting to postponed and critical (to remind us to come back to it).

Crell’s picture

Status: Postponed » Needs review
FileSize
1.71 KB

OK, since HEAD is now once again open, here's the patch again. Actually the system table caching issue was fixed late in the 4.7 cycle, and the fix for that broke this patch completely. :-) So this is a brand new version that is actually even simpler. Actually, it's just one line of code (plus doc updates), unless I'm missing something completely.

So, am I missing something completely?

drumm’s picture

Priority: Critical » Normal
ethanzhong’s picture

Thank you for your great help. Just this line of code takes me two whole days to search. Thank you very much. You do a great help for me.

Crell’s picture

Thanks Ethan. Is that an actual review, then? If so, can you change its status to being ready to commit, if you think it is? It will never get into core if someone doesn't review it and say that it's ready, which I can't do myself. :-)

sime’s picture

Crell, I have installed this and as a test installed ecommerce into the all directory. So great, and something I will use, except I need to patch 4.7 to do this first. :-)

Crell’s picture

Yeah, it should work on 4.7. If you want it in 4.8, though, please set it to RTBC so that Dries/drumm notice it. :-)

sime’s picture

OK then, I'd better test the themes directory and then I'll be confident that everything is ok. Doing this this morning.

In the meantime, I came across this book page about multisites http://drupal.org/node/43816, that refers to this similar "all" subject http://drupal.org/node/5942. Because the ticket is from a fairly long time ago, I wouldn't have thought twice, except for the fact that it's referenced in the handbook.

Crell’s picture

Wow, that is old. :-)

However, I don't think it's the same issue. The patch is about adding sites/sitename/module support, not a sites/all. We actually have the ability to put a module into all sites *right now* by putting them into /modules/. The idea of this patch is to provide another location for user-added modules, so that /modules/ can remain pristine with core-only code.

I suspect that handbook page should be updated and the link removed after this patch gets in. Or maybe even now, since the patch link is irrelevant.

sime’s picture

Status: Needs review » Reviewed & tested by the community

I have tested this patch (#3) using two domains with the same root. I installed an applied modules and themes in "all", "default" and "www.example.com" directories and the themes and modules are behaving normally.

This is a great benefit for a multisite installation - avoids maintaining duplicate non-core modules in each websites sites/ directory. Even better it is only 1 line of code. Thanks Crell.

Marking RTBC.

drumm’s picture

Status: Reviewed & tested by the community » Needs work

INSTALL.txt would need to be updated too.

Crell’s picture

Status: Needs work » Needs review
FileSize
12.62 KB

Yeah, I suppose it would at that.

At sepeck's suggestion, I broke modules out into their own section in INSTALL.txt. I then moved the stuff about site-specific modules there, and expanded it to cover sites/all. If someone else wants to take a crack at revising the docs, feel free. I'm more interested in the code.

Also, when this goes in I'd recommend adding sites/all/modules and sites/all/themes as empty directories to the default setup. I can't do that from a patch, apparently.

sime’s picture

Status: Needs review » Reviewed & tested by the community
FileSize
2.54 KB

Crell's patch had some changes to other files and I am ignoring those.

The new text reads well, I just made the paragraph (section 4) a little less pithy, because the example that follows is a good explanation in itself.

New patch attached.

sime’s picture

So, the current applicable patches are at #3 and #14

Crell’s picture

Pithy? :-)

Yeah, I forgot to trim the last patch, sorry about that. +1 RTBC the patch in #14.

Morbus Iff’s picture

-1 from me. I just don't see this as very useful. You can accomplish the same thing with symlinks in the sites/ directory, or by creating a modules/site-specific/ parent directory and sticking everything in there. I'm also not much of a fan of the new documentation, which makes it seem /required/ to use these new directories over the existing locations. Resetting back to "needs review" (regardless of my comment, a new patch was attached that needs review, not committing).

eaton’s picture

I have to disagree. Setting up symlinks should NOT be necessary for this kind of setup. On moderate-sized installations with a number of multisite installations, dropping key modules like Views, Devel, etc into the /all directory is self-explanitory IMO. That said, I agree that the documentation needs updating to clarify that the new /all directory is not required for use.

+1 for the concept, but as Morbus said the code needs review, not RTBC.

Crell’s picture

I did review #14. sime and I have been reviewing all along. :-)

Actually, making it implicit that sites/all is required is a good thing, IMHO. The discussion on the development list that this patch grew out of was about keeping /modules reserved for core modules, making it easier to do drag-drop-overwrite updates. The problem someone pointed out was the lack of a way to put a module into all sites on a given installation without using /modules, which is what this patch solves.

Anything specific to a given installation lives under /sites/ with this patch, further separating core code from site-admin-added stuff. (One can still drop a .module file in anywhere, but with this, the recommend location is always under /sites somewhere.)

And yes, simlinks are possible but are really a hack, not an answer.

(And all of the above applies to themes, too.)

sime’s picture

Status: Reviewed & tested by the community » Needs review
FileSize
2.77 KB

Here is a new INSTALL.txt patch which makes the instructions a little clearer (eg. recommended not required).

(Also, I acknowledge my earlier error of marking RTBC on a new patch.)

Crell’s picture

I'm OK with sime's text, although I still feel that we SHOULD imply that /modules shouldn't be touched.

That said, sime, your patch doesn't have the actual code in it that makes what's in the new text possible. :-) Please rerun the patch to include that file so that we can get this committed. Thanks.

drumm’s picture

Status: Needs review » Needs work
sime’s picture

Status: Needs work » Needs review
FileSize
4.48 KB

@Crell, ouch! :-)

Well, no pain no gain. He is a combined patch (system.module, install.txt) on head.

Gary Feldman’s picture

Three minor suggests on the text:

1. Delete the word "also" from "You can also download..."

2. Delete the sentence that begins "One way to install code." It no longer serves any purpose.

3. Use the full URL for the handbook, i.e. http://drupal.org/handbook, instead of saying "consult the Drupal handbook at drupal.org". This is just a courtesy to the reader, especially if they're using a text reader that automatically creates the links.

I'm also not fond of the sub.example.com example, and would rather see site1.example.com. It may be a subdomain in the DNS sense, but it's more likely to be a peer site from the Webmater's viewpoint. However, this usage appears elsewhere in the file, and I hesitate to suggest changing it here and not elsewhere.

Gary

Crell’s picture

FileSize
4.46 KB

Sheesh, we're spending more time on documentation than on the code itself. I don't know if that's good or bad. :-)

Anyway, here's yet another version that addresses Gary's points. As for sub.example.com, that may or may not be the best naming scheme but it's what the file is already using, so it's what this patch should use. Changing that naming scheme should be a separate issue, after this one gets in.

Can we get this in? Please?

webchick’s picture

> Sheesh, we're spending more time on documentation than on the code itself. I don't know if that's good or bad. :-)

It's good, if you want your patch committed. ;)

Shouldn't $searchdir[] = 'sites/all'; be wrapped in an if (file_exists()) check like $config/$directory?

sime’s picture

If sites/all directory will be added to CVS, as I hope it will, I assume we won't need the file_exists check.

webchick’s picture

Hm. A multisite install is an optional thing though... the vast majority of sites out there will have their contrib modules stored in folders in the normal "modules" directory. You can argue about whether or not that's a smart thing to do, but the upgrade burden would be tremendous if people were *forced* to move their stuff under sites/all (ever had to move more than a dozen folders/files over a slow FTP link? Now imagine 500 or more as in a typical site with several contrib modules and themes. :P For some people this will be the only option).

This seems like a good patch for power users, but I don't think it should be required.. unless I'm missing something? And since it will not be required, it should be wrapped in an if (file_exist()) call. Even if it's included in the tarball by default, people might not upload it since it's empty, etc. Seems like just a basic sanity check thing to me.

sime’s picture

No, people can still put modules anywhere. Although we have structured the new wording to encourage people to put non-core modules under sites/ to get them into good habits.

On a side note. Personal opinion. There are many times when I think this extra section, targetted at the average admin, is not in keeping with the rest of the INSTALL.txt file, which *I think* written above the head of the newbie. But to echo Crell, is this to much worry for a patch that won't break anything - I almost vote to leave out the documentation patch altogether, to look at this issue separately.

sime’s picture

Hmm. Although I agree about "people might not upload it if it's empty". I hadn't really thought about that...

drumm’s picture

CVS can't manage empty directories.

Crell’s picture

I actually tried with no sites/all directory existing, and it didn't break anything. It's been a while since I worked on the code itself, but I believe there's a file_exists() call later on for the entire array before each call. (Remember, there's no guarantee that sites/example.com/modules exists, either.) Putting it here would be redundant.

This patch does not actually remove any functionality. All it does it ADD a place where modules/themes can go. The documentation can recommend or suggest or insist as it likes, but functionally the /modules directory still works, as does sites/all. Existing sites won't be broken in the slightest.

Dries’s picture

I've been using the sites/*/*/* stuff for a while and often find it annoying that it is that many levels deep. I can't think of a good solution, but things are somewhat burried away in sites/*/*/*.

eaton’s picture

Dries: the only solution I can really think of is a dramatic reworking of the Drupal directory structure. Perhaps something that puts index.php, update.php, and cron.php at the main level, each site dirctory at the main level, and a /core directory at the main level containing the stuff that's currently loose in the drupal directory.

That's another can of snakes, though, as the dev mailing list can demonstrate... This patch, on its own, is definitely a boon. It means I have to make *fewer* visits to those buried subdirectories for updates and upgrades.

Dries’s picture

Eaton: agreed. It's another issue, and probably shouldn't affect this patch. Just wanted to point it out, in case it would have implications. :-)

moshe weitzman’s picture

i love the sites/all functionality, and the docs improvement. symlink solution imposes an unneeded burden on the admin. i also agree that we should ship with sites/all directory and even in that directory we should put subdirs for themes/ and modules/ and a README.txt in each. CVS needs a file in each directory so we have to use this README technique. How about:

"Place your custom directories in this directory. You may also place custom versions of core modules here."

At the sites/all, perhaps say: Place your custom modsules and themes into the subdirectories here. You might also make periodic backups of this directory in order to backup the custom code for your sites.

sime’s picture

FileSize
7.04 KB

+1 @moshe - empty modules & themes directories, yes.

This patch contains a sites/README.txt which /encourages/ the newbie explorer to look deeper. This is followed up with 2 more README's in
- sites/all/modules/
- sites/all/themes/

OK, I know that scope creep is not the best way to get a patch applied (sorry Crell!). Hmm. But to me, it all falls into the same bucket, making sites/ a welcoming place.

sime’s picture

oops, use this one instead

Crell’s picture

Y'annow, I'm going to have to -1 here. This is a ONE LINE OF CODE patch. There is exactly one line of executable code here. Everything else we're discussing is tutorials.

I actually agree with making sites/all exist in CVS, and if a README file is the way to convince CVS to do that, fine. But now that's another three files that we can discuss to death in minutia. We'll have to at some point, but there's nothing to discuss until the functionality is there.

This patch doesn't break any existing configurations or require people to move files. It just adds another option, which is then documented in INSTALL.txt (which I agree is good). Let's get that in now, and add a feature for people who bother to read INSTALL.txt and know how to create a directory.

For the rest of the world, let's discuss the details of how to direct them to do things "the right way" after we've created and documented the existence of the "right way".

-1 on #38. +1 on #25. After that, we can even keep this thread open and discuss better documentation, in code and on the site, on how to get people to do things "the right way". I'm game for that, after the code change has made it in.

On a side note, a README file instruction should NEVER EVER point to a a non-aliased page on the site. That's just tacky, even if it is a low nid. :-)

sime’s picture

Status: Needs review » Reviewed & tested by the community

No complaints here.

Actually, number #25 seems to be intact. AFAICT, all issues and objections are resolved. (There are a few ideas for further doc improvements, but let's defer them.)

So, with the uncertainty of a drupal newbie, RTBC patch at #25.

Dries’s picture

Status: Reviewed & tested by the community » Fixed

Committed to CVS HEAD, minus the README.txt. We can ponder about those some more.

Anonymous’s picture

Status: Fixed » Closed (fixed)
Jaza’s picture

Version: x.y.z » 4.7.0
FileSize
4.6 KB

Here's an updated patch for 4.7, in case anyone needs it. Original still applies to 4.7 fine, with minor offsets.