Currently Aegir supports multiple platforms, which I think is an excellent idea. However there is currently no way to share themes, modules, or install profiles across platforms. There are a number of use cases in which this would be beneficial.

What is proposed:
Move packages outside of the individual platforms, and symlink packages into platforms. This will also allow for granular organization of packages.

Some Use Cases:

1) Creating an upgrade path to multiple releases of Drupal. For example, let's say that I want to move my 6.12 site to 6.13, and I want to ensure that a package will work in the new version. As Aegir stands, I would have to have a separate copy of the given packages in each platform. This is redundant and unnecessary.

2) Modules can be grouped into "classes" of modules. This will allow aegir administrators to selectively include sets of modules or themes, or even install profiles to be included when a site is deployed. Currently, if modules are kept in /sites/all/modules within a platform, they are visible to all individual site administrators. In order to support multiple "types" of sites per platform, it is not always beneficial to allow access to all of these modules.

The short solution is to put modules in the /sites/example.com/modules folder. However, that means that the module is ONLY available to that particular site, and another copy of the module would need to be created in /sites/example2.com/modules for a second site using a similar install profile. For example, let's say that I want to create two e-commerce sites, two blogs, and a drupalbin type site for various clients. The ecommerce set of modules (ubercart, and other related modules) need to be available to multiple sites, but it would be inappropriate to have these modules available to the blogging sites. Likewise drupalbin modules would be inappropriate to include in either a blogging site or an ecommerce site, etc.

The better solution would be to create the following folders:

/var/aegir/packages
/var/aegir/packages/modules
/var/aegir/packages/modules/custom
/var/aegir/packages/modules/custom/release-X
/var/aegir/packages/modules/custom/release-X/all
/var/aegir/packages/modules/custom/release-X/example-module-set
/var/aegir/packages/modules/contrib
/var/aegir/packages/modules/contrib/release-X
/var/aegir/packages/modules/contrib/release-X/all
/var/aegir/packages/modules/contrib/release-X/example-module-set
/var/aegir/packages/profiles/release-x
/var/aegir/packages/profiles/release-x/all
/var/aegir/packages/profiles/release-x/example-profile-set
/var/aegir/packages/themes/release-x
/var/aegir/packages/themes/release-x/all
/var/aegir/packages/themes/release-x/example-theme-set

This will allow packages to be organized by major release, and would allow for selective symlinks to be created either manually or by drush, thus including only the packages that would be appropriate for that particular site or set of sites for a platform.

Comments

steveparks’s picture

steveparks commented

Hi ixlr8

Thanks for your enthusiasm for Aegir and your ideas for making it better!

Your idea is interesting and I can see some clear advantages.

You'll be pleased to know that the dev team have been considering how to manage packages for a while - but rather than go a separate 'aegir' way they have been contributing code to the core 'package manager' that is going to be in Drupal 7.

That should make package management much much easier.

I'll leave your ticket open for now because the dev team may have something to add.

But if they don't it's because the issue of managing packages in Aegir/Drupal is in hand - so watch for future Aegir releases.

Thanks again
Steve

steveparks’s picture

steveparks commented

Correction: in my post above I meant to refer to plugin_manager rather than package_manager.

Update: discussion from IRC...

steve_parks_, this is an interesting feature ixlr8 was commenting here the other night
but it introduces a cross-platform dependency on same packages
and packages folder become a new entity. Not the packages-in-platform current thing but a major-release-packages entity that must be glued to compatible platforms
and operating with packages should not be done from platforms
so ilxr8 is proposing an interesting thing (clusters of packages or "classes of modules")
but limiting a set of clusters to a platform fits better with current implementation
and is more suitable to be accepted
but then that removes what he's trying to aim for - which is that when he goes to upgrade from 6.12 to 6.13 the modules he needs are already on the new platform
it acn be acomplished in several ways: at installation time of new platform a task can import all the modules of another patform and do a update at the same time
or the folder can be copied manually ...
I think the drawbacks are smaller than the complexity of sharing modules between platforms
yeah - okay I like the idea of being able to setup packages for a platform as part of new platform creation
you could then have the option of platform variants - ie same drupal core - but with diff module combinations
eg - an ecommerce platform
etc

Nice idea - imagine being able to create new platform variants within aegir - choosing the existing base platform codebase, then choosing groups of, or individual modules or themes, and then aegir shifts the whole lot to be a new platform.

But i bet adrian has something to say on this!! :)

platypus media’s picture

platypus media commented

I did an Advanced Multisite Implementation BoF at Drupalcon DC. I touched on Aegir briefly, and Adrian gave a nice overview for everyone.

@Pilot: I like the idea of Aegir creating a specific platform for a set of modules, but that still means that there is an extra copy of Drupal to upgrade later, as well as numerous modules that will be used on multiple platforms (ie. cck, views, etc.) that will still need to be upgraded or patched multiple times. No offense intended, but I find that cumbersome, redundant and unnecessary.

Creating a centralized location for a single version of Drupal (6.12 for example) that can be used for multiple sites in multiple ways, will allow all sites to be upgraded to the next version (6.13) simultaneously, instead of having to do all the ecommerce sites, then all the blog sites, then all the video sites, etc. The same goes for contributed modules. When dealing with a large number of sites that may or may not use a specific module or set of modules, that's going to become difficult very quickly. It's one thing when we're looking at a couple of platforms, but that's not scalable. I think it would be much easier to upgrade modules, run update.php on all the sites, and call it a day. There's no harm in running update.php on a site that isn't affected by a code change, since there will obviously be no database queries that are necessary.

The only foreseeable downside would be the instance where a site needs a specific patch or hack to a module or core. However, there's an easy work around for that. Let's say that for one site's requirements, user.module needs an alteration that will work for that site, but would break every other site that is using the platform. Now don't get me wrong. It's never a good idea to hack core, and I don't condone that anyone does it, but sometimes requirements are requirements. The work around for that is to copy the user.module into the sites/sitename/modules/custom directory and make the alterations there. That way, the changes to the user.module will override the core modules, and in the event of an upgrade, the changes will not be lost. Additionally, and this is the beauty of it, is that the customized version of the module will override the core module.

I understand that what I'm proposing goes against the grain a bit, but I don't think that this would be as hard as it seems at first glance. Basically this should come down to a module that would be packaged with Aegir that would create a platform package manager to import packages into the appropriate package folder, creating classes of modules, etc. The second part would be that it would need to read the directory for available packages for a platform, and hook into the site node edit form to create a list of available packages, very similarly to how it is currently done with install profiles. Then on provision, the symlinks get written in the sites/sitename/modules/contrib or custom folder. I think the biggest challenge is coming up with an appropriate folder hierarchy that would be extensible and flexible, but not chaotic. I kind of see this working like being able to create a link to a folder containing multiple modules (a class of modules), or being able to drill down a level to import specific modules to a site for a specific purpose. For example, let's say that I'm making an ecommerce site, but I need one module from the blogging class of modules (I don't need or want all of them- just the one). Rather than install the entire class, I could just link in that one module.

Thoughts?

Mike

anarcat’s picture

anarcat commented
Project: Hosting » Hostmaster (Aegir)

Hi,

First, thanks for your contribution to the discussion, i think it's valuable input and a good point of view. I used to manage multi-site installs with common copies of the contrib modules, symlinked. I don't do that anymore, because upgrading one module will break too many sites together. That's not the approach aegir currently takes: different modules versions are on different platforms, I think that's fine. I don't see how that is not "scalable" or what exactly the problem is with that so that we should take a more complicated approach with clusters of modules, I don't see what it resolves.

To give you an example, I'm looking at implementing a "platform clone" drush command that would take one drupal install with all its modules, and create a new one, with everything updated to the latest (say).

Some argue this is all out of Aegir's responsability and a question of code management too...

anthonysomerset’s picture

anthonysomerset commented

i am new to aegir, currently running several sites on it, to keep them somewhat seperate, whilst keeping the advantage of multisite, whilst looking at how i can take it forward/add more sites

my biggest issue/request here is that if i have to upgrade to a new minor version of drupal i would have to manually copy all the modules from one platform to another or even redownload them using drush, some sort of centralised module location available for all specified platforms would be much preferred, on a platform or package management level you would either set the modules folder location at the platform level or if you want to say have categories of module, it could be tackled from the package manager side with some sort of make available to x and y platforms field

personally not interested in filtering modules per site although i can see why that is useful but having some form of shared modules between platforms would be more useful to me.

i guess this will all change again with drupal 7 though

redben’s picture

redben commented

subscribing

adrian’s picture

adrian commented
Status: Active » Closed (won't fix)

See drupal.org/project/drush_make for how to solve this.

This is outside of the scope of aegir.