Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.Basic Theory:
Drush make in the world of real profile development requires an understanding of a couple of basic concepts. The first concept is to understand that you have an essential need for no less than 2 make files. The "stub" make file, and your profile's make file. These together allow easy creation and distribution of light weight essential building blocks for your profile.
For more, see also: Managing Drush make files for an installation profile.
The Stub Make File:
The "stub" make file is a very simple construct. It references exactly two things. First it references the core version of Drupal you intend on using, and second it references the profile you intend on bundling with your distribution. There's an easy win here as well if you intend on bundling multiple profiles with your distribution. A perfect example of this could be an aegir platform.
Example stub make file
; make file for drupalorg_testing profile
core = 6.x
projects[] = drupal
api = 2
; Profile.
projects[drupalorg_testing][download][type] = "git"
projects[drupalorg_testing][download][module] = "drupalorg_testing"
projects[drupalorg_testing][download][branch] = "master"
By way of make files, this is fairly simplistic. Again just referencing the version of drupal core we want, telling it to then GET CORE, and specifying the install profile to use. The install profile has further make files and will be recursively made for us, so we needn't specify any additional modules etc. The profile should take care of all of those things for us. In a perfect world, these stub make files can be stored, referenced, whatever and need not ever actually be updated until you're changing versions of drupal core.
The Profile Make File
As mentioned above, the profile(s) you're specifying in the stub make file should have its own make file specific to it. This make file is generally going to specify the additional modules you need, any directory structure beyond the default that you desire, any themes you may need and finally patches to modules. Typically we anticipate that a profile's make file will actually receive quite a bit of updating in the long term. This is because the best practice is to specify specific version of modules you wish to use. This makes it easier to maintain patched code, test new versions, and push new distributions only after testing has properly been done, instead of simply expecting all new versions of all code to simply work together.
For a good example of a profile make file check out open atrium's make file:
http://cgit.drupalcode.org/openatrium/tree/drupal-org.make
OA's profile make file is a great example as it pulls data from git, directly from drupal modules, applies patches, and re-organizes the basic directory structure of the modules it's downloading, placing contrib items together, custom features together, etc. Many of the best practices and coolest tricks can be seen executed in this file.
Keeping it all together:
For myself, I typically build a distro.make file locally with our stub make file in it, and then simply run drush make, but the question becomes, "Where do I store that distro.make file?" and there are a couple of potential answers. If you choose to store it along side your profile, that's perfectly fine, drush make will just end up doing a little extra work retrieving items it's already retrieved once. As a perfectionist, I prefer to keep a README.txt file with my profile that contains the stub make file's contents and describes how to use them. This is ultimately developer's choice, but putting the stub make file in the repository with the profile often confuses the uninitiated, and leads them to believe they should download the profile before drush making anything. This is fundamentally incorrect as outlined above. The stub make file can be made from anywhere, and it will generate a full Drupal install.
Working with drupal.org's automatic packaging:
I won't cover this topic in detail except to say that d.o essentially builds a stub make file for you on the fly when it generates a new release for your profile/distribution. You can read more on how to work with drupal.org's packaging here: http://drupal.org/node/642116
Comments
Just use the drush make website
To save you a lot of time. Just use the drush make web tool. It generates a make file
http://drushmake.me/
I created a similar tool
I created a similar tool here: http://drupalmake.com/make
It allows for make template inheritance as well!
Mine is also still on just Drupal 6, but I can add that very soon.
@rupl @doublejosh: Thanks for
@rupl @doublejosh:
Thanks for the update, I just started playing with making making files and I wish someone had made me do it much much sooner! Thanks to both of you for putting up those sites.
http://www.radarearth.com
BTW: Mine still has a module namespace bug
Great. Mine still has an existing bug for module names that are longer than a certain length when I loaded up the list! Yikes!
Just confirm your make file after you create it and fix any modules that fail to download.
Also, sorry for the company branding. I'm going to put a custom theme on that subsite context so the Drupal community doesn't have to get sold to :)
BTW: Be sure to make use of the inheritance!
Question about inheritence...
Is inheritence a drush-make feature having anything to do with nested makefile support or a feature of your makefile maker?
:~: Senior Drupal Site Builder
Drush make installs contribs in installation profile directory
Perhaps this question is better covered in the forums, but I just read through #586148: Nested makefile support and from my own testing with Drush Make 2.x, it seems like the approach to packaging installation profiles with make files outlined in this article results in all modules, themes and libraries being downloaded to the profile's own directory. (ie, /profiles/xyz/modules/some_module, not to /sites/all/modules/some_module).
Setting the "subdir" variable doesn't change this, as it too is used relative to the installation profile's directory.
Am I missing something (very likely...)? I understand while keeping modules/themes/libraries in /profiles/xyz makes sense when distributing distributions. But as a build tool for quickly setting up the starting point of a custom site, it makes more sense to place these dependencies under /sites/all.
Sean Larkin
When building a profile, the
When building a profile, the modules available to it are available to all sites running on that profile, it's functionally "sites/all/modules" with the clear exception that other profiles within the code base can run different versions of the same modules. A great example would be in drupal 6, if you were running a single code base with both open atrium and some other profile, open atrium pulls a specific version of views and applies specific patches to that version, while your other profile, even though it also has views, will run it's version of views for sites installed under it's profile. This is an often overlooked feature of drupal in general. Also, modules in sites/all/modules are available to all sites across all profiles, so this is actually something you should be very mindful of when running multi-site installations.
Hopefully this answers your question to some extent.
Eclipse
Yes, I agree that this makes
Yes, I agree that this makes a "stub" makefile less useful. I tend to separate more as a "distro" makefile for everything in sites/all, core and the profile. Then the profile makefile for anything that is only used by that profile.
Previous discussions #913148: Please bring back install_path and #745016: install_path deprecated?
it's the path that matters to me.
Eclipse,
I understand how it works functionally, but I also agree with Joe below that this makes packaging make files with profiles less useful as a starting point for quickly building out custom sites. When modules are placed in /profiles/modules as well as /sites/all/modules, it makes updating those sites more difficult down the road.
-s
Sean Larkin
Not arguing that, but I
Not arguing that, but I submit to you that you should not be using sites/all/modules for anything BUT custom development. The situation in which drush_make shines the most is aegir platforms, in which the modules we utilize are those available from the platform. Again this is more of a multi-site approach, so upgrading sites is a much narrower process... i.e. we generally expect to NOT have to worry about upgrading any modules except those provided by the file (since you would have to address those individually per site and that would be very painful.
Eclipse
Eclipse
Those are large assumptions
Storing contrib modules in /sites/all/modules/contrib is a very standard practice. My understanding is that the whole point of the "subdir" variable is to allow for this.
Drush make is a general purpose tool for managing installation profiles. Aegir-based site deployments and upgrades is a less-generalized use case. Regardless, I think that you're missing the point that JoeSteward and I are trying to make, that there's no reason that I am aware of /not/ to allow the make file to set alternate module placement relative to the document root.
Sean Larkin
Right, well putting those
Right, well putting those references directly into the stub make file may get you the desired result. I never use it that way so I can't say for sure. Almost everything I ever do ends up being a custom install profile on an aegir system for maintenance purposes, so you'll forgive my narrow scope. That doesn't de-legitimize my other points though, and hopefully that's food for thought.
In a "custom" installation, I would still expect that you have dozens of modules you utilize for your every day work that could be encapsulated into an install profile. This is a really great way to limit the amount of stuff you have to manually maintain. And that was the point I was trying to make, both with the original article and with the conversation in the comments here. Just because you're used to it ending up in sites/all/modules doesn't mean that modules in some other location are bad/more difficult to manage. In fact, in this case, quite the opposite is true. If you know you're going to utilize views on every site you work with, then don't put it in sites/all/modules, that would just require you manually upgrading those things. Put it in the profile's module dir, and then you know that it is something maintained by your make file. Then anything in sites/all/modules (or sites/[site-name]/modules) becomes the extraneous stuff that you know is going to need more attention. (Also keep in mind that sites/[site-name]/modules can have different versions of the same module per directory as well.)
Bottom line, you're saying:
We have modules we use for custom development, and we want them in sites/all/modules.
And I'm saying:
No you want to build a custom install profile that configures your custom development environment in a good 'base state' for all your work, and then sites/all/modules should really only contain modules that will meet the criteria of: irregularly used, needed availability across multiple install profiles, and/or expected issues with upgrades. Optionally you can use sites/[site-name]/modules for when you need a one-off module available only to one site, or when you need a specific version of a module available to a particular site.
Hopefully that clears this all up. Don't get obsessive about sites/all/modules being your location for all your modules, it's not actually that important, and drush_make is making it very maintainable for you going forward no matter what location the modules may end up in.
Eclipse
Sites/all for multisites
I'm a bit surprised at the viewpoint that /profiles is an acceptable place for modules. I get the point of maintaining strictly by version, but we are running multisite installs for the very purpose of keeping modules in a central location so when we upgrade ANY sites using that module, we update them all at once. Isn't this essentially the same as updating every site that's using the same install profile?
We want to standardize our installs as much as possible. I was hoping that I could use a make file with an install profile to set up a new site within an existing multisite install, but after reading all this, I'm beginning to think that isn't feasible. Maybe the answer for me is to write a drush script that manually configures a given site.
Basically, I want the script to download any modules that aren't already there, enable the base module set, configure a set of roles and users, and set some sane module configuration. From there adding any one-off modules would be much easier.
My understanding was that if multiple sites were using the same module, it made more sense to have them in sites/all for performance and easy upgrading. Then any one-offs were in sites/domain.com/modules. Am I missing something here? By the way, in D7, I ALWAYS set up multisites simply because the sites.php aliasing for moving sites from development to staging to prod is so much easier because they will answer to multiple domains without serialization errors.
In short, "yes".What you're
In short, "yes".
What you're missing is the bigger picture here.
What you've just described is an install profile with drush make powering it. Maybe I didn't expand on this topic in the original documentation, but fundamentally, when it's time to upgrade your site for a client(s) you're just going to update your profile's specific make file, and then re-drush make the stub make file. This generates a whole new directory structure ready for migrating your customer into. Often times the solution of choice here is Aegir since it can migrate site(s) from one directory structure to the next. Other workflows are definitely possible here as well, for instance I believe you can remake on top of an existing directory structure, though I wouldn't advise it. Just like I wouldn't advise deploying a new module(s) update for a whole slew of customers simultaneously (which this method, btw, does not prevent). It's better to test it a bit.
With that all being said, I'm not sure why people have latched onto this sites/all/modules dogma quite so much. What's more important here is "don't put contrib modules in /modules" not "always put contrib modules in sites/all/modules". The former is true, the later is basically the best configuration to give a newbie. If you're reading this document, and understand sites/[sitename]/modules then you're probably not a newbie. That being said, we maintain profile specific modules in the profile's directory structure for the same reason you would maintain site specific modules in the sites/[sitename] structure... it's where it belongs and it buys you a lot of potential flexibility. This methodology is essentially the most flexible setup you could have in drupal (as far as I know).
To illustrate the specificity a little more:
sites/all/modules is a convenient place to put your modules for single site backup purposes, but when you start running multi-site and have custom profiles, putting modules in with their corresponding install profile makes managing the profile MUCH easier. This is doubly true when you start to consider that some install profiles may actually utilize patched specific versions of popular modules (like open atrium uses a specific version of views, with specific patches applied to it, and chances are you don't want that version of views available to all sites on a given platform (platform being a potential collection of profiles in a single code base).
Again, I'm not sure why people are so hung up on sites/all/modules... YES this is different. Try it out.
Eclipse
@EclipseGc It could be that
@EclipseGc It could be that almost every readme for module installation says to put your module in sites/all/modules
http://www.radarearth.com
Right, that's as opposed to
Right, that's as opposed to putting it into just /modules.
Eclipse
Most of the documentation
Most of the documentation says not to put any modules in /modules. That is for core modules. /sites/all/modules is for contributed modules.
http://www.radarearth.com
No, sites/all/modules is just
No, sites/all/modules is just ONE place that contributed module could go. My point was that they should NEVER go into /modules, and that sites/all/modules is just one of a number of acceptable places for contrib modules to go.
Eclipse
Thanks for explaining
I do appreciate that you took the time to reply in depth.
In our case, we're not updating client sites. We have about 15 corporate sites we're managing. For us, we usually *do* actually want them all on the same version because it makes maintenance easier. We *can't* usually just create a new directory structure and migrate the site(s) into it because we have devs who are jailed into a specific directory, so we don't have the option of just moving folders around. It also means that we'd have to update the vhost document root every time we updated. Sounds like install profiles are almost as much pain as what we currently do.
The other hangup is that we've trained our people to not put ANYTHING in the drupal root. Where do profiles live? In the Drupal root. Could we retrain them? Sure. But anytime you add an "except" to a process, it inevitably leads to confusion and pain. Plus when we're moving sites from our staging to prod, we would then have to move profiles AND the sites folder. Maybe there should be a sites/all/profiles folder so it's consistent? :P
I'll try it out, but I'm not really convinced this is the better way yet.
Sure, I understand your
Sure, I understand your hesitation here, understand that the objective is to remove from your developers any sort of module that is core to the profile. They can (and should) still put site specific modules into sites/[sitename]/modules. Likewise it's still acceptable for them to utilize sites/all/modules, I'm just trying to impress upon you that it's no the space for a profile specific module.
As to your issue with migrating across directory structures, understand that an install profile can still work just like a core upgrade does:
Backup your sites directory, drop your main dir, re-drush make the stub file, restore your sites dirs back in. In fact this should lighten the load of what you have to backup each time, not increase it. I want you to think of an install profile maintained this way as though it were a separate core, you don't need to backup the profiles directory because you're going to update the make file for new versions of modules that you've tested, and the process of re-making will populate the profiles directory again with the appropriate modules and the appropriate versions.
I approach this entire workflow from an Aegir perspective typically which would maintain all the things you were worried about. It's well worth at least experimenting with if you'd like to have a system that makes upgrading your sites from one version to another easier. And it understands make files pretty well.
Eclipse
How to test patches to .make?
I really hate posting support requests on a documentation page, but..
If you follow this practice, as Drupal.org testing profile does, how do you test changes to the .make file without blindly committing/pushing them?
Best suggestion?
My best suggestion is as follows:
1.) Have 2 repositories
2.) Have 2 stub make files (one for each repository)
3.) Commit to your dev repo, and build your dev make file until happy
4.) Commit changes to prod repo
Sorry, I know that's sub optimal, but since we're actually pulling the profile from a release or git repo itself, and it does all the real work, this kind of this weird dependency listing for this stuff... it'd be REALLY nice if there were some way to have a dev stub make that could be pointed at a local directory for the profile. I don't know of a way to do that, but it could be that I'm just ignorant. Should probably get one of the maintainers to comment on this topic.
Eclipse
Eclipse
In your
In your example
projects[drupalorg_testing][download][module] = "drupalorg_testing"I thought 'module' parameter in drush make it was for CVS only, not for git.
drupalorg_testing is supposed to be a folder? a module? a profile name ? what?
It is a module. That
Here is what I did to get a module from GIT:
http://www.radarearth.com
In this case it's actually an
In this case it's actually an install profile.
Eclipse
overloading make files
I guess that the real question is: Are makes a collect-then-process approach, or are they processed linearly as they are written?
It would be really nice to have it be the process be overloadable, as then I can override a profile, and not have to fork it to update a module that doesn't require heavy testing in my scenario, but may in the scope of the profile maintainer.