Please bring back install_path

some modules semantically do have natural place under "profiles", some - do not

What do you mean by this? Do you have any reasons or examples?

As for modifying so-called "out of the box" install profiles, there are two good solutions:

1. The old school way. Add your new/updated projects to sites/all where they will override the "suggestions" in the install profile. This has the downside of not allowing you to remove projects from the platform.

2. The new school way. Create a new profile that extends the "suggesting" install profile. We have two good mechanisms for this, drush make includes and Profiler base install profile extending. With these two you can effectively "override" Atrium, Open Publish, etc. with your own custom distro. This has the added benefit of encouraging a concise, readable, and documented record of what has been overridden and why, unlike freeform additions/modifications to a built distro's project files. Jeff has written about this specifically in why any deployed site, including ones based on an upstream install profile, should be described using a makefile (http://developmentseed.org/blog/2010/jul/27/drush-make-files-production-...).

I think we should encourage disciplined practices like these rather than enabling and encouraging sloppy maintenance and building.

I'm still not sure why these should live under sites/all? It sounds like you are trying to indicate to users that some modules are 'optional' and that putting them in sites/all is the proper way to do this. This doesn't make much sense to me -- there's nothing about sites/all that is inherently optional.

If anything I would simply put them in profiles/[x]/modules/optional which marks clearly that they differ from other modules in the distro as optional projects.

If you are serious about these modules being optional and being loosely integrated with your profile to the point where they can be messed around with by developers they should not be packaged or included with it at all. They can instead be placed in a distro makefile whose jurisdiction goes beyond what belongs to a specific install profile (e.g.)

core = 6.x
projects[] = drupal
projects[] = openpublish ; placed in profiles/openpublish
projects[] = opencalais ; placed in sites/all/modules

This type of makefile can't be packaged on Drupal.org but based on your use case you may want to request that they can be. The point here is not that projects shouldn't live in sites/all but that projects that are actually associated with your install profile really should live in your profile's code tree. As I've pointed out above, if there are projects that fall outside this requirement then there are ways to handle this.

Please refrain from ad hominem, especially given that I have described how you can use drush make to build the distro you described above. If this is truly a requirement of your use case, again, I believe it can be supported but with changes to Drupal.org packaging, not to drush make.

As for the actual design decision, the reason is very simple. Consider the following makefile.

core = 6.x
projects[] = drupal
projects[] = install_profile_a
projects[] = install_profile_b

If drush make were to allow projects to describe packaging outside of their jurisdiction, install_profile_a and install_profile_b could both place a project like Views in sites/all/modules. Suppose A uses Views 2.x and B uses Views 3.x. You have a conflict that cannot be resolved.

This is why a profile makefile should only describe the code that it has jurisdiction to as designed by the file scan limiting in drupal core.

I will leave the status of this issue here alone as it seems this is not mine to decide.

Woah, did not see this issue and it's looking like it's going pretty fast. I'd like to provide my take on this.

So, there are two reasons we don't have install_path anymore. The first is yhahn's #7. This *can* be smartly circumvented, by doing conditional detection, but that's a whole can of worms that I *raelly* don't want to open. The second is the fact that currently, however you build a makefile, the basic structure will be the same. You can build a makefile as a whole site with --contrib-destination=sites/all, or you can build it in a specific profile with --contrib-destination=. --no-core. Using one makefile, you can (pretty much) achieve the exact same result. With install_path, the second option is unattainable - say you're in profiles/X, how would it know to build it in sites/all? or a specific site?

This is a fixable problem, or at least the second part. yh and I have been talking about making drush make more contextually aware and all sorts of awesome, which would be great. But that's way down the pipeline, and our first goal is to get a stable release out. After that, I think this is an approachable and fixable problem in 3.x. Unfortunately, you and Crell both need this for the time being, so I'm not really sure what to say (I'd really like to give you a solution, but I'm just not sure how to do it in a clean way).

subscribing

I had opened another issue for this: #911722: Default path for projects, libraries

The standard I use for module locations is:

• sites/all/modules/contrib - all modules downloaded from d.o
• sites/all/modules/custom - custom modules which have not been written for redistribution outside of the organization (custom business logic, etc),
• sites/all/modules/features - modules built with Features.

Right now the only way to achieve this is to define [subdir] for every single module, which is a bit tedious. What I'd like to do is define a default which is then overridden (not appended to) by a per-project option so that I'm only overriding the path for a small subset of the modules used.

It would be /very/ useful to have a way of defining a default path for projects and libraries, e.g.:

paths[projects] = "sites/all/module/contrib"
paths[projects][theme] = "sites/all/themes"
paths[libraries] = "sites/all/libraries"

Right now the only way of doing this globally is the --contrib-destination argument. There are several problems with this:

• This has to be manually assigned every time "drush make" is executed, so it can't be baked into a standardized script.
• It doesn't differentiate between modules and themes, so everything is put in the same place.
• Any per-project overrides are nested within this, e.g. if you have:
  

  project[mywidget][subdir] = "custom"
  

  and you use the argument "--contrib-destination=sites/all/modules/contrib" then the mywidget module will be put in "sites/all/modules/contrib/custom".

The tedium of defining the same subdir over and over is a completely different and that's something that I do want to fix and am not sure how.

However, the --contrib-destination thing is something else.

The formula for determining the directory of a module looks like this.

[contrib-destination]/[project type (modules|themes|etc)]/[subdir]/[dirname]

(there is special casing for install profiles: profiles/[profile]/[subdir]/[dirname]).

so, setting contrib destination to sites/all/modules/contrib, it would put a module in sites/all/modules/contrib/modules/custom. Irakli's issue, if I understand correctly, is that he wants to install some modules in profiles/X and other modules in sites/all/modules. The problem I'm having with this approach is that currently, you can cd to sites/al (or, sites/default), and make with contrib-destination = ., and you can also cd to the site root and make with contrib-destination = sites/all and you'll get the same result. The downsid eof doing something about this is you remove the capability for the second scenario. so if someone were to distribute a makefile that contained a few modules and themes, and I wanted to bulid it in sites/drupal.org, but it was hardcoded to sites/all, I'd have to go into the makefile and manually change those. Or alternately there could be some other solution but it probably won't be easy/fun.

The tedium of defining the same subdir over and over is a completely different

In that case I'm going to re-open the other issue.

this *will* happen, after the 2.0 release

Lets get this done.

Subscribing, as my issue was marked a duplicate of this one. :-)

@dmitrig01 I'm in the same situation as irakli and am willing to write a solutions. I'm trying to figure out where the problem you highlighted in #8 comes into play. Where --contrib-destination=. --no-core is a problem.

If I do something like:

drush make --contrib-destination=. --no-core foo.make bar

The structure of what's created is:

- bar
-- profiles
--- My profile
---- modules
---- themes

If we add an install option that can be set to something like sites/all or sites/default we would end up with something like:

- bar
-- profiles
--- My profile
---- modules
---- theme
-- sites
---all
---- modules
---- themes

What is the problem I'm failing to see? If there isn't a major problem lets get this done.

@dmitrig01 Now that I'm in the code I think I see where the problem lies. I have a base make file that downloads core and an install profile. Then the install profile has a make file that does the rest of the work. This is similar to OA. In this case the --contrib-destination switch only seems to affect the location of the projects included in the base make file and not those in the profiles make file.

In situations where both --contrib-destination is set to a location and install_path are set and they conflict I would use --contrib-destination as the winner. Unless someone disagrees I'm going to roll a patch sometime early next week to add it (unless someone wants to beat me to it).

FYI, I'm leaving this on the 2.x branch because it should be trivial to add, reverse compatible, and needed in the short term.

Subscribe

Attached is a patch that makes this change.

Tried this patch out, have the same problems.

I'm working with COD (http://drupal.org/project/cod) and we have the same setup. There is a cod-dev.make which is the parent to cod.make.

When using:

$ drush make cod-dev.make [destination]

I'm given:

 - profiles
 -- cod
 ---- modules
 ---- themes

When using:

$ drush make --contrib-destination=sites/all/modules/cod cod-dev.make [destination]

I'm given the same thing.

After applying the patch and re-running the above command, I still have the same results.

Maybe I'm misunderstanding things here, but the modules that are downloaded need to go into /sites/all/modules... In the current location that drush make is putting them (/profiles/cod/modules) Drupal 7 installation simply doesn't pick them up and I'm left with a requirements problem.

@sirkitree I think you misunderstand this path. It allows you to specify the install path in the .make file. For example:

project[foo][install_path] = sites/all/modules

If this is in an install profile .make file the project foo will be placed in sites/all/modules rather than in profiles/[name]/modules. If --contrib-destination is specified the install_path is ignored.

If --contrib-destination is not working without install_path being in the loop it sounds like a different issue.

Ah, ok. Thank you for clarifying.

Verified that using something like project[foo][install_path] = sites/all/modules/contrib now works if I run something like $ drush make foo.make ../../location. However, if I utilize --contrib-destination like so: $ drush make --contrib-destination=sites/all/modules/whoa foo.make ../../location, modules ended up being placed in /profiles/foo/modules :(

Note the following for libraries:

; Libraries
libraries[jquery_ui][download][type] = "get"
libraries[jquery_ui][download][url] = "http://jquery-ui.googlecode.com/files/jquery.ui-1.6.zip"
libraries[jquery_ui][install_path] = "sites/all/modules/contrib"
libraries[jquery_ui][destination] = "jquery_ui"
libraries[jquery_ui][directory_name] = "jquery.ui"

There's something just a little funky going on with the path too now that I notice.

I have something like this in my .make file:

;jQuery UI
projects[jquery_ui][install_path] = sites/all/modules/contrib

; Theme
projects[fusion][type] = theme
projects[fusion][download][type] = git
projects[fusion][download][url] = "http://git.drupal.org/project/fusion.git"
projects[fusion][download][revision] = "master"
projects[fusion][install_path] = sites/all/themes

But the actual placement for the module and the theme ended up being this after make was run (without --contrib-destination):

$ find cod-7 -name jquery_ui
cod-7/sites/all/modules/contrib/modules/jquery_ui

$ find cod-7 -name fusion
cod-7/sites/all/themes/themes/fusion

So it seems that there is an extra "/module" and "/theme" being appended to this somehow.

@sirkitree Thanks for catching the but in the patch and with --contrib-destination. We should connect with dimitri on the latter one. It could be easily fixed in this patch as well.

I just found this issue by accident and wanted to share my own experiences. I do not mean to distract or derail this thread but I think my experience may be useful.

It seems to me that the OP is seeing a problem where there isn't any. I think yhahn's comments about "new school way" are important. While I appreciate that if someone wants to have some modules in sites/all/ and some in profiles/[x]/, all the power to them, I believe this is a pattern that should be avoided and discouraged.

Ever since I switched to a distro oriented development approach for all our new projects the sites/all directory is no longer used.... ever. After reading through this thread I still can't think of any reason to use it.

Sometimes the profile we're running is "stock Open Atrium", it could also be "stock Open Publish". It could also be "customized Open Atrium" or "customized Open Publish". Regardless the primary tools to build the profile is a .make file. If it's a customized profile we'll use includes[] = "stock make file" and then extend it from there. But all the contrib modules, themes and libraries are part of the profile and live in profiles/[x]/.

Just my 2 cents. take from it what you will :)

langworthy: That's fine if you're doing a distribution. If you just want an automated way to get a bunch of modules to kickstart your site, that's not a distribution and profiles/x is, I would argue, quite wrong in that case.

Crell: I wonder if there is a better place for this discussion? That being said I'm interested in hearing why you would argue it would be wrong to build out a site in such a way. I'm actually working on a blog post that discusses doing just that. I hope for it to show up on the Planet feed in a few days. We've been building all our new custom site builds as if they were distributions and we love it. We've found it's improved our development velocity, decreased maintenance time, and overall just made for a more enjoyable experience.

[ Powered by #1115636: Issue Macros and Templates - Drush Make]

Drush make is being merged into drush core (discussed in issue:#1310130: Put drush make in drush core)
This means that the issue queue is also moving. The Drush project has a component called 'Make' for this purpose.

We would like to take this opportunity to leave behind old/obsolete issues, allowing us to focus on a stable make command in core. E.g. one of the major tasks ahead is making more use of the Drush core code for handling downloads and decompression.

If you feel that this issue is still relevant, feel free to re-open and move it to the Drush queue.

More information will be posted on the drush_make and drush project pages.

NOTE: If it's important for modules to live outside the profiles directory, then why not add them to a build.make which pulls in the profile via recursion.... Just a quick thought.

Seems this is still potentially relevant and was prematurely closed as drush make moved into drush core... Re-opening to continue the discussion.

I hate these two-phase .make file workflows for distros. That seems like a crappy solution to this problem. I'd rather distros never had to do that. I'd vastly prefer you could just tell drush make where you want your stuff packaged (without tons of tedious repetition in your .make files).

Hey, so to add another twist in this story, in the very first comment on this thread the "new school way" referenced the Profiler module.

There is an issue over there about needing to duplicate modules in child profiles #970980: Child profiles needs to duplicate module files.

Basically, if you have a base profile with all the modules you want to re-use in the general daily course of running a Drupal shop - like Views, and you want to have a sub-profile that can contain all customisations for a particular project, you can't do it without dropping all the modules from the base profile in the sub profile as well, at which point I wonder how much I'm actually achieving by separating the two out beyond setting different dependencies and maybe a few other install scripts.

What I really want is to be able to have a version controlled base profile that all my developers can reference and a concise and easy to decipher description of what makes each of our projects unique.

The maintainer of Profiler, q0rban, suggested the following:

Yeah, I'm not sure here. If you can get the modules into sites/all/modules instead, you can circumvent this problem. Maybe modify the drush_make file to specify the directory you'd like the modules to be downloaded to?

The suggested "right way" to do things by yhahn doesn't actually work at all unless you have a way that sites installed in the sub-profile can "see" modules/themes installed in the base profile, which it can't if they're in /profiles/base_profile/modules.

I can only see three ways to achieve yhahn's (and my) desired way of working with profiles:

1. kill some kittens with symlinks pulling modules from each profile into the site's directory - breaks the "specificity" of how drupal scans for modules by putting everything in the one (wrong) place
2. allow a make file for a profile that is only intended to be a "base" profile to define that its modules are placed in sites/all so child profiles can see them - the issues with conflicting version numbers would have to be resolved somehow, unless you could mimic the concept of "base" and "child" in the makefiles too
3. get some noise around the issue so we could work to get the (great) idea of sub-profiles out of the Profiler module and into core

am i missing something, or is this pretty much the current situation?

helmo's suggestion in #35 is probably the only viable workaround at this point, but it really undermines the usefulness of base profiles as defined by Profiler :(

I was pretty close to the "wanting to punch my monitor" level of frustration, but then I figured it out.

Background: I'm building a simple installation profile for my agency that simplifies the task of getting a Drupal site running. Our sites differ pretty radically between clients, so the resulting install will not be some monolithic thing pre-defined within the makefiles. We use drush pm-download. We like drush pm-download. We will spill blood to continue using our shiny happy drush pm-download! Really, I honestly tried the whole profiles/*/modules thing, but then I got irritated when using drush pm-download to download a module already in the profile module directory -- not only does the existing module not get replaced as it would normally, but a brand new version gets downloaded to sites/all. I had like 4 versions of Views floating around a site the other day and decided this madness must stop.

1. In your parent makefile, build it as normal and have it download Drupal, and your profile as normal.
2. In your profile makefile (I.e., the one downloaded by the parent/install makefile), just put any custom bits of code that might make sense to actually put in profiles/*/modules. Also put your instructions to grab profiler if you enjoy doing things without having to unnecessarily write a bunch of PHP.
3. Do your .profile and .info files as normal. Put your profile makefile, your .profile and .info (Hereafter "the Profile Stuff") into a zip or git branch or where-ever you tell the parent/install makefile to grab it from.
4. AND THIS IS KEY -- put all the contrib modules, libraries, themes, etc. going in sites/all into your parent/install makefile after you tell it to download your Profile Stuff.

And then, good sirs -- VICTORY SHALL BE YOURS!

Ahem. I'm excited.

Aside: given that the above methodology allows contribs to be in both sites/all/modules and profiles/profilename/modules, without having to patch anything, and leaving intact the default functionality if you really, really want to put everything in profiles/profilename/modules, can we finally close this issue? Or is there something major I'm totally missing?

(Also, changed the version because it wouldn't let me post otherwise; hope that's okay...)

Yes, it means that you have to edit your base install makefile when your profile gets updated. That's a hack, not a solution. It may be a working hack, but it's still a hack.

@Crell -- I just read your issue at http://drupal.org/node/900990 and better understand where you're coming from now. Would it be possible to use "--contrib-destination sites" and then set subdir to "all/modules" and "default/modules" for stuff you want in each place?

@dmitrig01 et all -- Regardless, if I'm understanding #12 correctly, the big problem with re-adding an install_path-like feature is that it would confuse things with --contrib-destination? In short, it wouldn't know whether to set a relative path or an absolute path? If this is the case, why not add to subdir, such that paths beginning with "/" are always calculated from the level that the makefile itself is on, irrespective of whatever --contrib-destination is set to?

This issue was marked closed (won't fix) because Drush has moved to Github.

If desired, you may copy this task to our Github project and then post a link here to the new issue. Please also change the status of this issue to closed (duplicate).

Please ask support questions on Drupal Answers.