Since Drupal's composer.json file is in the root directory (which is far better than it being in the core directory), the question has been raised: Is it acceptable to modify Drupal's composer.json file?

This is an important question, however, it appears that the Drupal best practice of never modifying core still remains true.

In naderman & RobLoach's presentation at DrupalCon Munich, it was recommended that Drupal core become a dependency of Drupal. This would allow end-users to modify Drupal's composer.json file, while at the same time maintaining Drupal core's own composer.json file that would not be touched.

This setup is similar to the architecture imposed by Symfony and Symfony Standard.

For this to happen, it will require some major changes to how Drupal is developed.

Drupal core will be in it's own repository with it's own composer.json file.
I propose the composer.json name be:
drupal/core
This would also change the type from drupal-core to the default library.

Lastly a new repository will be created with the drupal directory structure (excluding core).
I propose the composer.json name be:
drupal/drupal
The composer.json file would have only have one requirement: drupal/drupal-core

This would be immensely helpful for developers as they'd be able to use Drupal's root composer file to manage the dependencies of the project, without affecting Drupal core.

The end user wouldn't see much of a difference, the packing script would need to be modified to run

composer create-project drupal/drupal;
cd project;
composer install;

and then package project into the zip file that is downloadable on drupal.org.

Doing this would make the change transparent to most Drupal users and provide all module developers with an immense amount of flexibility.

Thanks!

Files: 
CommentFileSizeAuthor
#24 1975220_24.patch395 bytesMile23
PASSED: [[SimpleTest]]: [PHP 5.4 MySQL] 66,463 pass(es).
[ View ]
#6 core-composer.patch1.76 KBRobLoach
PASSED: [[SimpleTest]]: [MySQL] 57,119 pass(es).
[ View ]

Comments

Category:feature» bug

Marking this issue as a bug report since it's impossible for module developers to use composer.json without breaking Drupal's best practices.

Thanks!

Title:Make Drupal Core a Dependency of DrupalIntroduce a new core/composer.json file

We moved the composer.json to the project root, so this would be a new composer.json file at core/composer.json, it seems.

We'll need to add the "drupal-core" type to Composer Installers at https://github.com/composer/installers/blob/master/src/Composer/Installe... .

So, it might look something like this:

Root Composer.json

name: "drupal/drupal"
type: Nothing

core/composer.json

name: "drupal/drupal-core"
type: "drupal-core"

Why would drupal core be something other than a library? Isn't that what Drupal core is?

Thanks!

The only reason to have a type:drupal-core is to allow having a custom installer that will install the code in core/ instead of vendor/drupal/drupal-core.

While I can understand why you'd want to do that for familiarity reasons, I still think it's a bad idea. It adds complexity and an unnecessary custom installer to the mix.

If you're gonna have a vendor dir anyway then you might as well tell people in your migration guide that the core dir is now under vendor/drupal/core (I'd also advocate using just drupal/core as package name, no need to repeat yourself IMO).

Good idea!

I updated the recommended core project name.

I think putting Drupal in the vendor directory is a good idea, but I'm not sure if anyone would have any objection to that, but I can't think of why that would cause a problem.

Thanks!

Status:Active» Needs review
StatusFileSize
new1.76 KB
PASSED: [[SimpleTest]]: [MySQL] 57,119 pass(es).
[ View ]

Sounds like a great idea, there are a lot of things that would need to change in order to have core path-independent. In the mean time, this change takes a step in the right direction. Mimics Symfony's use.

Issue tags:+composer

I looked at your patch @RobLoach. I like the direction it's going, but I think that could get messy, because what happens if someone modifies the composer.json file in root and then runs `composer update`? As far as I can tell, that's going to override whatever is in the core folder.

I think ideally what would happen is that the packages listed on the root level is just drupal... or maybe (at first) it's nothing at all. Just an empty file ready for users/modules to customize. I guess then we need to decide where we want 3rd party vendor libraries to be installed. We could just create a custom installer to put them in core/vendor I suppose.

This is similar to the way Symfony does it. If you look at the Symfony composer.json file, it lists out completly different packages then the Symfony Standard composer.json file:
https://github.com/symfony/symfony/blob/master/composer.json
https://github.com/symfony/symfony-standard/blob/master/composer.json

So basically everything that Drupal requires should be in core/composer.json. Which would effectively leave composer.json blank.

if we do list out "drupal/core", I think, in our custom installer we could also run or just ignore core's composer.json at first (since it's in the repository already).

I think, we shouldn't really rely on using Composer Installers to install Drupal core (since there is only a single project that will take that type). I think it would be better to add a custom installer that will put it in the right folder.

so then it would look like this:

root composer.json
name: "drupal/drupal"
type: "drupal"

core/composer.json
name: "drupal/core"
type: "library"

However, if we can't override what happens to "drupal/core" on install (with a custom installer). I'd be in favor of changing the type, but not just to use composer installers (for a single project). If there was going to be more than one project with that type then I'd be in favor of it, but that doesn't seem to be the case.

For reference, I discussed this here:
https://github.com/composer/installers/pull/38

Thanks!

Priority:Normal» Critical

I'm going to make this critical, because currently in Drupal 8 there is no standard way for contrib developers to work with dependencies other than the ones Drupal has provided in core. This is a major issue if contrib developers are to mimic the way core modules are created. Without this being resolved some how contrib authors are forced to develop modules in the exact same "Drupal-way" as Drupal 7.

Thanks!

+++ b/composer.jsonundefined
@@ -29,6 +29,9 @@
     "vendor-dir": "core/vendor",

Wouldn't this become a problem if you ran composer update?

Wouldn't it attempt to install drupal into the core/vendor folder?

Status:Needs review» Postponed (maintainer needs more info)

Can someone explain why this is:

1. A release blocker?

2. A bug report rather than a task?

It's not clear from the issue summary exactly what use-case currently isn't possible, looks to me like it's the ability to put arbitrary dependencies in core/vendor but wouldn't it be possible to ship a composer.json file with a contrib/custom module and have a vendor directory there instead?

@cosmicdreams,

I'm not sure I quit understand what you are saying. Are you asking about the proposed solution or what is currently in Drupal?

Thanks!

Status:Postponed (maintainer needs more info)» Active

@catch,

Let me see if I can clear some things up a bit.

The use-case is a developer who wants to utilize 3rd-party libraries, much in the same way Drupal core has done. There are several current solutions in Drupal 7 to do this, Libraries API is by far the most popular. However, it doesn't really manage dependencies all that well, or rather, not as well as Composer, which I assume is why Drupal core uses Composer.

Drupal 8 utilizes Composer to manage it's dependencies. This is an amazing step forward. Using a dependency manager not only makes it easy to update 3rd-party libraries, but it also prevents version conflicts between dependencies.

If you are familiar with composer, you'll know that a project has a "root" composer.json file, and every dependency has it's own composer.json file. You are correct, that a module should have it's own composer.json file with it's own dependencies. However, according to the way Composer works, I would need to list that module in the root composer.json file. Once I listed it in the root composer file, I could run `composer install` and all of the module's dependencies would be installed into the root vendor directory.

Let's say though, that we don't want site builders to modify composer.json, then, in your scenario, the site builder would have to `cd` into every module directory and run `composer install`. However, this presents a huge problem because the dependencies are no longer being managed across projects. This makes version conflicts a real possibility. Also, most site builders are not going to go through the hassle of doing that to update the dependencies of each and every module in a project, which introduces security concerns.

Ideally, Drupal Core should be a dependency of Drupal. We would be saying "Drupal requires Drupal Core". That way there is only 1 item in the root composer.json file which would be "drupal/core". Just like .htaccess and .gitignore, the composer.json file should be able to be modified by site builders and modules.

There is a stop gap in contrib called Composer Manager. This module looks through all of the modules for a composer.json file and generates a "root" composer.json file. I'm not sure what the plan is for this module though and how it will work with Drupal 8's composer.json file without causing versioning conflicts. For instance, what if someone requires a different version of guzzle? etc. Composer is already made to handle these kinds of conflicts, but right now, it doesn't look like that is the strategy that Drupal is taking.

So I think that we need to 1) Determine the best way for developer's and site builders to utilize composer.json. I believe this includes some fairly major reorganizing of the project, however, maybe it does not. and 2) Communicate that preferred method to developers and site builders.

I made this a release blocking issue, because depending on the solution to the problem, it ought to be figured out before telling developers how to use 3rd-party libraries.

I made this a bug rather than a task, because right now, there is NO way to properly manage 3rd-party libraries in Drupal contrib. Without a fix of some kind (which may just be documentation), then contrib developers are blocked from doing this properly.

I hope this answers most of your questions.

Thanks!

Category:bug» task

OK that's a task - it needs to be figured out but there's no functional bug. But I'm OK leaving this at critical to attract some more discussion - it does need to be sorted out one way or the other even if the answer isn't to change anything.

awesome. sounds like a plan. :)

Thanks!

Category:task» bug
Status:Active» Needs work

having a vendor directory in each module is really an even worse solution than using libraries module.
correcting status according to #8

Category:bug» task

xpost...

I agree.

Ideally there needs to be a single vendor directory.
Ideally that vendor directory would be in the root of Drupal.
Ideally Core would be inside of that vendor directory (vendor/drupal/core).

Thanks!

davidwbarrett++ well said

I should add one more:

Ideally the vendor directory should not be included in the repository. This means Drupal Core would be a separate repository.

Well this DEFINITELY sounds like something that needs to be talked through.

Is it necessary that drupal/core be placed into the vendor directory? Is this how other projects that use composer work?

I suppose we have to consider two use cases:

Drupal projects

Projects where a Drupal site needs to be built out. From this perspective, we currently expect to find a core directory that contains all of drupal including all of it's dependencies. What you seem to be saying is that we could allow composer to install drupal into /vendor/drupal/core and have all of it's dependencies install in /vendor.

In my opinion that would be weird for a while but I'd ultimately get used to it.

Projects that include Drupal through composer

Making these changes for Drupal projects would seem to make it easier to include Drupal into the mix for projects that want to interact with it. Drupal would be installed by composer into the vendor folder for this other project and it's OO code would autoload.

I can see why it's important not to include drupal's dependencies in the same folder of Drupal Core now. PSR-0 is supposed to allow more freedom in where the code is placed within a file system. I suspect the main motivation for including the vendor folder in /core is to ensure all of drupal is the core folder so we can say "Don't touch that" to site administratrators/builders.

@cosmicdreams,

correct. What I'm thinking is that it should work something like the way Symfony Standard and Symfony work.

Syfony Standard is to Drupal, what Symfony is to Drupal Core.

Also notice that when you Download Symfony. By default, you can download a package. The package is Syfony Standard with all of the dependencies (including Symfony itself) already loaded for you (no composer needed).

Doing it this way also lets you install Syfony Standard with the following command:

php composer.phar create-project symfony/framework-standard-edition path/ 2.3.5

Ideally, I think this is how it should work. It would provide the simplist and cleanest solution for Drupal developers and site builders. However, it would take some work to modify how Drupal Core developers work with Drupal. It would also take writing a script that would run `composer install` whenever a release is created.

Thanks!

Issue summary:View changes

Update the recommended core project name, as per Seldaek's suggestion.

Status:Needs work» Needs review
StatusFileSize
new395 bytes
PASSED: [[SimpleTest]]: [PHP 5.4 MySQL] 66,463 pass(es).
[ View ]

OK, we have two goals:

  1. Let people download Drupal and then start using it without learning any Composer at all.
  2. Let Composer-literate people do installations through Composer.

As it turns out, this is very easy to accomplish.

First we apply attached patch.

Second, we make a listing on packagist.org for drupal/drupal.

Third, we type composer create-project drupal/drupal [some path]

There is no fourth step.

The non-Composer users can just download the package.

The Composer users get Composer to do it for them, and also get version bumps from dependencies if they're available. Because Drupal's composer.json uses vendor-dir in it's config section, Composer will update that directory.

That's all there is to it.

Status:Needs review» Active

Mile23,

I'm not sure where you got those goals from. Yes, I think it's important to be able to install Drupal through composer, however, your patch does not answer the question raised in the initial post:

Is it acceptable to modify Drupal's composer.json file?

The patch you've submitted does not address the purpose of this issue, which is to allow a user-editable composer.json file in the root of Drupal.

Thanks!

I blame blood sugar. :-)