Problem/Motivation

Five years ago some of us bravely put forth adding metadata files to Drupal modules in #80952: Add .info files to modules to store meta information. Today, tons of problems are trying to be solved by .info files in a Drupal specific way. It'd would hopefully lower a lot of core developer workload and provide a better solution if we could consider an external standard like composer.json

Proposed resolution

With Drupal 8's growing use of NIH solutions I thought I should propose another solution for this problem developed by the php community, the composer.json file. It has places for I believe all of the features requested to be added to .info files, appears to support all the fields we add as part of our packaging, has some nice features like PSR-0 that could help solve other problems we haven't encountered yet, and has a nice "extra" field we could stash our custom things like CSS files in themes, and best of all is not a Drupalism!

Key ideas:
Composer.json is a format outlined in this schema:
https://github.com/composer/composer/blob/master/doc/composer-schema.json

Composer is also a PHP PSR-0 application for resolving dependencies generally used as a phar archive:
https://github.com/composer/composer

Packagist is a solr + symfony application serving http://packagist.org/ much like our project.module
https://github.com/composer/packagist

Pluses

  • Its could mean engadgement of more people in the php community outside of d.o(adding drupal support for you code might be as easy as droping a line in your composer.json)
  • NIH
  • Composer is a very featureful package management system based on npm and suse
  • Tests we don't have to maintain! http://travis-ci.org/#!/composer/composer
  • Likely better drush support through things like the "repositories" entry and being able to reference libraries outside the drupal world for downloading.
  • It would deprecate, or at least significantly improve the maintainability of project.module.
  • A lot of people know json

Minuses

  • The composer resolver does contain a lot of code
  • Its JSON and CMI at this point has chosen to go with XML. Configuration is not definition though so this should be a minimal overhead. drupal.org could also probably automate a lot of the management if we're smart.
  • JSON has no commenting like

    , // or /* */.

  • JSON is very strict?(maybe a good thing?)

Related Issues

Remaining tasks

Code review of composer.json

User interface ch0anges

Not really a UI thing other then implementing new features we'll be able to support.

API changes

How a .info file would be translated to a composer.json file:

name = Really Neat Widget
Composer's name is the canonical name of the package. This means that we move the package definition in for it, prefixed by drupal/. The human-readable name is moved to title, as sun mentions below:
"name": "drupal/reallyneatwidget",
"title": "Really Neat Widget"
description = Provides a really neat widget for your site's sidebar.
"description": "Provides a really neat widget for your site's sidebar."
core = 7.x
This is moved to "require". See note about dependencies below...
"require": {
  "drupal/drupal": "7.*"
}
stylesheets[all][] = node.css
composer.json files can hold "extra" meta-information, which would be used for stylesheets.
"extra": {
  "drupal": {
    "stylesheets": {
      "all": ["node.css"]
    }
  }
}
scripts[] = somescript.js
"extra": {
  "drupal": {
    "scripts": ["somescript.js"]
  }
}
files[] = example.test
Since #1541674: Remove the registry, we no longer need to define files[]. Simply removing the entry will fix it for a composer.json. If you're looking to do this in a Drupal 7 composer.json file though, Composer supports autoloading straight up files:
{
  "autoload": {
    "files": ["example.test"]
  }
}
dependencies[] = taxonomy
dependencies[] = exampleapi (>7.x-1.5)

Dependencies are moved over to "require".
"require": {
  "drupal/taxonomy": "7.*",
  "drupal/exampleapi": ">1.5"
}

Having Drupal packages declare their Drupal-version dependency within the version number impedes our ability to move fully towards this architecture. #1612910: Switch to Semantic Versioning for Drupal contrib extenstions (modules, themes, etc) would allow this. One intermediate solution around this would be to use the "extra" options for Drupal-specific dependencies:
"extra": {
  "drupal": {
    "dependencies": {
      "taxonomy": "7.*",
      "exampleapi": ">7.x-1.5"
    }
  }
}
package = Views
"extra": {
  "drupal": {
    "package": "Views"
  }
}

php = 5.3
"require": {
  "php": ">=5.3"
}
version = 7.x-1.5
"version": "1.5"
Due to Drupal's versioning system, we have issues moving to this version architecture. #1612910: Switch to Semantic Versioning for Drupal contrib extenstions (modules, themes, etc) would fix that. An intermediate solution could be...
"version": "1.5",
"extra": {
  "drupal": {
    "version": "7.x-1.5"
  }
}
configure = admin/config/content/example
"extra": {
  "drupal": {
    "configure": "admin/config/content/example"
  }
}
required = TRUE
"extra": {
  "drupal": {
    "required": true
  }
}
hidden = TRUE
"extra": {
  "drupal": {
    "hidden": true
  }
}
project = "views"
"extra": {
  "drupal": {
    "project": "views"
  }
}

As you can see, most of it would be moved to the "extra" information of composer.json. One of the issues is the Drupal version being part of the packaged version information. #1612910: Switch to Semantic Versioning for Drupal contrib extenstions (modules, themes, etc) and #586146: [policy, no patch] Decide if and how to implement semantic versioning for Drupal 8.x would help, but there are ways we can get around it in the mean time, as I outlined above. Here's an example of system.info as a composer.json file:

{
  "name": "drupal/system",
  "title": "System",
  "type": "drupal-module",
  "description": "Handles general site configuration for administrators.",
  "extra": {
    "drupal": {
      "package": "Core",
      "version": "VERSION",
      "required": true,
      "configure": "admin/config/system"
    }
  }
}
Files: 
CommentFileSizeAuthor
#30 screenshot2.png108.16 KBRobLoach

Comments

Another bonus would be thinks like "recommends" and "suggests" in addition to "depends". What we do with that is an open question, but it's already there in the format waiting for us to leverage, and we didn't have to do any work for it. I like that.

I really like this idea.

Composer's json file is saner than pear's package.xml and it's much closer to what we've already done in .info files. We could leverage this in drush to improve our package management and it could potentially allow us to shed a lot of code.

Also, I don't see JSON as being in conflict with CMI's XML. Choosing a standard format and packaging system is unrelated IMO.

Downside though: JSON is harder to read with the naked eye; also it doesn't allow comments in its original spec at least.

I don't find this json harder to read than a .info file. True about comments though...

https://github.com/symfony/symfony/blob/master/src/Symfony/Component/Bro...

Another advantage is that IDE's add pretty colors to the mix, which we currently don't have for our .info files.

In other words, big +1 :)

In a project where I used JSON for something I effectively extended the standard by simply stripping out lines starting with '//' before passing it to drupal_parse_json(). We could consider doing that here.

> I don't find this json harder to read than a .info file.

The quote marks and the { } make it noisier, I find. It adds a little bit of extra programmy-scariness for non-developers.

One could ask why a non-developer would need to open a .info file.

I'm all for it, very nice feature.

> One could ask why a non-developer would need to open a .info file.

Themers need to set regions.
We should consider getting some themers to look at this proposed change.

Issue summary:View changes

Add some related issues

Everyone, this is not a thread about file formats. It is a thread about packaging and the benefits of leveraging existing packaging systems.

Comments is an interesting observation but I personally think the schema is very self documenting. We can not under any circumstances put random bits that need to be stripped before processing. This would be yet another drupalism that would completely defeat the point of using a standard. Even so, I've updated minuses in the summary to note the lack of comments since its technically true.

I don't disagree that we should have lots of people review this including themers, we shouldn't push a format on the community without review. I think the usability is vastly overstated though.

Consider that almost everywhere else on the web these days "non-developers" are using js if not json itself. A themer that hasn't done at least a little jQuery or javascript is probably going to be struggling with Drupal anyways. Also, you can easily google and find information about both json and composer.json that's not drupal specific. The community of knowledge available to new users will be larger since its not just the Drupal community, its also the web.

The INI format on the other hand is mostly only used for ancient unix config files and weird bits of Windows normal people don't touch. But if we where really using the format we could maybe have syntax highlighting but we don't even use the actual ini format so normal parsers can't parse it. Its weird fragile Drupal magic.

So for non-developers(and developers) you're getting syntax highlighting and checking in your editor, increased documentation through the broader community of the web, and just a general empowerment to use the tools of their trade. I personally believe we commonly underestimate the common designer and themer(at least the one's I've met) and over complicate their experience by building "easier" drupalisms to support them. I think this issue would fix one of those cases.

A final important point, if I can be so bold as to lead this discussion a little, lets not start discussing the ease of other formats. CMI bikesheded that for months and you're welcome to read it. The real benefit is as @msonnabaum noted that this is a larger php community format with a package standards that will empower the more common user of the format, developers. If the format is acceptably usable for the community, the real discussion should be the merits of the package management.

> Comments is an interesting observation but I personally think the schema is very self documenting.

; Information added by drupal.org packaging script on 2011-04-28

> Comments is an interesting observation but I personally think the schema is very self documenting.
; Information added by drupal.org packaging script on 2011-04-28

https://github.com/composer/composer/blob/master/doc/composer-schema.jso...

Issue summary:View changes

Note JSON's lack of commenting.

Updated summary with some possible pain points brought up in #drupal-contribute discussion.

Issue summary:View changes

Document some possible pain points brought up in #drupal-contribute discussion

Issue summary:View changes

use the correct related module issue.

We could support both .info and composer.json at first, if we choose to go this way: it's doable I guess since in the end, the module ends up as a single line in system table. The only remaining problem would be to chose either one of the formats for the serialized info field, or drop it definitely since it's memory waste to load it for anything else than pure module/package/dependency management.

Component:system.module» base system

Now.

If you'd replace that JSON format with YAML, then that would actually make sense.

The YAML spec natively allows for comments and also supports complex data types (arrays ["sequences"], and hashmaps (associative arrays) ["mappings"]).

There are plenty of YAML libraries for PHP, but also for other languages (as partially listed on its homepage).

There's a PECL extension, which means you could perform the parsing (and also writing) in C code.

A short introduction to YAML can be found here (coincidentally by the author of Symfony). It basically has all the pros and none of the cons.

Now, that would make a lot more sense.

(I don't really care for Composer / packagist.org -- not even the OP clarifies why that is relevant in any way in the first place.)

The only downside I can see is that all of the user-space libraries suffer from the PCB syndrome (including Symfony's, which scores and possibly wins among all with ~45 KB in total, which is inane).

To put things into perspective, that PCB syndrome is way worse than the NIH syndrome. I'm a pragmatic realist, and so contrary to others, and despite all of the PSR-0/framework-ification/etc efforts, I do neither foresee nor expect Drupal to load less code anytime soon (rather more and more). Apparently, the sheer amount of code we're loading is one of the topmost performance problems we have.

Thus, even if the proposal was YAML, this is a very tough decision to make.

In total, however, count me -1 on JSON (and composer.json). If we replace the .info format with something else, then my new benchmark is YAML.

The OP has a lengthy list of the pros of moving to composer.

If you mean Packgist in particular, I don't anticipate Drupal code living there as a matter of course. However, it does mean we could run the same code base as Packagist, or slight variation thereof, rather than a custom (Drupal-proprietary) metadata file, server-side system, and XML-based format.

As a side benefit, it means pulling in additional 3rd party libraries as a contrib module dependency are using the same system we are.

JSON vs. YAML vs. XML is not the key issue. They key issue is less Drupal-proprietary code that we have to maintain ourselves, and that no existing developers outside Drupal comprehend.

I dont get why file formats are being discussed.

We're talking about leveraging composer, a package and dependency manager. We can argue that composer is not a good fit, but it's insane to discount everything composer buys us because the file format is json.

Composer overlaps with .info, project module, and even drush make. It's not just a different file format, it's a huge improvement in how we manage and think about dependencies. THis would make it trivial for a contrib project to declare a dependency on an external library and provide autoload information for it.

Composer is clearly influenced by node's npm and ruby's Budler, which I find to be a pleasure to use whenever I work in node or ruby. Both those systems make coming back to Drupal/php feel like we're years behind in how we share and leverage code outside the project.

Sorry, this apparently wasn't properly communicated from an earlier IRC discussion into this issue:

The file format is an important issue, because there are things like especially theme .info files, which are authored and edited by people, who may not be technically versed.

It is in our best interest to have and use a format that is "forgiving" about minor inaccuracies. JSON apparently is the exact opposite of that; it's ridiculously rigid, and comes with a plethora of special cases, which not even most technically versed developers understand.

In addition, it is very common to document special or otherwise noteworthy conditions in an extension's definition through comments, especially in contrib.

Composer appears to only support JSON, so the format implicitly becomes a central issue.

I understand your point, I'm just saying we should discuss this without worrying about the file format until we have a better idea of how composer could be used within the project. Once we do, we can weigh all the potential benefits against the file format issue.

> They key issue is less Drupal-proprietary code that we have to maintain ourselves, and that no existing developers outside Drupal comprehend.

I'm sure that's a goal we're all behind.
But in the range of ungrokable concepts and unmaintainable code, info files and their parsing is surely pretty low down. Let's keep a sense of perspective here.

And at least from the user point of view, YAML looks much better. Let's not forget the primary audience of code is humans.

Heya. Being one of the devs of Composer and Packagist I'll try to address some of the concerns.

First of all if you are not too familiar with Composer please take some time and read my two blog posts, we are still missing docs but that hopefully enlightens you a bit.

As noted in #17, the main benefit I see to something like Drupal using Composer, beyond the fact you'll have less code to maintain, is that your modules (or any contrib stuff) can start depending on third party library transparently, without having to bundle them or anything. That means "reconciling" the Drupal and PHP communities to me, which sounds great.

Now the way I see it, a project like Drupal would typically embed/wrap Composer as a library rather than use it on the command line, because your users are used to interact with that stuff in a web interface, and I assume many also don't have CLI access anyway. What this means is you are much more flexible and you can extend/override stuff from Composer. Should the file format really be considered a major issue, you can add another loader that reads Yaml, .info files or even from the DB, and let Composer work with that. That said, I would really advise against it because I think having one file format for packages is just much more straightforward (everyone knows it and easily spots a composer.json) than if there are many formats around. That is the main reason we only support JSON. It makes integrations quite easy, for example knpbundles.com listing Symfony2 Bundles started recognizing composer.json and linking back to Packagist automatically. If they had to check for arbitrary files it would make it more complicated.

I hope this helps move the discussion forward.

P.S. @neclimdul: Composer is not -as is written in the proposal above- a Symfony application. It uses Symony's Console component to provide CLI interaction, but that's about it.

Just so people don't take it as a given that YAML > JSON: I actually detest YAML, and don't consider it more readable at all. I also rather like XML, as long as it's a well designed schema. Let's not pretend that there's a universal consensus that JSON is bad. There isn't.

Agree, and Seldaek is right about one particular detail: using composer.json would allow us to handle third party dependencies; Even thought I'm still unsure it would reconcile us with the PHP community: Drupal is too far away from it since too long, it would need more than just that.

Component:base system» system.module

@seldaek Oh my bad on the Symfony's app thing. That's what I'd heard and I should have looked into it more. Thanks for all the other bits and I agree about your points about the format.

I think there are at least 2 other strikes against YAML though other then not being the standard.
1) YAML is really really slow[1]
2) We can't really count on the PECL module being installed and we already require JSON support[2]

heyrocker also noted in his CMI review that its fragile and easy to break which isn't really a selling point either.

So we're really hung up on this format thing but I think we're missing the point. First off, these files are specifically for driving code. We're building module pages, class autloaders and project hosting solutions. This is specifically /for/ developers. Themers are just going to copy and paste and change a couple of lines 99% of the time. Heck, I have always done that with .info files and I helped come up with the original standard! Whether its Drupal info, YAML, XML or JSON, that is going to be the case.

So, can we discussion the merits of sharing a package standard with the rest of the world?

I've got this dream that made me post this issue. In it, we don't need a "module" as we see it now for PHP libraries using composer.json because we just require composer.json and it registers the namespace with our autoloader. In my dream I can point Drush at a composer.json file on the web in any repository and it can install it for my site. Or it can even download bundled parts of a module hosted externally for licensing reasons.

I'd also like to not have to design, implement and support all those features in the OP when someone already did plus more. This is not NIH or PCB this is empowering Drupal to do more with less resources and make use of an even larger community. I also have a dream where I can walk into my local PHP users group and talk package management without them looking like I'm an idiot. That might be a long shot but this is my dream. Can we discuss that dream already?

My general feeling is that we need to focus on our core competency: Building a first-class content management framework that kicks ass. Server-managed package management is not part of that core competency. We struggle to get people to work on infrastructure tasks, in part because a lot of it is very specific Drupal-proprietary code. That seems a perfectly good thing for us to outsource and focus our ever-limited engineering resources on our core compentency, making the Drupal framlication rock.

Standard or emerging-standard tools that let us do that (and thus maintain less code ourselves) are therefore a good thing by default, in my view. Obviously the devil is in the details of a particular implementation, but in general I support "do less, focus more".

Well. All of that being said, I'm sure there are almost zero technical barriers for making drupal.org additionally produce a composer.json file in tarballs, or however else you imagine composer.json files to work within the infrastructure.

What I mean is that there's nothing preventing you from introducing composer.json support on various levels; be it pre-declared/manually-authored or automatically generated.

If that happens to help some new-born packaging approach for PHP, then I don't see why we shouldn't be able to expose corresponding files/data at the right locations.

Aside from that, however, composer.json is merely a definition schema and format. It's not like Drupal wouldn't have some of the concepts already. And it's also not like there wouldn't be already existing efforts to enhance the concepts. And lastly, it's also not like the concepts used by Composer are entirely new blasphemy; most of our existing efforts are oriented on the *nix packaging patterns already, and so is Composer.

What I mean is that the main logic for the definition still needs to be implemented in the application layer. Changing the definition on its own does not change anything at that layer. So while having a definition schema is a good idea (but nothing new), the actual challenges for Drupal are in the application layer's implementation.

And to continue on that, it's not like we wouldn't have had all of the ideas for the application layer in the past already. Fact is that most issues are stalled, blocked, and depending on "hard-core" people working on them. Assuming that a definition file/format change would automatically resolve all of those issues is a bit naive. ;) Even with the envisioned WSCCI/Framework initiative efforts and changes, those systematic challenges will remain the same.

Closely related to that, there are various claims in here that this change would "ease" the handling of external library dependencies. While I could see that to be the case and nicely work out for custom PHP frameworks à la Symfony & Co, I refuse and totally reject that argument for a modular system like Drupal. If it was that easy in Drupal, the Libraries API project wouldn't exist at all, and on top of that, Libraries API wouldn't still struggle hard to find answers for seemingly unsolvable problems. (Reality shows, only few people understand what 4-Dimensional actually means, and even those that do don't have any answers.)

I must confess I did make the mistake of thinking this was just about the format... :/

But I've read both the blog posts linked in #21 and read them again I still don't feel terribly enlightened as to how this actually would work.

As in, suppose module A declares module B as a dependency and module C as an 'enhancement' and module D as a 'conflict'. How does Composer make that happen? Surely we still need stuff in system.module to enforce the dependency on the modules page and flag up the problems etc etc. I'm not understanding how much of the work Compose would do for us, and indeed *how* it would do that work. For isntance, it sounds to me like we'd need to have a package repository it can talk to on d.org.

Also, if the format is swappable, here's a radical suggestion: why don't we keep our existing format? Then the net change to DX is zero. We're all about how hiding the internals means we can change them at will without breaking APIs: this would be changing the internals without breaking the API to humans. Also, the code for parsing info files is about a dozen lines.

(And finally: what on earth does PCB stand for? Yes, I googled it.)

So, great question on how composer would make that happen. I'll look into it.

The code for parsing info files is in fact not a dozen lines. We've got this goofy brittle regex thing going on which is(apart from the constant magic) the equivalent of just json_decode.

re PCB: I couldn't figure out why we where talking about printed circuit boards either but apparently its sun's acronym for "Poor Code Bloat."

StatusFileSize
new108.16 KB

I've put together the beginnings of a Drush wrapper for Composer over at #1419058: Switch scope of the Drupal Composer namespace. Might help people understand Composer a bit more. Screenshot attached.

Also opened up a Drupal fork over at GitHub with composer.json used to download jQuery UI for some experimental funtertainment. Of course, we'd have to define our own Composer repositories and such, but the idea is there.

Hehe, it seems actually really nice! It would make life of people that deploy Drupal into a closed network environment really hard thought, but it sounds appealing still!

It would make life of people that deploy Drupal into a closed network environment really hard thought, but it sounds appealing still!

You can host your own private repositories with Composer. You can even remove the Packagist one if you really want to.

Some clarifications are probably in order, because this thread seems to imply that our .info files can be directly replaced by composer.json. They cannot, as they both have slightly different (and partially overlapping) purposes:

  • composer.json is a packaging / archive metadata. Its purpose is to tell you how to get a particular package (and in addition which other packages you might need, etc.). It covers the same ground as packages metadata in OS distributions (.deb, .rpm, etc.), and to CommonJS Packages specification. The equivalent in Drupal is information stored on Drupal.org in the Project and Project releases nodes that are exposed to the world in the update status feed.
  • Our .info file is a component-level metadata. Its purpose is to tell your how to use a particular component, and especially what you can find in it (introspection). It covers roughly the same ground as the config/environment.rb, CommonJS Modules, etc.

(Note that Symfony doesn't have any introspection capability. PSR-0 either.)

We tried at one point to *also* use .info files as a packaging metadata, and it became the untangled mess it is right now: modules can (but are not forced to) declare belong to a "project" and a "project status url"; one module of a project is going to be elected as the "main" module, in a way that is undefined, etc. (just look at the code of the update module or of Drush PM to get convinced).

So, composer.json as a clean packaging metadata, yes. composer.json as a complete replacement for our info files, I don't think so.

#33 expresses exactly what I had in mind since this issue's inception, thankfully better articulated than I could ever have.

I agree that .info files don't need to be replaced if it's only in order to keep iso-functionallity, but if it happens that we need more features, it might worth it to at least study potential replacements such as composer.json. There is good reasons why we may want it, one I can see right now is that we could reproduce the download and make features of Drush without Drush.

Great stuff gang,

How could this help the use case of finding out when my modules need to be updated or installing new modules straight from Drupal's admin ui. Would composer.json be able to handle the problemset of discovering new modules to install?

So, I guess not surprisingly I disagree with Damien. I think the core of it is I see the system table as tell us how we use components. In random places in the Drupal interface we act like a package manager and read from those files and populate the system table building a working environment but the files themselves only tell us information about the component. Any where else in the world this would be called what it is, package management. I think this is demonstrated even more clearly by the fact that the vast majority of the fields in .info files are package related(name, description, version, grouping, dependencies, php compatibility, etc). info files map directly to composer.json.

Info files where always setup in this manner and the things that are less package management related are actually late arrivals to the format. The fact that things are spread out in the update feed and that the project url thing is goofy and underused has nothing to do with them not being a package format and everything to do with not being able to commit large changes to core/growing organically and us not being very good at it(dww, you're awesome and this is not a crack at you). This alone should be a reason for us to consider using composer.json actually.

Looking at the few components of .info files that don't map directly they are mostly our Drupalism asset registration systems and maybe theme regions. I'm not sure how these are introspective but I do think they can fit comfortably in composer.json if we think about our metadata files as dynamically building the application, not actually running it.

  1. Class files. Our current autoloading system is Weird. PSR-0 is better for the ease of registration, is consistent with PHP land, is already being adopted into d8 and is built into composer.json so that's the easy one.
  2. CSS and JSS files. So, this is a clever system and useful especially when your thinking in the building the application sense. I think adding an 'asset' field in the 'extra' section of composer.json seems like a perfect place for this sort of information and the module page(or drush or whatever) can "install" the files into the theme systems "put these assets on pages" system. We could probably even work out a standard with the composer people for conditional information and cool stuff.
  3. Theme regions. OK I'm not actually sure on this one but we could probably put it in the extra field too. I like PE/Panels's layout system better and I'd like to see blocks in D8 work more like that anyways but that's to be seen.

I've got some ideas on how we can address some other disconnects between drupal and composer to come.

neclimdul++

neclimdul: This would actually be a good Core Conversation proposal, methinks. :-) http://denver2012.drupal.org/core-conversations

Issue summary:View changes

note module per directory policy issue

So the novel solution to address some of the minuses didn't pan out so the list hasn't changed.

Would composer.json be able to handle the problemset of discovering new modules to install?

So, discovering new modules on the file system this would be mostly the same minus the module per directory requirement. Discovering new modules through the UI this would also work mostly the same but we would have the additional keys like keywords, suggests, and recommend to empower the module redesign people.

However discovering packages available for download(and are not in the suggests and recommended category) that would be out of the immediate scope of converting to the file because it would require non-composer code inside of Drupal for querying that system. The two options I would see though are:
1) We use d.o's project functionality mostly as it stands now without any real composery/packagist stuff and just have composer do its thing out of git like RobLoach is playing with now.
2) We could look at how Packagist is setup and move to something similar. This enters us into a community of package hosting instead of a sort of walled garden. This would also likely lower the bar for your own module hosting solution not on d.o for better or worse(a CC theme site?)

Not sure how Drupal could use Composer for module/theme deployment, but Drupal Core itself could use Composer for its third party vendors: #1424924: Use Composer for updating Symfony components (without removing Symfony code from repo). This makes upgrading and management of third party dependencies much easier than managing them all ourselves.

Status:Active» Postponed

Not sure Composer makes sense for the module/theme space. It might make sense for third party libraries though: #1424924: Use Composer for updating Symfony components (without removing Symfony code from repo). I'm going to set this to postponed for now, maybe even won't fix. Let's keep the discussion open at #1424924: Use Composer for updating Symfony components (without removing Symfony code from repo) though.

Issue tags:+composer

tagging

I agree with #33 .info and composer.json files have different targets.

We should use composer to manage dependency without trying to use the tools for what it was not made for.

One composer.json file (optional, themes do not have always dependencies) and a .info (or a better format if we find one) can certainly leave together.

Still don't see composer.json as a replacement of .info files. But, we now have composer/installers, which is helpful. LESS, Symfony and Zend Framework are a few examples of Drupal modules that support installation via Composer.

#1612910: Switch to Semantic Versioning for Drupal contrib extenstions (modules, themes, etc) would allow versioning to work properly with Composer.

This issue may be interesting in the context of the discussion here as well: #1545684: Separate version data from the .info file

Title:Consider composer.json as a replacement for .info filesConsider composer.json to manage dependencies instead of .info files

composer.json is made to manage dependencies and package info but not stuff like theme regions. Let's make it clear in the title.

This is what it looks like

{
    "name": "monolog/monolog",
    "type": "library",
    "description": "Logging for PHP 5.3",
    "keywords": ["log","logging"],
    "homepage": "http://github.com/Seldaek/monolog",
    "license": "MIT",
    "authors": [
        {
            "name": "Jordi Boggiano",
            "email": "j.boggiano@seld.be",
            "homepage": "http://seld.be",
            "role": "Developer"
        }
    ],
    "require": {
        "php": ">=5.3.0"
    },
    "autoload": {
        "psr-0": {
            "Monolog": "src"
        }
    }
}

There's an extra option, which can be used to hold any extraneous information, like theme regions if desired.

This can perhaps be a good things but I have to say I can't think about all the implications of using this extra option.

Status:Postponed» Active

Related: #1793074: Convert .info files to YAML

EDIT: Whoops, I guess I should refresh before posting a comment.

Issue summary:View changes

Better information about Composer

Issue summary:View changes

Moved from #52

Issue summary:View changes

Updated issue summary.

Issue summary:View changes

Updated issue summary.

Issue summary:View changes

fg

RE: 'title' property:
https://github.com/composer/composer/issues/1140

In a corresponding PR for it, @stof mentions that the Composer schema would already allow for the property, but it simply wouldn't be used (or rather ignored) everywhere.

Thanks, sun. Updated it accordingly. Following that PR. Will take a look once I get the chance.

There seems to be a lot going on in drupal 8. Is this going towards a json file for package management and a yaml file for module/theme metadata? Is this 'either/or' or 'and'

Looks awesome Rob. Thanks man.

@dcrocks Composer natively expects json, but if we really want, we can write a wrapper around composer and use it as a library to read the same stuff out of a yaml file. I kinda prefer that approach, but any step toward composer is awesome.

There's also this, but it's pretty clunky:
https://github.com/igorw/composer-yaml

#52 should be moved into summary

Issue summary:View changes

Updated issue summary.

++ Damien in #33.

A module (e.g. devel_node_access) is not a package.
A Drupal "project" (e.g. what you download at drupal.org/project/devel) can be considered a package. Only with a different download mechanism.
composer.json would make sense as a file to put at the root of a *project*.
But we still want something to put at the root of a module folder, e.g. of views_ui. This is currently views_ui.info, and could become views.yml.

So, currently we have:

Drupal package / project: devel, admin_menu, etc.

Drupal module: devel, devel_node_access, etc.
Usually shipped with a Drupal package / project.
The special thing about a Drupal module is it can be enabled or disabled. And it can declare a schema to Drupal. Etc.
Modules can have dependencies to other modules.

Composer package: monolog, guzzle, etc.
You can not disable or enable a composer package. You can just download it, and it is there.

-----------

Module dependencies are not package dependencies.
If devel_node_access depends on devel, this has nothing to do with package dependencies.
it does not mean we want the package be downloaded, but one of the modules within the package be enabled. (and actually we don't care which package that is, so long as it contains a module with that name)

Btw, it does make sense for a module to declare a dependency on a composer package.
It also could make sense for a Drupal "project" to declare a dependency on a composer package. But it is probably better to do that on module level, because then we don't need unnecessary code as long as the module is disabled.

-----------

This issue can still make sense:

1) Look for modules within downloaded composer packages.
E.g. we provide a mechanism to download packages with composer in a dedicated folder, e.g. sites/all/composer/vendor (D7), or just composer/vendor (D8). Then we allow those packages to contain Drupal modules. These need to be declared in the composer.json, so we don't have to do ridiculous file scans.

2) Build on composer as a package manager, and make drupal.org the main repository for that.
That is, every Drupal "project" (e.g. admin_menu) will have a composer.json - but NOT replacing the module info file.

3) Allow modules to depend on composer packages.
This does not need to be in a composer.json, instead it can be in the module.info or module.yml file. Because a composer.json is to describe a package (= sth you can download), whereas the module.info or module.yml describes a module (sth you can enable/disable within Drupal)

Bump. I predict a lot of contributed modules for Drupal 8 will make use of composer to include external code, seeing as core already does this.

This issue has been open for around 13 months without a patch. I think that it's safe to say that this is not happening for Drupal 8.

However, I know that this would be useful for some people, so the alternative that I propose is that we allow contrib modules/themes/profiles to define a composer.json file, and for the ones that choose not to, we'll autogenerate a composer.json file as part of the packaging process. This will allow people to manage their modules/themes/etc with Composer, and I don't think there's anything that needs to happen in core in order for that to happen. This differs from the existing proposal in that we wouldn't really care about it in core - it's just a nice thing to have for people to build their sites. In fact, there's no reason that this can't happen right now for Drupal 6 and 7 packages.

What I'm saying is that we can essentially built out the packaging system such that we can use composer.json as a replacement for Drush Make files. This requires no core code changes and is relatively easy to do.

Version:8.x-dev» 9.x-dev

Component:system.module» base system
Issue tags:+Extension system

@cweagans, #60: Hey Cameron, did you file an issue for that proposal or is there one already filed?

@cweagans: http://drupal.org/project/composer_manager ... There are a bunch of modules that use a composer.json file already.

@klonos, I have not opened an issue for that.

@Rob, Interesting. I hadn't come across that yet. Looks pretty cool. Correct me if I'm wrong, but wouldn't all modules need to provide a composer.json file to use composer as a replacement for drush make?

*Redacted*

@cweagans Drush Make is rather different than Composer and I don't really see one replacing the other. Although their use cases are similar, there are things that one does that the other should not do.

That said, there are some things we could port into Composer Installers' DrupalInstaller. I just put up a pull request to locate the Drupal root location. This is similar to how you can run drush dl stringoverrides from anywhere and it'll find out where to install String Overrides.

I've updated the Symfony module to use composer/installers. That means you can stick the following in a composer.json anywhere in your Drupal 7 folder, and it'll install the Symfony module to sites/all/modules/symfony:

{
  "require": {
    "drupal/symfony": "7.2.0-alpha5"
  },
  "repositories": [
    {
      "type": "vcs",
      "url": "http://git.drupal.org/project/symfony.git"
    }
  ]
}

Either way, Composer Installers is a much different discussion than using a composer.json in place of .info files in Drupal Core :-) . Get in the Drush Composer issue queue and we'll talk there!

Re #67:

> Drush Make is rather different than Composer and I don't really see one replacing the other. Although their use cases are similar, there are things that one does that the other should not do.

Could you elaborate?

Like Rob said, probably not the place :) This thread pings like a 60 ppl every comment

Can you recommend another thread Rob?

Status:Active» Postponed

Issue summary:View changes

gvx

Component:base system» extension system
Issue summary:View changes
Issue tags:-Extension system