After months of design, implementation, and testing, the initial version of the new system for releasing Drupal contributions is finally installed and running on drupal.org. This change impacts everyone who uses or maintains contributions (modules, themes, etc) from drupal.org. Contributions have a new version number scheme, and at long last, official releases.


Changes for Drupal users

The most visible change for end-users of Drupal contributions is that the version numbers have now changed. Please read the Version numbers handbook page for the details.

Additionally, you will now be able to download official releases of the contributions your site depends on. These will have a fixed, unchanging set of code, so you will always be able to easily identify exactly what code your site is running. If a new release is available, instead of having to compare dates on release packages, you can just look at the version number to tell.

Finally, if your site is running an older version of core (which is quite common given how fast Drupal core releases come), but you would still like to use new features from some of the modules you are running, it is now possible for module maintainers to provide both stable and development releases of their modules for any versions of Drupal core they wish to support. See the project pages for the modules you are using for details, or submit issues to their issue queue requesting that they provide a development release if one is not available and you are willing to sponsor their efforts to provide one.


Changes for Drupal developers


Along with the new release system, there have been a lot of updates and changes to the Drupal CVS repository this year. Because of these changes, all CVS account holders and project maintainers should review the updated CVS and Project documentation even if they think they are already familiar with CVS. The handbook outlines the policies and procedures to use going forward for maintaining your projects, managing your releases and proper branching and tagging of your contribution on Drupal.org. More people using the same processes means there will be more people to help others and this will standardize our release process for the future.

Be sure to review the new version number documentation so you will know how to properly take advantage of the release system for your modules and themes.

If you are unfamiliar with CVS branches and tags, please read the updated CVS introduction to these concepts. There is also a new Maintainer quick-start guide to help people new to CVS get up to speed on the basics of using Drupal's contributions repository.

Note: During this conversion, release nodes were automatically generated for all of your existing stable branches (e.g. DRUPAL-4-7, DRUPAL-4-6, etc) that were being packaged (e.g. the old 4.7.0 version of your module or theme). So, you will not need to add release nodes for these branches, and when trying to add new release nodes, these branches will not show up in the list of available CVS identifiers, since they're already in use by an existing release.


Thanks

Primary financial sponsor: Mailfriends

The major financial supporter of this effort has been Mailfriends The original Penpal community. Without their very generous contribution and sponsorship, the work would still be incomplete.

Other financial support

Additional major donors include:

I also want to thank everyone from the Drupal community who made small donations (too numerous to list individually, but taken together, they contributed over $1000).

There's still lots of future work relating to this (read below), and even with all of these generous donors, more support is needed. Please consider donating to this effort, so that I can continue to build on the foundation we've just established:




Technical help

Apologies to anyone I'm forgetting, but these are main people who helped me make the system happen:

  • Kjartan was instrumental in reviewing the huge patches to the project.module and cvs.module that made this system possible, in deploying beta versions of the system on scratch.drupal.org, and in the final update to put the system into effect on drupal.org
  • killes also reviewed my work, provided design input, helped optimizing the conversion process, and helped answer many mysteries I encountered.
  • chx (as always) solved all of the problems I faced when trying to get some very tricky forms working in the Forms API. He also reviewed the patches looking for potential security problems, and throughout the life-span of this effort has been incredibly helpful and supportive.
  • Heine provided essential security audits, reviews, and input.
  • merlinofchaos did the initial conversion of turning releases into real nodes and provided invaluable input and support during the initial planning and design stages.
  • webchick asked all the right tough questions to help make the system usable and understandable, and helped me with a major effort to update all the documentation listed above.
  • Eaton also helped shed light on the inner workings of the Form API and helped me solve some of the tricky problems.

And of course, thanks to Dries for letting me fix this stuff. ;)


Background


Albert Einstein wisely said, “Make everything as simple as possible, but not simpler.” Unfortunately, Drupal development happens in a very complicated context, with rapidly changing versions of the core API, the interaction of different modules, the need for stable and development versions of many modules, and so on. Balancing the needs of a simple, usable system with something powerful enough to solve the problems we currently face has been a constant challenge.

If you're interested in finding out how the new releases system got to this point, here's a collection of the major threads of discussion and design that led to its current implementation:


Future work

There is now a New Release System group at groups.drupal.org which will become a central place to keep track of work on the system. Please feel free to subscribe if you're interested in progress updates, and especially if you're interested in helping with testing, documentation, and such. ;)

At that group, you can find a complete list of future work which outlines all of the feature requests and tasks that myself and the beta testers have submitted. None of these items were important enough to delay the deployment of the system, but they should all get done eventually.


Providing feedback


First, please search the list of future work before you submit new issues, since chances are good myself or someone else has already thought of whatever you want to tell us. ;) However, if your feature or bug isn't listed there, here's where you should report things:

In all cases, file your issues against the "4.7.x-2.0" version of these modules.

Thanks!
-Derek

Comments

robertDouglass’s picture

This is a significant milestone in the Drupal project's development. The increased stability and consistency that these changes will bring will attract an even wider audience. This brings real significant value to Drupal. It is with much gratitude that I digg this story.

- Robert Douglass

-----
Lullabot | My Drupal book | My Digg stories

Robardi56’s picture

Great system !! This was much needed. Thanks.

EDIT: once this naming information will be added to the module page (http://drupal.org/node/87298 for v5.0), this will make our life so much easier maintaining drupal modules !!

Brakkar

careterfun’s picture

That future work really is encouraging, it want be so long that we'll
have a system with almost no flaws. I'm looking forward to that day.

Apreciate the work everyone is doing.

bonobo’s picture

Yeah -- I've just been looking through the modules pages -- the new system makes for a much easier time sorting out what modules are available for what version of Drupal --

This is great work --

Cheers,

Bill

-------
http://www.funnymonkey.com
Tools for Teachers

zoon_unit’s picture

I tend to view modules by date to see which ones have been updated. Unfortunately, now ALL modules show as updated, due to the module system being updated. acckkk!

One BIG suggestion. When ordering modules and themes by date, and also filtering by Drupal version, shouldn't the listing ONLY show modules that have been updated for the version the listing is filtered by?

In other words, if I filter modules by then all modules will be listed by the date of the latest version update, regardless of WHICH version.

If I filter modules by <4.7.x> then modules should be listed in the order of the date their 4.7.x modules were updated.

Make sense??

webchick’s picture

# If it's specific to browsing projects, finding releases, etc, open a new issue here: http://drupal.org/node/add/project_issue/project (please be sure to set the Component field to "Releases").

zoon_unit’s picture

Submitted request here:

http://drupal.org/node/96997

Is this correct?

sepeck’s picture

The new system caused all packages to receive new package dates so everything is starting fresh from today. This will resolve over time. See comment for background"

-Steven Peck
---------
Test site, always start with a test site.
Drupal Best Practices Guide -|- Black Mountain

-Steven Peck
---------
Test site, always start with a test site.
Drupal Best Practices Guide

zoon_unit’s picture

That's to be expected, I suppose. The system itself looks wonderful! I'll submit an official feature request. It's time for me to learn the proper way of doing things. :-)

webchick’s picture

I just want to call attention again to the Maintainer's quickstart guide. This is a one-pager that has all the commands you need know in order to create releases, branches, and such. If you need further background info on any of this, dww has done a fantastic job writing very good reference docs.

Thanks so much, Derek. We all owe you big-time for this, which is truly a quantum leap for contributors. :)

And also big thanks to the folks at Mailfriends (and the other financial contributors) for making this possible.

fax8’s picture

Already followed suggested steps for my video module. Created a 4.7.x-1.0 release.
Everything was fine.

Thanks to everybody who contributed in the new release system.

Fabio Varesano

------------
www.varesano.net - Fabio Varesano Personal Homepage

pwolanin’s picture

I've been making stable 4.7.x and beta 5.x releases of my projects today, and as both a contributor and site maintainer this is going to be a great improvement that will benefit the whole Drupal community.

---
Work: BioRAFT

Rainy Day’s picture

First let me say this is truly a great advancement. Thank you everyone who have worked to make this happen!

To make it even more useful, however, i suggest adding the following:

1) On the modules download page, when browsing by name, it would be incredibly useful to see the latest stable version number for each module. This will help admins who are checking for updates.

2) The modules admin page should show which version of a module is installed.

3) It would be wonderful if Drupal core were able to compare installed modules against the latest available stable version on-line and flag any modules which have updates available, right in an admin page. Perhaps this might be done when the admin clicks a “check for updates” button on the module page?

4) Even better still... Instead of a manual checking method, a way to automate it. Perhaps the admin could select an interval of time (e.g. every day, once a week, once a month) and the cron system would automatically compare installed modules against what’s available, then send the admin an eMail (with the name and version numbers of updated modules) when module updates are available.

bonobo’s picture

http://drupal.org/node/add/project_issue/project

The form has a "Category" field -- select "feature request"

Cheers,

Bill

-------
http://www.funnymonkey.com
Tools for Teachers

dww’s picture

please don't submit new issues, these are *all* duplicate... ;)
search first. start here: http://groups.drupal.org/node/1830.
thanks,
-derek

___________________
3281d Consulting

dww’s picture

sorry if some of the titles are confusing, but in fact all of this is already in the issue queue:

re 1: #89538: need branch-aware notion of "default" or "latest" releases

re 2: #87298: Add version info to modules page

re 3: #94154: module page could say if new releases are available

re 4: on second thought, i guess there's no specific issue yet (that i know of) but this has been proposed repeatedly in the forums and mailing lists...

cheers, ;)
-derek

___________________
3281d Consulting

Rainy Day’s picture

Looking forward to a day when these features are all implemented.

rkendall’s picture

Just tried it out and made an 'official release' of my new No Koala theme, I think it may have even been the first theme to do so. No problems at all.

Thanks again.

--
Ross Kendall
UK based Web and IT consultant specialising in Free and Open Source Software technologies.
http://rosskendall.com

Walt Esquivel’s picture

I need to say thanks to everyone who contributed in any way (coding, financial support, testing, reviewing, etc.) to the new release system.

In particular, thanks to Derek (dww) for putting so much effort into coordinating so many people and so much code into such an incredibly useful and updated method to the way things are done at Drupal. Also, particular thanks to Mailfriends for their outstanding financial support. I made a small financial donation and urge others to do the same. As dww mentioned above:

There's still lots of future work relating to this..., and even with all of these generous donors, more support is needed. Please consider donating to this effort, so that I can continue to build on the foundation we've just established...

The new release system is a great improvement that will benefit the entire Drupal community.

Again, thanks so much to everyone involved! What a great Drupal community we have!!!

Walt Esquivel, MBA; MA; President, Wellness Corps; Captain, USMC (Veteran)
$50 Hosting Discount Helps Projects Needing Financing

Robardi56’s picture

I'm following some modules, that are regularly updated.
I thought that with each series of commits, the patchlevel would be incremented.

Apparently it is not the case, so again, I must try to compare dates to know if my module is up to date or not.

So is this the fault of the module developer, or did I misunderstood something about this new system ?

Brakkar

pwolanin’s picture

I think the way it is supposed to work is that when the maintainer is satisfied with the bug fixes/features/etc in those commits, the maintainer will then tag and make a new release. Some maintainers (myself included) might want to commit a number of small changes, rather than trying to get it all perfect at once and commit one big change. The advantage of the new system is that by making a release, the maintainer can signify when the ongoing work has reached the next plateau and at that point you may want to update.

---
Work: BioRAFT

Robardi56’s picture

I thought patchlevel was only small bug fixes, while MAJOR would indicate major revision. The problem with what you said, is that there can be lots of commits between a patchlevel increment. So when someone refers to a patchlevel, it could be a different version than someone referring to the same patch level that was downloaded days after tons of commits got in. So all in all, the date of last commits remains the only credible reference when someone refers to his module version, and the only way to know if one is using the module with the latest bug fixes.

I guess I got it wrong. I thought the patchlevel would increment with each commit.

Cordially,
Brakkar

pwolanin’s picture

I don't think your assumptions are quite correct. The whole point of this system is that any given release will be immutable- if you download version 1.1 today, or 3 months from now it will be the same code regardless of the commits in between. Once a new release is made, e.g. version 1.2, the same holds. Just like the core versions- the Drupal 4.7.4 release will never be changed at all. The bug fixes going into the 4.7 branch will eventually appear as part of the 4.7.5 release.

Obviously if you need the latest fix or feature you may download a development version that's between versions, and in this case you don't have a stable referrant. In addition, of course, whether module maintainers are responsible about making release will also depend on the individual.

---
Work: BioRAFT

dww’s picture

@brakkar: I guess I got it wrong. I thought the patchlevel would increment with each commit.

indeed. pwolanin is exactly right. imagine for a second how unbelievably painful it would be to try to use Drupal (just the core) if there was a new patch level release generated for every commit. take a look at http://drupal.org/project/cvs/3060 to see what i'm talking about. that's insanity. ;) instead, the core maintainers make lots of commits between official releases but only bother to create a new release (signifying to all users of Drupal they should upgrade) when those changes have converged on a reasonable set of fixes to existing bugs, or when a security vulnerability is discovered and fixed.

so, just like core, contrib maintainers can now decide when to make official releases, and when the patch level should change. a new release means users should upgrade. a new commit to CVS does not. only the human maintainer can make this decision correctly, which is why making an official release requires manual intervention by the project maintainer.

of course, if you're still relying on the periodic snapshot "releases" from the end of a branch (with 'x' as the patch level and '-dev' afterwards) then yes, exactly, you don't know the patch level, since your code changes on every commit. you clearly need to re(read) the version info handbook page, especially the section on development snapshots.

enjoy,
-derek

___________________
3281d Consulting

pwolanin’s picture

Just a quick note: the handbook page: http://drupal.org/handbook/version-info#dev

Seems to be somewhat inaccurate as far as how the contrib snapshots are labeled. From each module's page, right now the dev snapshots show up as 4.7.x-1.x. The handbook page suggests that they will show up as something like 4.7.x-1.x-dev (which would actually be clearer and more explicit)

---
Work: BioRAFT

dww’s picture

somehow, this seems to have been screwed up during the nightmarish conversion deploying the new system on drupal.org. sadly, in the aftermath, i seem to have missed this detail. :( all the newer snapshots created after the deployment have the "-dev" like they were supposed to, but the automatically generated snapshot nodes did not.

i just used some php snippets and a shell script to rename all these release nodes and their corresponding tarballs. reality should finally match the correct documentation. ;)

thanks for the report!
-derek

___________________
3281d Consulting

styro’s picture

If you want to reference the exect state of a file after a specific commit, you can use the CVS version of that file. You can get that from the ID string at the top of the file or browse it through the CVS web interface.

Sure it won't snapshot the state of the whole module at that time the same way a Subversion revision ID does, but it would still be useful 90% of the time as most modules really only have one or two important files and any given bug is usually fixed by editing a single file.

But even so, only having the date isn't that big a problem as you can check out the state of the whole module at a given date from CVS anyway.

--
Anton
New to Drupal? | Forum posting tips | Troubleshooting FAQ

kurikinton’s picture

Is there any progress on setting up an RSS feed of releases for each project as mentioned here:
http://drupal.org/node/77562

Chris
SeeBox Inc.

dww’s picture

per-project RSS feeds for release nodes lives here: http://drupal.org/node/94138
it's on the todo list (http://groups.drupal.org/node/1830), that's as far as it's gotten.
feel free to help (post a patch there, donate (paypal button above), etc.)

thanks,
-derek

___________________
3281d Consulting

kurikinton’s picture

Got it, thanks!

Chris Harrington
SeeBox Inc.

DrMiaow’s picture

There seems to be no version of this module for Drupal7.

I'd be open to maintaining and hosting releases of my Module in my Drupal7 instance, but this seems to be an old Drupal6 module?

Is there a new equivalent in Drupal7?

dww’s picture

___________________
3281d Consulting

DrMiaow’s picture

Thanks!

Looks like I will need to wait till Project is re-imagined for D7 and at least goes into alpha.

Today I've manage to construct a proof of concept of a stand alone package manifest generator (I can at least get D7 to recognise that my package has an update pending) so I will use that for now.

Thanks again for the quick response.

James.