New method for contributed project releases and versions

Yep, this would be a vast improvement. It would also open up possibilities for multi-module packages, e.g., tracking by component module.

This also touches an other issue I have : one of my modules (taxonews) is bi-compatible (4.6 and 4.7), but there seems to be no way to handle this.

currently the "best" way to handle a module that works with either 4.6 or 4.7 drupal would be to leave all the code in CVS head, and then add non-branch tags called "DRUAPL-4-6" and "DRUPAL-4-7" to the revisions from the at the head of the trunk. whenever you make a fix, you just move these two tags, instead of applying the same patches to 2 or 3 branches. but, we should consider this use-case when thinking about how to get project to talk directly to cvs.

some brainstorming about how this might work:

• each project could have a setting for the regexp that recognized its release tags. by default, the regexp would be .*, namely, any tag in that module was considered a valid release. but, folks could also say something like DRUPAL-\d-\d-\d? were valid tags to make releases from... if it was really fancy, there'd be a way to use the same regexp to specify what version string to use from each tag. not sure if there's precedent for a UI for that sort of thing anywhere else in drupal, yet. perhaps we don't need/want the complication of handling this regexp per project, and want to just force all projects to use the same tag naming conventions, in which case this could be a global project setting (from admin/settings/project), not something handled in each project's node.
• via hook_cron(), project could scan the cvs repo for each project (there's already settings in the project that says what directory and repo a given project lives in) and build up a list of valid release tags for each project. this would be cached in the DB (probably in cvs.module's tables, if they're not there already). it's a little fuzzy to me where the line of responsibility should be drawn between project.module and cvs.module for some of this.
• these release tags and/or version strings would then be used by project for the "version" drop-down box, basically all the things we use the existing releases for.

to make releases, module admins just add a new tag for themselves. this will require changing the existing tag ACL stuff to be more flexible (and fine-grained), but that's a separate issue (i guess i should create one in the drupal.org maintainance queue for that, in fact). we could add settings for cron-driven script hooks to automatically checkout, package, and create tarballs if new tags show up we don't have tarballs for. i guess most of that is handled by the real cron on drupal.org, not hook_cron within drupal itself. but, this seems like exactly the kind of thing people will want to be able to do with project at other sites (if they're doing cvs-integrated projects at all), so we might as well make it as easy and self-contained as possible to do all that.

i'm not yet familiar with the drupal cvs.module and its existing DB tables. i'll definitely check that stuff out in detail when i get the chance. just wanted to write down some ideas while they're still fresh in my brain.

Ok, this is half of a plan. I am willing to help on this in my spare time, especially if we can divide up coding work amongst a couple of people. I'm not very familiar with interfacing with CVS, for example.

1) First, we rename the current system so that each release that is currently built is called a 'nightly' build. THis matches with how many other systems do it, and will more immediately tell people that such a thing is built daily. Even if it's built more often than that.

2) Second, we implement a release management system. Each release must consist of:

• branch (i.e, DRUPAL-4-6, DRUPAL-4-7) etc.
• Major version number (X).
• Minor version number (Y).
• Tertiary version number (Z).
• Release notes.

Releases are built off of a specific CVS tag. Suggestion: RELEASE-X-Y-Z.

3) We implement a 'create release' page. The above information is taken, and (if this is possible) it checks to see if the right tag exists in the right branch. If so, it schedules the release to be picked up in the next cron run.

4) Cron sees the scheduled release, tars it all up, and tells the release manager where the tar file is.

5) The release manager lists all available releases, and for what branch, with the brief description.

Notes:

1) Releases, once released, are static. If there are bug fixes, create a new release.
2) It should be possible with some clever CVS magic to actually embed release numbers into the code, but I am not sure how to do that, or if I'm right in remembering that it's possible. Posting how to do that on the documentation for creating a release would make people very happy.
3) Don't constrain version numbers.
4) Releases are named, roughly, "MODULENAME release X.Y.Z for DRUPAL-BRANCH".

Additional notes, from conversation:

Tertiary may not be necessary. However, major and minor are necessary, at least for me.

Naming scheme, from IRC conversation, appears to be looking like drupalversion.x-X.Y. For example, for Views 1.0:

Views 4.7.x-1.0

after a debate on IRC, i think i convinced everyone that the version strings for contrib modules should be, for example: "4.7.x-Y.Z" where "x" is a string literal, the character 'x'. so, everything before the '-' tells you what version of drupal core this module is compatible with. since the core API doesn't change across 4.7.x, a given module ported to the 4.7 API should work with any drupal core release from the 4.7.x series. the "Y.Z" parts are just a major/minor release number for the module maintainer to play with as they see fit. if merlin wants to have "views-4.7.x-1.Y" as the "stable" version of views (with only bug fixes), and an experimental "views-4.7.x-2.Z" as a "development" version (with new features) that's upto him.

in terms of the current automated nightly tarballs, i think contribs should use the same method that's now going to happen in core via http://drupal.org/node/65952. so, if "views-4.7.x-1.3" is out, the nightly tarballs would be called "views-4.7.x-1.4-dev". i think using version numbers like this are better than just "nightly", since they're much less ambiguous for the purposes of issue tracking, debugging, etc. we could do some brute-force with md5 checksums (much like we do now) to make sure that a new one of these isn't actually put up for download unless it changed from the last one. or, we could use sneaky cvs hooks to update a "dirty bit" in some table that "project X on branch Y" has been changed since the last release (which we'd do from xcvs-commitinfo, or the CVSROOT/commitinfo hook for cvs-aware folks reading this). then, the nightly cron runs would just look in this table to see if project foo's dirty bit is set on any given branch, and if so, generate a new tarball for it.

i'm definitely in favor of package maintainers manually creating releases, instead of having it driven either by cron or by the existance of cvs tags. this is much more like core handles releases, and how anyone really does software releases, for that matter.

it's not possible, even for a cvs wizard like myself, to get cvs to automatically imbed these version numbers in your code. the RCS revision of the file, no problem. but, the unique version string as described here, no chance. so, project maintainers will just have to do this themselves. alternately, we could auto-generate a "VERSION.txt" file and stick that in the tarball, based on the version they create using merlin's proposed form. then, there'd be nothing to forget, but there'd also be nothing imbedded in a comment at the top of each file like the $Id$ string. though, now that i think of it, if we autogenerate a VERSION.txt file, there's no reason with a little regexp hackery that we couldn't automatically concatenate that as a comment at the top of each file (provided we knew based on the file extension what a valid comment specifier was for that file type). i.e.

<?php
// version string goes here
// $Id ...

but, that might be too sneaky for our own good, and perhaps we just want to leave it out. alternatively, we could make some convention like: %%version%%, and we just manually replace that with the version string, so then folks are free to put:

// %%version%%

wherever they see fit (even inside a quoted string to print out in some message in the admin pages or something). that, too, might be more trouble than it's worth, especially since we can't get this replacement to happen when folks check out from cvs... (i don't think... i'll do a little research on this).

otherwise, i'm all down with merlin's proposal. especially the "Releases, once released, are static. If there are bug fixes, create a new release." part. ;)

cool. cvs supports a $Name$ keyword expansion. this gets converted into the name of the sticky tag you used to checkout the source with. so, if you've got this in your source:

<?php
// $Id$
// $Name$
...

then if you check out from a branch, like "cvs co -r DRUPAL-4-7 contributions/modules/views", you'd get:

// $Name: DRUPAL-4-7 $

however, if you check out from a release tag (as proposed), and did something like "cvs co -r RELEASE-4_7_x-1_3 contributions/modules/views", you'd get:

// $Name: RELEASE-4_7_x-1_3 $

not quite ideal, but perhaps useful...

I'll give it some thought.

I don't think people can maintain two branches for one Drupal branch (e.g. views-4.7.0-1.x and views-4.7.0-2.x). How would that work in terms of CVS? I think I'd stick with views-4.7.0-x because that maps cleanly on the CVS structure.

At any time, I want to be able to 'cvs update' a contributed module that is branched for DRUPAL-4-7, and have stable code. I want to be able to update core, and all contributed modules, with a single CVS command. I also want to be able to checkout contributed modules without having to look up branch names or version numbers (i.e. using the DRUPAL-4-7 branch name).

My preference for versioning of modules, from the perspective of the dependency stuff i am working on,

is: . , with every sequential commit to a module automatically updating the module metadata.

ie: take a module called mymodule.

First commit to the 4.7 branch must be : modulename-4.7.0 , a second commit automatically updates it to modulename-4.7.1 , third commit 4.7.2 etc.

HEAD versions should preferably in the format of modulename-cvs- , where commit id is a count incremented whenever that branch is committed to. (although i suppose date will also work)

Why do i want it like this?

Because it's a hell of a lot simpler to check a single digit than to try and check random tags that the developer can add, from within the system, and it gives all the modules a consistent versioning.

I would like the versioning of modules to happen automatically on cvs side, because it is essentially like updating the document readme, that little edit that often gets neglected.

The more I think about it, the more I dislike the proposal of module maintainers being able to maintain two or more major rewrites of their code against a single Drupal branch. Rewrites should happen in CVS HEAD (just like we do with Drupal core), and their release should be synchronized with the next release of major Drupal core. So, in the running example, views-4.7.0-2.0 would be views-4.8.0-1.0 the day it is branched as DRUPAL-4-8, or views-4.7.0-2 if the author choose to deprecate the old views version.

It is a lot easier to understand and work with, creates a lot more momentum for major Drupal releases, and encourages stability between stable Drupal core releases.

I like adrian's approach (it's the same as my suggested approach), however, I wouldn't increase the version number with every single commit. Rather, I'd increase it a most once a day or once every two days. Often, people deal with patches or issues in batches.

Automatically increasing the version number if very useful for two reasons:

1. It is really easy which is very important.
2. It enforces that stable branches, are actually stable. We should not allow people to commit experimental code to stable branches.

But, if we are going to create that many releases, we'll need a way to keep our disk usage down. Possibly by removing deprecated releases ... *scrug* ... or by getting more disks.

Frankly, I believe Dries' method hamstrings contrib developers by making it impossible for us to have stable code + experimental code in the same branch. As it stands now, I've been struggling to figure out a good way to do a proper release now that I'm about done with a version 1.0 of Views. And this method will make it impossible for me to realistically work on a v2.0 of Views.

The last 6 months of chasing HEAD has been exceedingly frustrating for me as a contrib developer. I am very interested in preventing that from happening again.

Adrian's method is basically what we have now, where each commit creates a different version, except at least with numbering there's at least some ability to track, but it's not much better than tracking CVS IDs, and ultimately would suck for developers like me.

I find both presentations to be basically unfriendly to serious contrib authors.

dries, i hear what you're saying about stability, and momentum for new major drupal releases. however, as we've seen via recent threads on development, the reality of the world is somewhat different from our idealized expectations. some drupal sites are going to stick with older versions of core for a longer period of time. yet, people using these releases might want to do their own development, or sponsor others to develop, new functionality in contrib modules they want for their sites. we might not love that, but it's going to happen, and we can't really stop it. however, we can provide a mechanism for developers to manage this reality in a way that keeps things sane, prevents needless forks of modules, folks maintaining alternate copies of modules outside of the drupal.org infrastructure, etc. we could say that the "stable" branch for any given contrib module matches the corresponding drupal core branch. therefore, "cvs update -r DRUPAL-4-7" will always work, for core and contrib, and give you the latest stable code (which would have "[foo]-4.7.x-1.N" releases). however, if a contrib maintainer wants to add a "DRUPAL-4-7_2" branch, we should let them. it's up to them to document this and let people know about it, and it encourages them to keep the stable branch stable. if we have a means for them to tag releases from this branch, add release notes, and have that show up on the downloads page from their project node, i don't really see the harm. given the speed at which drupal core's API changes, i can see why maintainers might want to add new features against a stable core API, especially if they have clients/sites that are planning to stick with that release longer than 3-5 months.

on a personal note, there's still interest in adding functionality to the signup module against 4.6. i'd love to have a way to keep the DRUPAL-4-6 version of signup stable, and only fix bugs, but to have a place to add new functionality in a way that's still compatible with the 4.6 core API. another personal example: lots of people are interested (for good reason) in me continuing to add small but useful new functionality to the project module. we want this new functionality on drupal.org, too. but, sticking to the usual rules, i shouldn't be adding new functionality in a "stable" 4.7 release of project, and none of these improvements would happen on drupal.org until the 4.8 code freeze and we're ready to move drupal.org to a 4.8.0 release candidate. instead, drupal.org could be running the "project-4.7.x-2.N" release, yet the "stable" project-4.7.x-1.N releases would be there for external sites that just want a stable release of project, and whatever bug fixes i'm doing, but not the new functionality i'm adding. sure, maintaining these branches is more work for me, but if i'm willing to do the work, why can't drupal.org provide a means for me to share the fruits of my labor?

that said, i'm *VERY* much opposed to automatically making new "releases" on every commit, or even every night. drupal core doesn't do this, and for good reason. if people want the latest development snapshot, they can checkout from cvs or they can use the automated nightly tarballs. however, project maintainers can and should be able to manually specify when a given set of coherent changes are done and a new "release" should be identified, tagged, and packaged. true, some contrib maintainers are lazy, or careless, but we can't optimize the system for them. at least for those of us who take our software seriously and do a good job, there should be a means to make official releases. these should be available in the issue tracker for bug reports, they should have release notes, a CVS tag you can check out from, etc, etc. just like official release of drupal core. hell, there could even be release annoucements on the appropriate groups.drupal.org groups, and/or an RSS feed for release announcements from the project node. the lazy/careless among us won't bother, and they'll always just have the nightly "[foo]-4.7.x-1.1dev" release as whatever they've been working on. but that's their problem, not mine. if i want to be able to say "ok, i've fixed enough bugs to really make it worth it for sites relying on this module to upgrade", i should be able to make a new release. if i fix a security hole in my module, and i want to ensure everyone understands they should upgrade right away (and what they should upgrade to), i want to make a new release. if a new "release" happens on every commit, we're generating too much noise and not enough signal, and users of my module will become numb to a new "release" coming out. just like drupal core, if we go through the trouble of calling it a new release, that means we, the authors, think it's definitely worth your time to upgrade.

speaking of signal to noise ratios, i think it's time for me to shut up again. ;)

1 last quick thought: the solution to version numbers and releases for contrib modules on the cvs trunk: just call them "4.8.x-0.N", and the nightly tarballs will be "4.8.x-0.(N+1)-dev", instead of the tremendously ambiguous and confusing "cvs". so, if i don't make any official releases of signup against the trunk, the nightly tarballs would just be called "signup-4.8.x-0.1-dev.tar.gz" if i decide at a certain point that i've added a bunch of major new functionality that's worth testing and trying out, i can make a "signup-4.8.x-0.1.tar.gz" release, and the nightlies will become "signup-4.8.x-0.2-dev.tar.gz". in the release notes for the 0.1 release, i can specify the date of the core API (which might be in rapid flux) that it was release against and known to work with for folks who want to try it out.

when the DRUPAL-4-8 branch is finally created, we move to "4.8.x-1.0" to indicate we're officially supporting a real 4.8.x release of the module...

i think all of this can be easily expressed in a regexp, and easily explained in a handbook page.

re individual release numbers: i think this is not just more user-friendly, but also more developer-friendly. here's a short list of reasons i see this as a step forward for everyone:

1. easier to handle bug reports, so developers know exactly what code was broken
2. less confusion/ambiguity for everyone than the current "4.7.0 (date)" system. it's great to be able to directly refer to a specific forum post or comment with a unique URL, why not have a unique id for releases, too? people tend to leave out the date (and folks checking out from cvs have no good access to the date that a given modification was first packaged on the web) when discussing a version of a module (in email, in forums, etc).
3. it's more obvious to end users when they should upgrade
4. it's another sign of a "well-maintained" module if the maintainer is willing to make real releases
5. easier for end users to manage their own modifications to modules (e.g. importing official module releases from the maintainer into your own CVS repository on a vendor branch), since you can do it based on actual releases, not just dates.
6. easier to make security releases, to write the announcements, etc. in addition to just saying "look at the $Id$ tag in these 10 files to make sure you're secure", we could start with "all versions of module foo prior to foo-4.7.x-1.4 are vulnerable" and then list the $Id$ tags in an apendix of the security announcement.
7. it would help encourage developers to keep the changelog at least reasonably updated in their module. if they're going to go through the effort to make an official release, it's more likely they'll update the changelog for that release.

that's a quick list. i'm sure i could come up with others, but i have to go right now. ;)

The last 6 months of chasing HEAD has been exceedingly frustrating for me as a contrib developer. I am very interested in preventing that from happening again.

Merlin, putting code in CVS HEAD doesn't require it to be compatible with the CVS HEAD of Drupal core -- it was _your_ choice to chase CVS HEAD. You can't blame the system for that because code in CVS HEAD can be compatible with Drupal 4.6 or Drupal 4.7. There are no rules for CVS HEAD. If you don't want to chase CVS HEAD, then don't.

Sorry, but I'm *very* opposed to the DRUPAL-4-7 branch not being stable at all times. If you want to commit experimental code, use CVS HEAD or create a sandbox.

I like contrib to be less restrictive than core development, eg I want to be able to add features when the next Drupal release is still months away.

You can add new features but when they get committed to DRUPAL-4-7 they should be stable. To develop and test these features use CVS HEAD or create a sandbox.

Frankly, I believe Dries' method hamstrings contrib developers by making it impossible for us to have stable code + experimental code in the same branch.

Feel free to think so, but I believe the opposite is true. I disagree strongly.

The day you have stable code and experimental code in the same branch, people will run into a lot of upgrade problems. Here is how I upgrade my Drupal sites:

$ cvs update

This upgrades core and all contributed modules. The day this is no longer guaranteed to give me a stable Drupal site, upgrades become a real pain.

Experimental code should go into CVS HEAD. Period.

Sorry, but I'm *very* opposed to the DRUPAL-4-7 branch not being stable
at all times.

I don't understand how having actual releases for a given branch, plus the ability to continue development for the branch counts as not being stable at all times. That's the whole point of this release mechanism. So we can have stable releases and continue to work.

The way you propose encourages module contributors to immediately leave behind the stable branch and develop for an unstable coding system. This system created an atmosphere where users were clamoring for code which existed...but only for a developmental Drupal that we were explicitly not recommending they run.

The API for 4.8 has already changed. This is fine, this is expected with Drupal, but in the current system, I cannot commit patches for people who are keeping up with HEAD as well as continue development for 4.7. With your proposal, that situation does not change. That doesn't make sense to me. It seems to say that we don't like our contrib developers, or that we don't think contrib projects can be important enough to merit a real release system.

Drupal core has a reasonable release system that's sensible. Why force contrib developers into something that makes our life more difficult?

At any time, I want to be able to 'cvs update' a contributed module that is branched for DRUPAL-4-7, and have stable code. I want to be able to update core, and all contributed modules, with a single CVS command. I also want to be able to checkout contributed modules without having to look up branch names or version numbers (i.e. using the DRUPAL-4-7 branch name).

We can always implement a STABLE tag or some sort to mark the last stable entry on the branch. I understand where you are coming from, but I think it is a mistake to assume that just because Drupal has tagged a release and is now stable that all contrib modules for that branch must also be immediately stable. Module development lags behind core, due to dependencies. When modules have dependencies on each other, this gets even worse.

If this is your only reason to objecting to a real release system for contrib modules, I find it to be a poor one, and think it is shortsighted. It makes the job of contrib developers harder.

I see from a later response, however, that no amount of arguing or evidence matters. You wish to pin everything on cvs update. Your project, your repository, your choice, I guess.

Have we considered end user confusion (those who download the modules)?

For example, we now have 4.7.2 for core. If mymodule is at 4.7.1, they may think that it is incompatible, and we may get questions on whether they should use it or not.

Since we know that we don't break APIs within a Y cycle, it is better to have 4.7.x-A, where A is a number.

I realize that having 4.7.x-A.B may be necessary for a few complex modules which have their own APIs within a branch, but it may be too confusing or too much work for the vast majority of modules.

Your project, your repository, your choice, I guess.

Are you trying to tell me that I don't listen to people or that I continuously choose to ignore other people's opinion? If so, I'd ask for an apology.

I actually misread dww's comment about how we can keep DRUPAL-4-7 stable at all times -- which will always be a key requirement -- while still allowing development against stable branches.

I understand that your motiviation comes from a commercial interest in Drupal, and that in the commercial world, customers demand features to be added against stable versions, and that the ability to satisify a customer's need is of primary importance. Hence, I'll give this problem some more thought so we can come up with a sensible solution.

realize that having 4.7.x-A.B may be necessary for a few complex modules which have their own APIs within a branch, but it may be too confusing or too much work for the vast majority of modules.
Hmm, we might allow both. So you module can either release as mymodule-4.7.x-1, mymodule-4.7.x-2, mymodule-4.7.x-3 or as mymodule-4.7.x-1.0, mymodule-4.7.x-1.1, mymodule-4.7.x-1.2, mymodule-4.7.x-2.0.

Apart from these details, I'm totally FOR the proposal. It makes not only module development a lot easier, but also this release system would be much better for the average user, as they can see when it's worthwhile for them to update their modules. With the 2-digit version numbers, you can even differentiate between simple bug fix release (to which the user should always upgrade) and releases that include new functionality.
I think it's a bad idea to tie module development to core development, as the release cycles between modules and core just differ two much. Modules should be able to make the difference between a bug fix release and a release including new functionality, as core does. Of course, for some small modules this might seem bloated, but no module maintainer is forced to release this way. However, for lots of modules this functionality is really important, and I'm pretty sure, once introduced, lots of contrib developers will use the possiblity to have a release cycle.

And concerning cvs updates, the idea of having a 'stable' tag is one, another would be to introduce even more branches (like DRUPAL-4-7 for the latest stable release and DRUPAL-4-7-HEAD for the latest development release for drupal 4.7).
In any case, the cvs update problem is solvable, and it shouldn't stop the whole idea of module releases.

best regards,
frando

Are you trying to tell me that I don't listen to people or that I continuously choose to ignore other people's opinion? If so, I'd ask for an apology.

Not exactly, though when I wrote that I was annoyed that dww had written some very well-thought out comments with scenarios and support, but what got responded to was my much less well-thought words that I probably should've held onto until today. I know you listen to people, and I likely do owe you an apologize as I am coming off a little half-cocked. But I do feel that you're ignoring a lot of great reasons (dww's very well-thought out posting) for one feature.

Another comment on the CVS update: With Drupal 4.7 now having the ability to phone home, it should be easy for Drupal to provide the admin with a list of modules, their current versions, and what new versions are available, giving them the option to upgrade. With a good package like that, the use of cvs to remain up-to-date would be a thing of the past. I think this is going to ultimately be the way we go anyhow, as it's a logical extension of the install/update system we have now, and something I know many people are very keenly interested in having.

Also, cvs update doesn't run update.php for you. Something that could be critical. So it's one more step than you posted.

As I said, I actually misread dww's comment about how we can keep DRUPAL-4-7 stable at all times while still allowing development against stable branches. I'll think about it some more...

Sorry, but I'm *very* opposed to the DRUPAL-4-7 branch not being stable at all times. If you want to commit experimental code, use CVS HEAD or create a sandbox.

Let me give an example of how this plays out with VotingAPI. There are now about 6-8 modules that use the API in some fashion, and depend on it for their core functionality. If I want to introduce API changes, obviously I'm going to develop in CVS HEAD until it's ready. I may be coding against DRUPAL-4-7, but it's still in HEAD. Once the changes are stable, though, and i know I want to roll it out, I have three options.

1. Move the code from HEAD to DRUPAL-4-7, breaking modules that use the old VotingAPI functions.
2. Keep the code in CVS HEAD until DRUPAL-4-8 opens, tying the new changes to 4.8 of core.
3. Start a new project named VotingApi2 in the DRUPAL-4-7 branch.
The plan the dww and merlinofchaos have outlined gives me a fourth option:
Develop in CVS HEAD, then tag the stable version of the new API as RELEASE-2-0 in the DRUPAL-4-7 branch. This would result in VotingAPI 2.0 for Drupal 4.7, something that makes sense AND is easy for module developers to work with. A simple $ cvs update wouldn't pull down the new 2.0 tagged release; it would still be grabbing the 1.0 tagged release, and thus wouldn't break any modules using the existing API. In this way, it makes it EASIER to keep a site working properly with a simple update command.

Also, duh. Starting a new OL is not the same as closing the old one.

it's nice to see so much interest and passion in this issue, though let's keep in mind we're all on the same side here. ;) i have the utmost respect for both dries and merlin (and the rest of the drupal development community), and feel like i have much to learn from all of you.

some comments on recent posts:

1. [most importantly] cvs is a means, not an end. all software projects that use a revision control system should figure out how they want to manage their code and releases, then decide how they want to lay out their repository directory structure and branching scheme to best facilitate that. working in reverse makes no sense to me. projects should never decide how to develop their code based on what's the most convinient way to use cvs.
2. my proposal continues to allow "cvs update -r DRUPAL-4-7" to always give you a stable version of both core and whatever contrib modules you've checked out.
3. i'm very opposed to using non-branch tags for "STATIC". *everyone* knows that "stable" code has bugs, and we need to be able to commit bug fixes. cvs branches are made for this problem. for any given drupal core API branch, there should be a stable contrib branch (of the same branch name), and optionally 1 or more development branches where new functionality can be added.
4. i'm *NOT* proposing stable and development code in the same branch. now that you can only add tags to a project you own (or have been granted cvs access to) no one should fear giving module maintainers the power to make more use of branches. especially once automated release packaging requires the maintainer to manually specifiy a branch to use. branches are one of the most powerful aspects of cvs, and we should take better advantage of them than we currently are.[*]
5. i think it's confusing (for developers and end-users alike) to have code in the trunk of the repository that's targetted for a different branch of drupal core.
6. "experimental code" is different from "new functionality". lots of modules want to be able to add reasonable new functionality to an otherwise stable API or codebase.
7. kbahey: please (re)read my comments in #8 and #19. i'm very much concerned with end-user confusion. i'm totally opposed to calling things "4.7.0" or "4.7.3" that are compatible with all releases of 4.7.x. however, given my desires to allow multiple branches of a contrib module for a given core API, i think it's more clear and consistent to always use "4.7.x-A.B", instead of somtimes using that, and other times using "4.7.x-A". furthermore, given my comments in #17, i think always using "A.B", where A==0 for the cvs trunk, adds additional clarity and reduces ambiguity. so, people would always know:
   • "Z.Y.x-0.N" is development code on the trunk, slated to work with the Z.Y drupal API, which hasn't been officially released yet.
   • "Z.Y.x-1.N" is stable code on a branch, known to work with the Z.Y drupal API.
   • "Z.Y.x-2.N" is development code on a branch, known to work with the Z.Y drupal API.
8. i'll grant, this naming system is somewhat complicated. however, keep in mind that this is a *very* complicated situation. "everything should be as simple as possible, but no simpler". ;) we have X different releases of drupal core's API at any given time, and Y different point releases of drupal core for a given API. for each core API, there might be 1, 2 or even 3 possible versions of a module that work with that core API (a stable series, a development series, and maybe even an "experimental" series). each of these series needs to be able to have point releases for bug fixes, security holes, or new features. users have to know what they're downloading, what it's compatible with, how stable it is believed to be, how much risk they're willing to take to get new features, etc, etc. given all those constraints, i believe my proposal is the simplest way to identify and manage all the possible releases of a given contrib module that is completely unambiguous, but not completely insane. ;)
9. nothing in my proposal forces contrib authors to do all this work. if they just want to do what dries proposes, they can. once they tag their module for a given stable branch of core, they can leave that branch for bug fixes only, and develop new functionality on the trunk. just like now, it's up to them to document what core API their trunk is supposed to work with. they never have to add additional branches if they don't want. they don't even have to make official releases off that stable branch, if they don't want. their users will just be stuck in basically the exact same boat we have now with some ambiguously named nightly tarball and a date (except instead of calling it "foo-4.7.0", we'll call it "foo-4.7.x-1.0-dev"). again, i'm not optimizing this system for the lazy among us, but i'm also not going to burden them or penalize them if they don't play along. if they don't care, things are almost identical to how everything works now. however, those of us who care get *vastly* more powerful and flexible solutions to our problems that we can make as little or as much use of as we see fit.

i hope this is enough to convince everyone this is the way to go. i'd rather spend what little time i have for drupal hacking over the next month actually writing code and documentation to facilitate this proposal, instead of continuing to advocate for it. ;) however, i know this is a big change, and i'd like everyone to feel confident this is a move in the right direction before we start walking down this path.

thanks,
-derek

[*]: side note -- at my day job, we use branches like drupal uses sandboxes. it's a different scenario, since there's less of a core vs. contrib divide, and everyone is basically working on the same giant code base. however, any random project someone wants to do just gets a new branch. then, us core developers decide if/when/where to merge this branch back into a real release branch.

sorry to add yet more to an already bloated thread. a few (hopefully simplifying) thoughts from another IRC session :

i'd be willing to give up the flexibility for N different branches of a module for a given drupal core API and stick with just two: "DRUPAL-z-y" (stable, i.e. "foo-4.7.x-1.N" releases) and "DRUPAL-z-y-DEV" (development, i.e. "foo-4.7.x-2.N" releases). we can still keep the special meaning i'm associating with "4.8.x-0.N" as described in comment #17.

so, the only branch names people ever have to deal with are "DRUPAL-z-y" and "DRUPAL-z-y-DEV" (e.g. "DRUPAL-4-7" or "DRUPAL-4-7-DEV"). if folks really want to make a 3rd, "experimental" release for a given core API, they can use their sandbox, i suppose. ;)

the release tags should be of the form:

"[module_name]-Z_Y_x-M_N" (e.g. "project-4_7_x-1_3").

so, you can always split on '-' and get 3 fields: module name, drupal core API it's compatible with, version number. if you split the 3rd field on '_' you get 2 things: major and minor. major is either 0 (from the trunk), 1 (stable), or 2 (development). so long as we stick with this convention (which we could easily enforce with a smarter regexp in the xcvs-taginfo.php script, for those who know what that is), then everything would be swell, even using the "$Name$" RCS keyword expansion to imbed the version string automatically in source files (comment #9).

we'd still want the new tab/form on the project nodes to create point releases, basically as decribed by merlin in comment #6 (with obvious modifications for the version numbers).

with these modifications, hunmonk and webchick seem to agree this is a good idea, too. ;)

I feel versioning numbers should be as simple as possible. My vote would be for a module version based on the secondary drupal version (ie 4.7) plus a single incremental number for the module (the latest version of mymodule for 4.7 would have a tag such as 4.7-12).

I vote HEAD continue to be used for what it's pretty much used for now: latest greatest code that anticipates the next big release (ie 4.8). HEAD's only responsibility is to try to remain somewhat stable and functional in context of HEAD.

For any 'experimental' code, a branch should be created (for the module). This is what branches are for. Branches allow you to go off and completely do your own thing without messing anyone else up. IMHO, HEAD is not the place for dramatically new/different code. When your branch proves stable and plays well with HEAD, it can then be merged with HEAD.

If a bug is found in mymodule 4.7-12, then i check out that very same tag, fix the problem, and then tag it 4.7-13 and submit it for release.

Harry

-1 to harry's last comment. ;) *major* effort went into explaining all the reasons your proposal is insufficient. please do not change the title, or the scope of this issue. ;) i've already got both killes and dries happy with my proposal (which i was told was going to take a miracle). please don't start to undo any of that work.

thanks,
-derek

FYI: we've moved the discussion on implementation details over to http://groups.drupal.org/node/847

i wasn't aware that i was attempting to undo anything. sorry.

i was just pushing for simplicity where things were not yet finalized, such as version numbering.

henceforth i shall assume a secretarial role regarding this issue so as to not upset anyone. &*)

Don't apologize; that particular issue, tho, is a sensitive one, since quite a few people want to simplify it to just one version #, but I've argued extensively that I really really really really really really really need a major and a minor version #, and all modules that provide API to other modules also need this. And the argument got a bit hot. =)

Restoring the issue title.

Harry: somewhat confusingly, the 'title' is the title of the whole issue, not of your comment. We change it only if to refine it to better capture the issue.

I may be off base, but I'm approaching the project module from an external train of thought. I also hope this is the correct thread to post this on. Everyone seems to be thinking of the Project module ONLY in terms of how it is applied by drupal.org and maintaining module projects for drupal. If this is the case, then I would not make this module publicly available. I see a lot of other drupal based sites that would love to take advantage of this and the project_issue module, but they are coded so specific to drupal.org that they are mostly unusable by other sites, without some major modification.

For example, I would like to use the modules to track my own project, but my public web server does not have access to my code repository for security reasons. Now what happens when you base the entire versioning system on being able to look at a CVS repository? First of all you are locking your implementation to CVS (I use Subversion, like many more are starting to). Second, what if I don't follow the same versioning standards you all are discussing in this thread?

I'm just trying to think of the people out there, like myself, who have downloaded the project and project_issue modules and come to find that they are incomplete and totally specific to the drupal development environment. If this is how it is going to stay, then I would suggest removing these two modules from the modules directory until they are changed to be more suitable and complete for public use.

thanks for the comments, wersmart -- you're not totally off base (though i think your proposal to remove project/project_issues from the public site until they're more generic is a little extreme).

Everyone seems to be thinking of the Project module ONLY in terms of how it is applied by drupal.org and maintaining module projects for drupal. If this is the case, then I would not make this module publicly available.

that used to be almost entirely true. however, in the last few months, i've put *major* effort into changing this (which http://drupal.org/node/74995 is just another part of). it's still a work in progress, to be sure, but it's getting *much* better. my future plans for the direction of these modules it to make them better for drupal.org *and* external sites. i'm with you 100% on this.

Now what happens when you base the entire versioning system on being able to look at a CVS repository? First of all you are locking your implementation to CVS (I use Subversion, like many more are starting to). Second, what if I don't follow the same versioning standards you all are discussing in this thread?

good questions. i'll grant that the language in this particular thread is hard to follow, since there are 3 inter-related but separate questions going on:

1. what should our conventions in CVS be (branches, tags, etc)?,
2. what's the process whereby contrib authors make a new release?
3. how should those be made available for download?

we're not being all that careful to keep these separate, though they should be... also, keep in mind there are 3 separate (now 4, hehe) sets of code that have to be touched for all this to work:

1. the access control scripts that our CVS repository uses
2. the script run via cron that packages new tarballs of contrib releases
3. the project.module (to define releases and provide downloads)
4. the project_issue.module (to find out what version #s should be available for issues in a given project).

#1 + #2 are clearly drupal.org-specific code (though i'd still be in-favor of including these scripts as examples in the relevent modules, so sites that want to use them as a starting point have something to go from).

#3 from that list is an interesting question -- how much of this new method for adding a release should be done in the project.module (which is public) and how much should be in some drupal.org-specfic place? i tend to think that as much as possible, cool stuff you can do at d.o should be available for download if you happen to need the same thing. however, we should do this work in such a way that it doesn't *force* you to generate releases like this. how the project module currently "adds releases" is not great, and need to be improved. so, one thing to keep in mind while working on this thread is keeping the process for adding releases to project nodes flexible.

maybe project.module should just expose an API for how to associate new releases with projects, and then we'll write a new module to do all this cvs-specific stuff. maybe releases should just be a 1st-class node type (http://drupal.org/node/75053), so sites can add whatever fields they want to their "releases". these are the sorts of implementation details i was hoping to discuss in http://groups.drupal.org/node/847, since this particular thread is already huge and hard to follow as it is. ;)

#4 from that list is a known todo item on the whole reorganization to split issues into project_issue.module (in fact, i just submitted this as http://drupal.org/node/75565). ;) once that was done, project_issue.module wouldn't need to be modified at all to handle the new system for versions and releases. if you were using project_issue.module on your own site, you wouldn't see any change at all.

in terms of your final paragraph... there's a lot of code available for download at drupal.org that's a hell of a lot less "complete" than project + project_issue. ;) sure, it's not yet a 100% issue tracking solution, it's a lot better than nothing, and it's getting better all the time. while there are some lingering assumptions about how we do things on drupal.org, i know of quite a few sites (my own sites included) that make use of these modules, and don't need anything like what drupal.org is doing. plus, these modules would never get better if they weren't available for people to try out, suggest improvements, work on the code, etc.

so, i welcome (and share) your perspective on keeping these modules useful to places other than drupal.org, and look forward to seeing a lot of nice issues (and hopefully patches) in the issue queues to fix some of the lingering deficiencies. however, they're definitely going to stay up there, available for other people to download and use. ;)

thanks,
-derek

by the way, http://drupal.org/node/75053 (making project releases real nodes) is definitely an official part of this plan, now. that's been approved by dries and killes, and simplifies a lot of the problems discussed in here (e.g. making a new UI to define releases -- problem solved, it's node/add/project_release). oh, and i'm hereby officially working on this. ;)

here's the handout for the presentation i'm giving at drupalcon about this.
enjoy,
-derek

http://drupal.org/node/94000
ahh, one of the happiest "fixed" follow-ups i've ever made... ;)