GHOP #59: Document release logic for module developers

a) Yes, we need these docs.

b) You're in fact wrong about some of the details of how update.module notifies users regarding alphas and betas. This is yet more evidence to support (a). ;)

Seems likely I'll have to be the one to write these docs, so I might as well assign this to myself...

http://badcamp07.org/07/post/audio-talk
http://lists.drupal.org/archives/development/2007-10/msg00255.html

That's basically everything someone needs to know to write these docs. I asked multiple times for help (both devel list and documentation list) and no on volunteered. Maybe someone can write this up as a GHOP task, at least. I'm not the only one who can write this documentation, but there are plenty of other urgent tasks that require me, so it'd sure be nice if someone else took a stab at this.

Made into a GHOP task: http://code.google.com/p/google-highly-open-participation-drupal/issues/...

Please no one else work on this.

Assigning to self as GHOP. Will be listening to dww's talk shortly.

Here are some preliminary notes from the talk. A lot of it seems to already be covered in the docs.

Talk on maintaining your Drupal modules
=======================================

[1:28] Nobody's obligated to do this

[2:24] Once you upload it, you expect that other people are going to use it. Thus responsibilities:
1. Security vulnerabilities
2. Clear statement of intentions
  1. Forum post
  2. Sandbox
  3. CVS project without official release
  4. Official releases
For those of you who can't be bothered to maintain: if it's specific, don't release it; if it could be useful, post it, but say you won't be able to maintain it.
3. Keep userbase in mind (more popular modules have more problems)
4. Bad modules give Drupal a bad name

[13:52] Goals as a maintainer
1. Release things right, and less support requests arise
2. Make it easy to keep moving forward (new features)
3. Make it possible for people to use your module
4. Handle security issues quickly

[18:04] Three main areas of tools
1. Revision control system (CVS specifically): Branches and Tags
2. Releases
  1. Development snapshots
  2. Official releases (immutable)
3. Drupal.org project nodes

[19:20] Branches and Tags
1. Branch forks development progress
2. Tag gives a name to a specific set of files at certain revisions, cannot be changed
(Explanation of branches and tagging workflow)

[27:09] Releases
1. Official release = tag, makes things easy
2. Development snapshot = most recent work on branch/HEAD, must be dated
  - Never tell regular users to use snaps
  - Useful for helper developers who don't want to use CVS

- Branches useful for maintaining module code for multiple Drupal APIs. Older branches can be maintenance only.
- Keeps stable things stable while you do crazy development stuff.
- Release notes explain what each branch is, and changes in the release (are out of the way, so often this is reiterated in the text below the teaser) (different from CVS logs in that it is more condensed)
- Some talk about unit testing (Simpletest, Selenium)

[47:17] How and when to make a release

And there I stop. Will finish later.

Completed talk notes. These by themselves will be quite useful.

I'm a bit torn on how much to put on the documentation. Derik's talk was very informative, but having worked with version control systems before most of it is common knowledge not specific to Drupal. I suppose I'll just add the non-obvious aspects of CVS, but focus primarily on how it applies to Drupal.

Here is the deliverable: a new page on Module developer's guide called "Publishing your module on Drupal.org" with a tutorial on releasing modules that covers most of all the cases Derik covers in the talk. Images on the back of napkins may or may not be included :-)

[47:17] How and when to make a release
- Make a release when you think it's worth it for the users to upgrade
- When a new release is posted, admins *will* get notified about it
- Don't make an official release after every change
- Security fix get an immediate release
- Critical bugs should get a new release, use your judgment, many use a beta

[51:33] "update_status is smarter than you think"
- update_status is a module you install that tells you about what other modules need to be updated
- It knows what you have and what's available, and calculates whether or not you need to upgrade
- When you add a beta, it is added as "also available", but doesn't tell the person to upgrade; only when there's no "extra" column
- If you add a security release, update_status goes nutso with big red letters "YOUR SITE IS INSECURE" (this is one of the specific purposes of update_status)
Case study "update_status":
  - 1.x version was fundamentally flawed
  - Once 2.x-rc1 was released, it was recommended, since it was much better
  - "OH, YOU MUST NEVER, RECOMMEND, A NON-OFFICIAL VERSION URR!" -- Derek in a histrionic voice
- If you need to fix a release, make a NEW release (1.4, not 1.3-2; update_status can't understand it)
- Version with "extra" is not good
Case study "eCommerce":
  - Alpha version is recommended version
  - Admin has option to pick recommended branch
  - update_status will then prefer that branch over all the others
  - eCommerce set 4.0 as recommended, even though the only release was 4.0alpha
  - Then everyone complained
- Issue queues more looked at than forums
Digression: If you have CVS access to a project, you can do everything except give other people access
Future features: New vocablary, "paid-for", etc.
- CVS: node/59, send a description, once a week the CVS admin reviews the apps and accepts/denies
- Securities issues are good! They mean that the module has been reviewed
- CVS/SVN: Would require too much retraining/tool changing, for just one new feature (not enought better). Would require something more drastic like git, and not even

[1:20:40] Using CVS HEAD branch
1. Cutting-edge development on HEAD, always release from there, lots of work but most productive
2. Every new branch needs a branch, which then gets tags
Note that Core doesn't worry about this stuff (branch never adds new features (feature-freeze), all goes in HEAD).
- Branch lets you release a security fix without having to add all the other gunk (new features)

As long as you communicate your intentions, most things should be fine.

To clarify: the main point of this issue is the subset of that talk specifically about how the releases on drupal.org interact with update_status. Maintainers need to understand the results of their actions and how update_status behaves. That's the main thing that's not yet documented in the d.o handbooks. But, anything else you want to document after listening to that talk, or updates/clarifications/enhancements to the existing handbooks (especially all the stuff under http://drupal.org/handbooks/cvs), would be great. Thanks!

Marking needs work, to reflect dww's comments.

In that case, I suppose Managing releases is a better, albeit less visible, place to insert the docs.

Here we go:

http://drupal.org/node/197584

I was actually at this BADCamp talk, asking questions and whatnot.
So I'll offer some preliminary feedback.

First, great job in condensing it down to the essential elements.

It might be helpful to include the full tags that people would be typing in and what they're translated to.

Taking a look at the sticky tags listed in the web-based CVS of the Panels module gives some real world examples:
http://cvs.drupal.org/viewvc.py/drupal/contributions/modules/panels/?pat...

And looking at the official release node at http://drupal.org/node/194291 -- we can see that the CVS tag of DRUPAL-5--2-0-ALPHA14 was translated into panels-5.x-2.0-alpha14.tar.gz

So "DRUPAL-5--2-0-ALPHA14" = "5.x-2.0-alpha14"

So it might be helpful to provide the examples in one or both of these syntaxes because you're just using a truncated "2.x-rc1" makes it seem like Drupal 2.0 -- since the X usually implies the point release of the major Drupal release. So I'd recommend replacing the 2.x-rc1 with the full 5.x-2.0-RC1 to make this clearer -- and I think using the actual tag of "DRUPAL-5--2-0-RC1" will make it even more clear for people typing it into CVS.

The other feedback that I'd give is that I'd try to come up with more descriptive sub-headlines -- rather than the dummy case study title. So just skimming "Very Berry Baz" and "Foolish Foo" underneath the section of "Do's and Don'ts" doesn't give me very much useful information. So maybe come up with the essence as a title.

Otherwise this is looking really good.

I agree with kent's comments above. It's important to use "fully-qualified" version strings, instead of the abreviated versions. It's confusing enough as it is, I'd rather all the docs were completely explicit and used the full strings. Plus, including the corresponding CVS tags would help people understand what's going on.

A few other minor nits:

A) on d.o, if the version string is 5.x-2.3-rc the 3 in there (the last part before the "extra") is referred to as the "patch level", not "minor". It's a long story, don't ask. ;)

B) The do's/don't section should make it clear that the best approach is to wait until the X.x-Y.0 official release (no extra/beta/alpha/rc) before switching recommended versions.

C) The part of the intro that mentioned that update_status is now in core for D6 was lost somewhere from when I last looked until the current revision. I think this is important to keep in the document.

D) Maybe include a link to the BADCamp audio and your notes at the bottom?

In spite of the gripes, it's great to have this written up. Thanks again!

Updates posted. Here is the diff.

These updates are only for KentBye's comments. Addressing dww's concerns now

Derek's changes have been incorporated.

I did not add CVS equivalents for the sample release numbers littered further down the text, because I felt that would add too much gunk to the text. I can if you want, though. They have been expanded to their full version.

Okay.
ezyang is on top of all of my suggestions, and I had a few more.
He added in CVS tags in parentheses for each abbreviated use. Clarifying "CVS Tag of" with each first use in the paragraph.
He also changed the example from 5.x-1.44 to 5.x-1.38 with the latest release as 5.x-1.45 to give the impression of way too frequent releases to reflect dww's speech.
The titles are also much more descriptive of: "What Not To Do: The Case of Foolish Foo"

So he has my approval, and I'll let dww make the final call for the completedness of the task.

Great job ezyang.

Looking very nice. almost there. ;)

However, the case studies still give the wrong impression. It should be:

DO: only switch the recommended when you have an official X.x-Y.0 release without extra

DON'T: make a beta or rc recommended, unless you have a *really* compelling reason, like the previous version is so broken that even the rc is better that that pile of cruft.

NEVER: make an alpha recommended. if you want alpha testers, ask for them nicely, don't force your entire user base to upgrade to alpha code.

see what i mean?

thanks!
-derek

Case studies modified accordingly.

Whee this is fun. Good now?

Great, works for me. Further edits can always be taken on by the documentation team. I think you've earned your t-shirt for this one. ;)

Thanks again!
-Derek

FYI: I emailed the devel list about this: http://lists.drupal.org/pipermail/development/2007-December/027760.html