Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.
By Harry Slaughter on
Drupal Developer Releases
Though I've only been working with Drupal for about two months, I do have nearly ten years experience developing and testing (mostly web based) applications professionally.
When I recently took a contract to build a complete website in a short period of time, I began researching CMSs (which I've never before used). After about two weeks I narrowed the field down to Drupal, Mambo and TikiWiki. After more in depth research into these three solutions, it was obvious that Drupal was the most developer friendly and "least hacked" of the three, and I began to use it.
Because version 4.7 is soon to be released, I decided it would be wise to build my site based on the latest code. Since there are no development releases, I simply did what I assume most others do and used the CVS HEAD branch. This worked fine for a few weeks until about last week when I came across an apparent incompatibility with PHP5 and some of the code in one of the core modules. Since my Drupal base was derived from a non-tagged set of code, I now have no real way of reverting to a 'working version' as I have no idea when this incompatibility was introduced. Now I could stop all that I'm doing and investigate it, but that's not a very efficient way to work. In fact, I generally would never build upon a dynamic code base, but since that's all that was available for post 4.6.3 code, I didn't have another option. I'd like to propose the idea of developer releases for Drupal. I'm sure most of you know what I'm talking about already, but the idea is that relatively stable code bases that are not yet ready for the final release can be tagged so that testers and developers have reference points to work with.
Those with a bit of knowledge of other open source projects know that developer releases are common practice. I'm not aware of any major OSS project that does not adopt this process, but that certainly isn't to say they don't exist. Naturally, all professional development processes require internal development releases for the sake of testing.
----
Some of the benefits of developer releases:
- Something to test. One of the most fundamental aspects of software testing is to always know what you're testing. All testing, bugs, and fixes should always be relative to a static set of code (IE a version number). The reasons for this should be largely obvious, but the main reason is that there is no way to reliably reproduce bugs between non-identical code bases.
- Something for the hardcore Drupal fans to use (and test). Sites that don't mind being on the bleeding edge would have the option of running their site based on a developer release. This has the side effect of creating "betatesters" (assuming that those running dev releases could be encouraged to participate in the bug reporting process).
- Something to revert to. In the event that a serious problem is found in a developer release, all users of that release have the benefit of being able to revert to the previous release without giving up all the features associated with the dev code base.
- Accurate bug reports. Reduce the number of "works for me" responses to bug reports. If the submitter indicates the version of Drupal the bug is found in along with relevant system information, the developer should always be able to reproduce that bug.
I'll end the list there for now.
----
A comprehensive list of drawbacks:
- NULL
:)
Here are the arguments against developer releases (that I could think of):
1) It's too much work.
Response: cvs rtag -B -d DRUPAL-4-6-3-1 drupal
2) It's not worth it.
Response: If you find these tags meaningless, ignore them.
3) If it ain't broke, don't fix it.
Response: This would not be considered a fix but an improvement.
----
There are certainly tangential issues regarding this proposition, but hopefully those discussions can be put off until the primary concern has been discussed.
Thanks
Harry Slaughter
Comments
Yup... agreed
Harry, I think you've nailed an important problem on the head here. There's an inherent tension between two needs:
(1) Drupal developers and users need a stable base from which to do their own work. APIs change frequently and backward compatiblity is not valued. (For the record, I disagree with the attitude on backward compatiblity, but I don't see any prospect for change within the community here.)
(2) Drupal developers need to be using the latest code against which to submit patches. Code changes that don't apply against the latest CVS are generally rejected.
I've been using Drupal for about two years now. For most of that time, I've broken those guidelines and followed CVS HEAD for my own site development work. For the most part that has worked quite well for me. The 4.7 cycle has been much rougher than previous cycles. I've been badly burned by using the 4.7 work-in-progress, especially over the past month. For that reason, I'm beginning to think that I can no longer afford the time to follow CVS HEAD despite the appeal of cool new features that are under development and a desire to make contributions to the community.
This is NOT meant as a slam against the Drupal developers... really! This is a large and complex piece of software and it takes time and care to get it right, and they do get there. The designated public releases are very solid. I do commercial software development (large-scale desktop applications) in my "day job" and I'm very, very well aware that software has to get broken --sometimes for an extended period-- in order to gain the cool new features that we all want.
In their defense, I'll say that you were warned. The developer community recommends that you stay with released versions (the latest is 4.6.3) and warns that CVS HEAD may be unstable. Often times, they've been wrong and HEAD is very usable. But it's a risk and sometimes (like lately) that risk goes badly.
What you are proposing (and that I like) is similar to what I see in my professional work. We are increasingly focusing on "milestone" releases, where milestones are set every 4-6 weeks. New feature development happens for most of each interval. At the end of the milestone interval, we pause for a week or so to clean up, do extra testing, and resolve bugs. The end result of those "clean up" weeks is something we often ship to our alpha/beta testers to get additional feedback.
-Eric
--
Eric Scouten Photography | www.ericscouten.com (Drupal powered!)
Fine art prints from the world of nature
I like this
This would end the problems with months old bug repors against HEAD. Does it apply any more? What code base was that bug against? The bug report would be against DRUPAL-4-7-test-16 and that's it.
Failing back would be a great thing, too. Even if you are not supposed to run a live site with HEAD, there are sites which are, not so mission critical and you do run them with HEAD. Having a fall back point just gives you more testers, no doubt.
About when to tag: after you commit something major.
--
Read my developer blog on Drupal4hu.
--
Drupal development: making the world better, one patch at a time. | A bedroom without a teddy is like a face without a smile.
Yes ...
Nightly build releases will be very important for the increasing Drupal developer community.
Then some releases can be tagged as special milestones or whatever.
These are the two important tags:
HEAD will always be risky/flimsy.
nightly builds
i'm definitely not for or against nightly builds. to be honest, i don't really understand the point in them. maybe someone can enlighten me.
i guess maybe it is something like a mini release cycle that encourages developers to leave HEAD in a functional state before the nightly build takes place?
--
Devbee - http://devbee.net/
nightly builds, hmm
I fail to see why to use nightly builds. We don't build anything, its PHP. We don't change teh code that frequent that it would need an everyday test.
Some of my devel sites run a cvs up @ 00.01 every day, that /is/ a nightly if you wish ;)
---
[Bèr Kessels | Drupal services www.webschuur.com]
bad language
i think we both meant to say 'nightly tags'.
i agree it is unnecessary.
--
Devbee - http://devbee.net/
Limiting attitude.
I find that attitude very narrow minded.
If you demand that users learn CVS to be able to test something, I think that's very arrogant and unnecessary.
Drupal developers sometimes seem to have issues themselves with CVS, even if using the web-based viewcvs or other.
Shrink-wrapping into a build to let plain users test their cases is still the best in my opinion.
I thought Drupal core developers were all for as much testing help as possible, but apparantly there are some minor self-contradicting attitudes.
I don't understand how you fail to see the logic in letting users test recent developement versions in an uncomplicated manner.
Users can download the cvs
Users can download the cvs package that's rebuilt once or twice per day, not sure which. And they can report bugs precisely by reporting the date/version that CVS inserts at the top of each file.
That's easier than learning CVS...and really, it's the best you can do without learning CVS. No patches.
You are wrong.
That date is no better than the version number on the same line. It is the date of the CVS commit, not the "snapshot date" of the system.
If you think that simple users will sift through modules, tracing PHP debug to where the offending module/code is located and report this in an intelligble way, you are completely wrong.
Users need a simple way of reporting:
"I took the dd-mm snapshot and tested it for the bug, and it is still there."
not:
"I used a snapshot - don't remember which date exactly - but the error is there."
When you aggregate this into a lot of bugs/issues, testing and many volatile versions .. you need a simple system.
Not even all Drupal developers understand how CVS works. Internally CVS keeps the latest version as full text, while earlier versions gets truncated into diffs, because statistically the latest version is what gets requested most often. CVS, RCS and all the old versioning systems are quite "beastly" to master well. Don't ask this of users who are willing to help testing and doing regression testing. Testing is very extensive and laborous work, and every pair of eyeballs will help. If you have any experience in developing software/systems with a user interface you will know this.
that's all I need
I just find that's all I need to locate the affected version. If the person reporting the bug doesn't know how to use CVS, they're not going to submit a patch, at which point I don't see an advantage to me, as a developer of someone saying "the bug is in dev release 5" versus "the bug is in the version committed on this day." I still have to go get that version/dated file, duplicate the bug, type in some code and generate the patch.
I'm not. More testing is welcome, I just feel it can be done now. I see this like it's switching red labels for green lables.
Testing and patching are very different contributions ...
I do not lessen the act of testing on the part of less advanced users to the act of submitting a patch by a developer. I consider both acts valuable contributions.
Can anyone with experience from large software projects truthfully say that (regression) testing is any less important than submitting patches - with regards to project progression, bug-resolving and QA in general ?
I remember when switching to ISO 9002 and more strict QA when working in NOKIA Telecom. R&D ... or when rolling out updates and specialized versions of TANDBERG videoconferencing software on embedded platforms; regression testing was an immensive integral and important part of the final release process. I don't think anyone demands the same stringent development process by Drupal core developers; but why hamper the process by not opening for a simplified testing process so that any user easily can make a very valuable contribution ?
Worrying about more bug reports or bugs discovered is not something which works for any serious project in the long run either. Instead, it's better to embrace whatever opportunity to get better progress and quality.
You also misunderstood the problem I was pointing at. It doesn't help if you know the year, the month, the week or the day of a CVS snapshot ... because 2 minutes after that snapshot, a lot of patches might have been committed. That's why a fixed snapshot with an unambigous label is needed.
The complete build into a fixed timeframe helps less advanced users setup the testing environment and report back on any issues.
I haven't lessened anything.
I haven't lessened anything. I'm pointing out that the actual tasks each has to perform don't change because you use development releases.
If most people find a particular kind of labeling easier to deal with, fine. I use CVS (on Windows no less, proving my technical tyro status), and when I finish coding I do an update before rolling a patch, so it will have no impact on me. I'm just saying there's no obstacle to folks who are interested but unfamiliar with CVS from testing as things stand. I'd rather things get tested by as many as possible.
Obstacles in the wrapping ...
The point I'm making is that needing to check out a volatile snapshot of HEAD from CVS is an obstacle. It's an obstacle to correctly reporting bug status. It's an obstacle to those who are unfamiliar with CVS. It's an obstacle to testing.
Without trying to make this a pissing contest about experience (although some might misread my call for recollection [if any] of large software projects as such), I think we can agree that unsophisticated users of Drupal will have some problems in testing and accurately reporting bugs discovered - or as reported by others.
If you can't accurately describe what you are using, how can you explain to others so they can see for themselves ?
needing to check out a
http://drupal.org/project/Drupal%20project/cvs
There's a download link at the bottom of the page. Checking out code via CVS isn't necessary.
sigh...I think I need to keep my minimalist butt out of these conversations and just write the code I need to write.
Easy linking was not the point ...
The point was that it was a volatile snapshot of Drupal CVS HEAD. As I explained that will profoundly shape how you can set an accurate status of a bug with regards to a test installation version.
When a user takes the snapshot of Drupal CVS head for a test installation; a microsecond after - the next user taking a snapshot of Drupal CVS head for a test installation could have a completely different source base.
It could lead to very confusing conflicting bug reports, where one user says: "it's a bug on Drupal from day dd month mm" and another user says: "sorry, friend, I have a test installation from day dd month mm as well, and it's working fine here. It must be specific to your setup - or your fat fingers". So which one is it ? A bug or not ? Then someone else need to look into the issue .. and so on ad nauseum.
We're into hairsplitting now, avoiding the logical point being made, and commenting for the sake of discussion itself.
I see no argumentation of why you think (the volatile) CVS HEAD is good enough, and not making an easy installation process for testing environments with a proper release which can actually be confirmed and recreated by other users. Instead of being minimalistic, it seems like omitting the actual point. Frankly, I don't see how I can phrase this any clearer to get the logical point across.
Your logical point is clear
1- CVS HEAD is what we develop against.
2- The tagging doesn't change what is necessary to install a testing environment
Your logical point is clear...it's just that tagging alone doesn't do it.
I don't argue, by the way. I just point out what I see. And if you do start a thread on improving the testing situation for non-techies, I'd be happy to join into a DISCUSSION...you'd find out goals coincide, I'm just seriously realistic about who is willing (and capable) of doing what. But that's for another thread...or not at all, who knows?
what?
As others have mentioned, there are not commits on a daily basis. So how can it be limiting to tag releases when you hit the agreed upon miles stones? Please, lets keep this at the discussion level without the extraneous judgements :)
-sp
---------
Test site, always start with a test site.
Drupal Best Practices Guide -|- Black Mountain
-Steven Peck
---------
Test site, always start with a test site.
Drupal Best Practices Guide
Sure ... don't have to be daily/nightly
Like I said earlier (further down on comment-listing):
"Nightly, or weekly builds if activity is that low, makes it easier to report bugs, find bugs and do regression testing."
I'm not specifically adamant about a daily timeframe, which would be unreasonable. However, a certain fixed timeframe is very sound thinking in my opinion.
With fixed timeframe complete builds (not tagging CVS only), users can always follow the semi-latest Drupal updates and actually make intelligble reporting on bugs and issues. The fixed timeframe can be whatever reflects the developement activity.
Eclipse and many other large projects use a nightly timeframe, but that might not necessarily reflect the level of activity among Drupal core developers which use spare/free time to maintain the project without quitting their dayjob.
As we know the Bryght and Civispace spinoffs from Drupal have full time developers, but they are in no way obliged to update Drupal core on a daily basis. Besides, Dries seems to keep a tight quality control on contributed code back into Drupal core, which probably keeps HEAD a lot more sane than a more free-for-all approach would end up.
mixup
first of all a "build" generally applies to applications that are compiled.
second, i think what you're referring to is a nicely packaged version of the developer release that users can download and install in the same way they do production releases.
i have no idea what the release process is for drupal, but if the packaging is automated and can be run against an arbitrary tag, then i'm for it too.
but doing this on a daily basis would be counterproductive.
the time between dev releases would probably vary based on what's being develloped. but i'd think the dev releases shouldn't be more than every two weeks or more.
--
Devbee - http://devbee.net/
I agree, minimal timeframe is not important
What suites Drupal core developers, and Dries especially, is what counts. He has the full right to determine the pace of the developement and frequency of any releases.
It is also beneficial to Drupal with Dries' and others' strict QA control of contributed patches to Drupal core.
Nightlies ..
As you also stated, HEAD is a very volatile reference point.
If you consider bug-reporting systems, how would you report a bug against a HEAD snapshot you did some days ago ?
In your first post, this was one of the main problems you showed concern about.
If you wan't a low threshold for reporting and testing you need to put some effort into making testing easy.
For an CVS-unwitting user, it's easy to report a bug related to a reference point - and which is easy to recreate by anyone by using similar data on similar build.
But if the report says a HEAD snapshot some time ago, the only response would be: "Please test again on HEAD to see if it's still a bug."
In the case of HEAD being even more broken or problematic for the user, the user can never reconfirm the existence of the bug, nor do any regression testing on earlier snapshots - save major ones like milestones, release-candidates or what have you.
Nightly, or weekly builds if activity is that low, makes it easier to report bugs, find bugs and do regression testing.
If you are serious about calling for more bug-squashing and regression testing, I suggest you consider carefully all beneficial measures and not squabble about features not specifically targeted to yourselves as developers.
As everyone with background in professional software developement like yourselves know, regression testing is tiresome and can be extremely overwhelming. The best way to get the job done is by streamlining the bug-reporting and bug-report updating/resolving.
With experience from very large software projects and testing suites, I think nightly builds - or equivalent larger timeframes - are important to developers and testers.
Back on track
eldarin, the start of this topic was developer releases -- i.e. not a focus on end user testing. I think your comments have clouded the issues somewhat. Feel free to start a thread on end user testing of non-released code, but let's keep the conversation on track here.
Over to you, Harry and rest of devs.
My opinion on the whole issue: needs infrastructure support in project.module for one, but it's ultimately a developer issue.
And: might this be easier with Subversion? -- that is, could moving to subversion and performing the infrastructure changes necessary for that get us to the same end result?
Project.module
Aren't release tag-labels like "4.6" , "4.5" etc. based on some sort of tag-definition table ?
I admit that the thought of having to serve pages with a (quickly) several hundred items large dropdown selector is nauseating.
An idea could be the following for supporting a large number of selectable tags:
use javascript to verify the tag from a text-field. The tagname would include a crc-like part which could be a modulo of the date or similar. That would minimize the risk of people doing typos when typing the tag-name they refer to in the project issue report. On submission the project module would check if the release/build tag-label was valid in case of typed tag-label. Selection from the "short-list" would fill in the tag-label text-field.
An example of such a tag-label would be:
drupal-cvshead-20050912-1
which would mean a CVS snapshot from head tagged at midnight/noon/whatever GMT and 20050912 modulo 9 = 1 for a small input check and avoiding typos.
Thank you
Thank you.
beta release tags and subversion, please
I think svn would be a huge improvement, but you may still have similar display issues. Since each commit causes an increment in the repository revision number, presenting a list of all the revisions to the developers or users to download and install would be confusing. Rather, only trunk and a list of tags/branches should be presented.
svn would be a significant improvement in that finding which revison you are working with is easier than having to match a ui issue to the datetime head was downloaded and then correlate that with the source file(s) in cvs at that time. (Not to ignore svn's numerous other advantages, of course).
I think we should move to svn for the developers benefit, and tag beta revisions to simplify the reporting process for users and beta testers.
milstone tags, are a good idea.
Not only because it allows for a more stable development no HEAD. but also to work in a more milestone (task) related way. OSS projects, and Drupal too, often have the problem that work is done very ad-hoc. Patches are submitten "when time permits" No-one has any idea to predict when his or her patch will be committed. If at all. No one has any clue what to expect "tomorrow". CVS logs are far too dynamic, and cluttered to keep up with on a daily (hourly) base, and to filter the important changes from.
starting with some basic milestone releases will help this a bit, since it will make it a lot clearer what is going on.
I am definately supporting milestone development releases. As long as they are clearly devel releases
We should avoid at all costs to get "Why Module FooBar breaks on 4.6-test-15". Support for devel releases is a no-go area, imo.
---
[Bèr Kessels | Drupal services www.webschuur.com]
Just use SVN!
Subversion has commits, so that the whole version controlled module gets a new version number with every commit. That actually instantly solves the versioning problem... From then on, you can reference SVN version numbers, and can easily get to that version.
Note that you can do the same with dates in CVS, but it is not too easy...
yeah. i think svn is a better solution too =)
Managing branches with cvs is a pain in the fscking ass.
--
The future is so Bryght, I have to wear shades.
Hehe, branching ...
Yes, doing concurrent merging and keeping branches up to date is really a pain with CVS. It works, but demands a lot of CVS/diff expertise many times.
I never used Subversion or any newer versioning systems since CVS is so prevailent in most established developer environments using other GNU tools.
E.g from my experience having vendor libraries which need tweaking on embedded platforms and then keeping these updated when the vendor releases new version ... it's a royal pain in the rear.
But ultimately branching is a developer issue, not so much a user or testing issue.
This thread was started as a developer issue, but I feel that "more birds could be killed with one stone" if easy access was given for users to participate more actively in testing - as requested some days ago by other core developers.
I guess keeping CVS branches active for a large set of modules from Drupal is a veritable nightmare.
too many tags
one of the main purposes of dev releases is so that testers and developers have identical code bases to find and fix bugs in. these releases would need to be installed just like a prod release. the assumption would that that as of DATE, testers and developers are doing their work based on VERSION_X.
a 'tag with every commit' doesn't give a single useable baseline to use for a period of time.
--
Devbee - http://devbee.net/
not an SVN thread
if you'd like to debate the merits of SVN (which I know nothing about), please start a new thread.
--
Devbee - http://devbee.net/
Aye.
I have not really had time to read the entire thing but I definitely support a developer release of Drupal.
--
Please read the handbook, search the forums, then ask...
http://drupal.etopian.net (Modules and Drupal Services)
Me too
Sorry for just "me-too-ing" but I recently were in a discussion with someone who also uses HEAD, and we spent a lot of time discussing whether the problem still apllied because we had different HEADs (drupal, not the heads we seemed to have up our asses).
precisely
precisely the type of situation dev releases would help prevent.
--
Living in fear of patch hell?
Want a stable development environment?
Support Dev Releases: http://drupal.org/node/30903
Support Code cleanup too: http://drupal.org/node/28540
--
Devbee - http://devbee.net/