Note, 2012-04-06: The current backport policy is now documented here: Backport policy

The process for backporting fixes from Drupal 7 to Drupal 6 was/is quite broken.

#895014: All fields of a node type are lost on module disable and #1017672: D6 to D7 update process permanently deletes comment bodies and other data, and throws fatal SQL errors are the best examples of it not working at all - i.e. we have a bug in Drupal 6 that blocked the Drupal 7 release, but released Drupal 7 anyway because there was /never/ an issue or patch for Drupal 6 in the queue, and it also wasn't being tracked in the Drupal 7 queue either.

Another issue we have is that patches to Drupal 7 often needed followup (minor or major bug fixes, documentation, tests added) for Drupal 7, as well as a backport to Drupal 6, and a single issue can't handle both things at once across two version.

So let's fix that for Drupal 7.

There are at least three things to figure out:

- Once Drupal 8 opens up, do we always patch against Drupal 8 and backport, or will some things go into 7 and forward port?

- For backports, should we start opening parallel Drupal 7 and 8 issues, but mark one or the other postponed?

- There's been indications from Dries that API additions, performance improvements etc. might well be accepted to Drupal 7 if they don't break existing APIs, but I don't know to what extent this has been discussed, or what webchick thinks about it either. I know I personally have several performance patches (mainly around i/o and memory at the moment) split between the Drupal 7 and 8 queues, that once ready would be backport candidates, but I'm not even sure which version to target the patches at at the moment.

There are 224 Major issues against Drupal 7 at the moment, not to mention all the 'normal' ones, so this is something we should figure out a process for sooner rather than later.

Posting this against Drupal 7 since that's where the backports land, and it will likely be webchick doing a lot of the commits.

Marking this as critical since it should be sorted out before we get to a 7.1 and/or before 8.x is branched if at all possible.

Comments

mlncn’s picture

Subscribing, and voting for: 8 first, open parallel issues for the backports, and allow as much as possible to be backported including API additions.

Actually, bugs and perhaps major performance issues should be allowed to be fixed in 7 first; 8 is the default and a case has to be made for doing 7 first (such as: "the proper fix will break APIs"); either way, one issue is active and the other is postponed.

benjamin, agaric

klonos’s picture

I believe that it is best to perhaps be keeping an eye in the core's usage stats and act based on core version popularity or adoption growth rate if you like. IOW, by looking at the stats and the resulting graph, one would realize that:

- D5 has more or less remained the same with a *very* slow tendency to drop. If one sees its stats in bulk, they'll see that throughout 2010 it was mainly fluctuating between 12k and 15k (max was 20k, but at certain/random periods only).
- D6 was 250k and reached 300k mid-year before finally going 350k!
- D7 was 1k at the start of the year and reached 5k to 6k by the end of it. When the final release was near, it jumped over 20k.

This observation of numbers actually happens to reflect how major versions of core were/are treated:

- D5 is officially now the deprecated/non-supported version, but was considered to be obsolete for quite some time now by most.
- D6 is what the majority of the community uses (and my personal opinion is that it will continue to be so for some time).
- D7 is the new player and it did have a great start, but still it is only used by roughly 5% of the people.

The way I see it, there are two sort of patches, ones that fix things and others that add new stuff. These pretty much reflect the available options in the 'Category' drop-down menu of the issue creating form: 'bug report' & 'feature request' (if one excludes the 'task' and 'support request' items that don't have patches throughout their 'life' anyways).

I guess that what I am trying to say is that patches that resolve issues should perhaps be -for now- implemented against D6 then forward-ported to D7 and finally to D8. An exception to this would be D7 *specific* issues that will of course be moving 'upwards' only (D7 -> D8). This practice would help both end users and developers as well in the long run. End users because the fix(es) would be first released for the version that is used by most of them and developers because they will have more feedback than going the other way. That translates to: more 'satisfied customers' + faster development.

Patches that add new features should be D7, then *first* backported to D6 (while and for as long as it remains the choice of most and if applicable), then finally forward-ported to D8. If a new feature brings API change(s) though, it should be D8 -> D7 -> D6 (if and where possible/applicable).

Once we have the majority of live sites moved to D7, then can rethink this and have fixes go D7 first and then D6/D8. Features with API changes should always be D8 -> D7 -> D6 IMHO though.

Thanx for taking the time to read all this ;)

klonos’s picture

I forgot to link to the stats so that others can see them + I wanted to add that if one looks at the graph, they'll also realize that the 6.x line (blue) happens to move along with the total (orange). Just another clue to realize that the majority of people still use D6 and that fixes should be focused in that branch -for now-.

David_Rothstein’s picture

@klonos: Those stats are likely way off for Drupal 5, as it says at the top of that page; it misses tons of D5 sites that didn't go out of their way to download and install the contrib Update Status module. It is also likely miscounting a bunch of D7 sites, since there are mass-hosted D7 setups such as Drupal Gardens and others that deliberately don't turn on the Update Manager (since the site admins don't manage the codebase) and therefore don't report back to drupal.org. Clearly, Drupal 6 is still currently the most popular by far, but I really don't see how we could have a policy of fixing bugs there first, not without a massive commitment from developers to actually work on forward-porting the fix once it's fixed in D6. I don't see any evidence that that commitment is there.

I like the idea of having two issues for the same bug (one marked postponed) though. I'm not sure it would necessarily have to be one issue for D7 and one issue for D8, at least not for starters. Maybe instead, when we find a bug that affects D7 but which we'd like to do a more far-reaching fix for in D8, we'd open up two D8 issues. The first one would focus on the minimal fix with the intention of getting it quickly committed and backported to D7. The second issue could be for everything else, including non-critical followups, etc. That way the bug gets fixed in D7 as quickly as possible, but there's still no chance it will accidentally get left out of D8 either (if the more far-reaching fix gets bumped to Drupal 9, then at least we still have the original minimal fix in D8 already).

catch’s picture

@David - I like that idea as well.

plach’s picture

klonos’s picture

@David:

Those stats are likely way off...

...fair enough. My whole reply in #2 was based on these stats, so I guess I was mislead ;)

...besides, I am still way far down in the learning curve (from starting to commit fixes myself) to express a strong opinion over this matter. I don't mean to say that I don't care, but my post in #2 simply revolved around the

That translates to: more 'satisfied customers' + faster development.

idea.

Thanx for taking the time to read my thoughts everybody. I really appreciate it ;)

lotyrin’s picture

I'd really like to see project_issue fleshed out to support explicit relationships between issues and/or multiple versions or projects.

Sometimes a fix for an issue exists in more than one project (inter-project interoperability, for example) or between versions (forward/back-ports) and it really ought to show up in both queues. Sometimes things bikeshed and you need to break them apart to keep the issues manageable, but you still want to be able to keep track of things.

I'd lean toward multiple issues, as the patch always comes before the port, and one side of an effort to increase interoperability might get completed before the other, and complex versions/projects don't solve issues of bikeshedding/issue splitting at all.

Something like adding a new table to allow relationships like issue "X is meta containing/is a duplicate of/is a port of/depends on Y", and adding them in a more structured way than linking to them from comments, so they can be tracked.

But then, there's a lot of things I'd like to see in project/project_issue, and I don't know if its going to be feasible to wait for them to be developed before we start having to think about Drupal 8.

Dave Reid’s picture

.

pillarsdotnet’s picture

Nephele’s picture

I'd like to express support for the overall idea of establishing how patches are now going to be handled. And in particular, I'm in favour of the idea mentioned in the original issue that "API additions, performance improvements etc. might well be accepted to Drupal 7 if they don't break existing APIs".

As an example, I just posted a patch to #366511: Decouple Comment module from Tracker. It's a two-year-old 7.x feature request that as far as I can tell got bumped to 8.x primarily because it was left sitting around for too long -- and therefore the patch was no longer compatible with HEAD the next time someone looked at it. Even more importantly, the proposed patch would have no effect on the API or any existing users of the module. So it seems like a perfect example of the type of feature request that could still be considered for inclusion in 7.x.

From the point of view of a new Drupal user, it would be really nice if some feature requests like this could be considered for 7.1. I'm just trying to put together a new Drupal site with the features requested by my bosses, and in the process finding tweaks that I need to make to the code -- tweaks that I wasn't able to contribute at this time last year because I'd never even heard of Drupal a year ago. I'm willing to put some extra effort into converting my tweaks into proper patches, and contributing those patches back to the community, but realistically I (and presumably others in similar shoes) am only going to make that effort if it seems like my patches will ever make it into an update that I'm going to see. Not only does Drupal 8 seem so remote right now, but additionally I'm putting together a site in Drupal 7 specifically so that I won't have to do a major version upgrade for as long as possible. So I have very little motivation to contribute patches to core if those patches are for Drupal 8.x -- even though I realize that's how Drupal's development cycle works.

Therefore, my thoughts are that I'd like to see some guidelines that allow patches to be applied to 7.x (and subsequently forward ported to 8.x). This would only be appropriate for certain types of patches (one criterion presumably being only patches that don't change the existing API), but those patches would include feature requests as well as just bugs. And at some point in time, it will become too cumbersome to regularly forward-port patches -- maybe after six months, or maybe after 7.1 is released -- at which point patches to 7.x would be less acceptable.

rfay’s picture

subscribe

rschwab’s picture

+1 for David's outline in #4.

Crell’s picture

+1 subscribe.

alexanderpas’s picture

I personally think we need two approaches here:

- Bug Fixes:
Should be fixed in the lowest (supported) version where the bug first popped up, with parallel (postponed) threads for higher versions.

- New Features:
New Features should be posted against the highest Drupal (development) version, and only when it is implemented there, we can think about backporting.

bleen’s picture

I agree with #8 ... in order for a two-ticket approach to be viable we need to have a better way of linking two issues together then just a random comment somewhere in the thread indicating a related issue.

catch’s picture

Not the same thing as #8/#16, but a quick way to link those issues together would be #1060798: Enable apachsolr 'related posts' block for issues (and forums?) - which I've wanted for ages but this discussion reminded me to post the issue.

klonos’s picture

At some point I have several times suggested in various issues (mainly for project_issue) to add the feature of being able to link issues with each other. This would be a 'depends-on' & 'is-blocked-by' relationship kinda thing, the way it is done in bugzilla. Here's a screenshot that I've posted in #651484: Enable CCK and node_reference - it shows what I'm talking about:
 

As you can see, it would also provide information of what needs to be fixed first for issues that require other issues to be fixed before they move on + provide a graphical tree view of related issues so that people know where they stand and get a rough idea of the percentage of each issue's solution completion (so you have less people asking "Are we there yet?" questions). Pretty much like the way trac does it in its Roadmap feature.

Seeing what Alexander suggests in #16 gives me the opportunity to add to my proposal to also have some sort of 'is-backport-of' & 'is-forward-port-of' relationship between issue nodes. Why do I say 'issue nodes' you ask? Because the only way I see this being implemented is through Node relationships or NodeReferrer (perhaps Reverse node reference too).

claar’s picture

Agree with #4.

klausi’s picture

I don't like the idea of forward porting patches instead of backporting them. When we fix bugs we want to make sure that they don't come back - if they are fixed in D7 only and nobody cares about porting them to D8 we will get a flawed D8 development version. Then we add features to D8 without the bug fixes in place and we get a real mess. We risk that some bug fixes never get forward ported and as a result D8 will have regrsessions compared to D7.

So I think splitting issues per Drupal version is a good idea, but a patch should not be allowed to go into D7 before the corresponding D8 patch has landed. We want to encourage people to write patches against the version that they use in their production environments, but we also want to ensure that the Drupal code quality does not decrease for future versions. So we have to make clear that no bugs are left behind and that the most recent development branch is a priority. You are welcome to develop and share your D7 patches, but don't expect them to go in as long as the issue in D8 is not fixed.

Well, at least this is my opinion.

jherencia’s picture

Subscribe.

Lars Toomre’s picture

I appreciate the need/desire to keep the current development environment current for all known bugs. However, given the long lead time before anyone will ever use D8, I think that it is also incredibly important that bug patches also be prepared for D7. Too many times we saw items patched in D7 and never back-ported to D6 as an example.

Perhaps for some period of time (18 months?), the rule should be that no bug patch gets applied to D8 until both the D7 and D8 patches are ready to be committed. That way both the development code and the production code are kept current for bug fixes. (Ideally someone focusing on the patch/bug issue will also address what needs to be done for the version two back [e.g., D6] if the bug exists in that code base too.)

My two cents ... Cheers!!

lotyrin’s picture

I've seen a few suggestions like "changes shouldn't get applied to a until ready for b", and I'd like to shoot all of those down. If the code is ready, it should be in the repo. With the move to git, its questionable if we should even wait until it's ready (per issue branches, etc.).

I'm not sure what problem these suggestions are trying to solve, anyway. I don't see anything wrong with there being a fix or feature with a later back or forward port. If features or fixes are sitting around without backports, that's a problem with there being a lack of effort or interest, not a problem with the process.

mikey_p’s picture

As soon as the git migration is complete, I'm planning on moving https://github.com/shomeya/pi_related over to be hosted on d.o and plan to start working towards getting it deployed on d.o eventually.

klonos’s picture

README.txt:

This module provides a simple mechanism for relating issues from Project issue
tracking module, to other issues, and provides backlinks for those relations.

This module also allows administrators to define custom relationships that can
be used for relating issues.

...great! Keeps us updated Michael.

dww’s picture

@mikey_p: We should fast-track pi_related. I think that's going to help a *lot* of our problems in the issue queues. Frankly, I'd like to just merge it into project_issue itself. That could even start before the Git launch. ;) We should really be discussing this elsewhere, but just wanted to express my support for getting that done ASAP, not "eventually", as folks figure out the plan for D8 dev.

Generally +1 to separate issues for each version of core that are all linked together. Each one can be in a separate state, assigned to a different person, different patches and test bot results, etc. I think it's just cleaner all the way around, instead of making monster issues that are harder to follow.

Cheers,
-Derek

David_Rothstein’s picture

I've seen a few suggestions like "changes shouldn't get applied to a until ready for b", and I'd like to shoot all of those down.

Those aren't suggestions, exactly - they're reiterations of the current policy :)

Consider this: Tons of time and effort was spent adding a testing framework to Drupal 7, the main point of which is to avoid regressions. The idea is that once a bug is fixed, if it comes with a test it will never be able to reoccur again in a future version Drupal. So I really don't see why we'd want to introduce a policy that deliberately end-runs around that and allows or even encourages regressions to occur.

My suggestion in #4 was intended to be a compromise that allows bug fixes to be backported as quickly as possible, while still avoiding the possibility of a regression... Is there any reason not to go with that?

catch’s picture

Status: Active » Needs review

I don't think so, and if enough people have expressed support for #4 (and even started writing modules to assist that proposed workflow) I'm marking this RTBC in the hope we can get some feedback from webchick and Dries.

catch’s picture

Status: Needs review » Reviewed & tested by the community

erm, actually RTBC this time.

webchick’s picture

Status: Reviewed & tested by the community » Needs review

Actually, I'd like the opportunity to chime in here, but I'm traveling.

mikey_p’s picture

Code for related issues module is now on drupal.org: http://drupal.org/project/pi_related

I'll be working towards an alpha/beta release by Drupalcon Chicago.

Pancho’s picture

+1 for David's proposal #4.
Keeping a minimal fix and the Best Solution™ apart is a very good idea.

sun’s picture

I'm concernerned about 1. Duplicating discussions in more than one issue (especially when follow-up problems may be found in one of the forked issues), 2. Historical value of commit messages pointing to forked issues not containing the full and proper context, as well as possibly also not providing correct history and credit.

catch’s picture

I discussed this in person with Sun at DrupalCon, and agree splitting issues is potentially a worse situation than what we have now, forking discussions etc.

However I'd be very up for splitting stop the bleeding fixes from perfection, with that latter worked on and committed to both versions unless there's a really good reason not to. For those it's fine to have two or more issues, since we often end up with this anyway.

Also, bump.

klonos’s picture

Perhaps we should take another look at #66484: Allow issues to be filed against multiple versions/branches. then(?). This way, we can have people working in various versions affected. The only thing is that patches uploaded for testing should be marked/tagged/named in a way that describes what version they are aimed for. As far as I know the testbot only runs against D7, so this should be considered too #1007210: Test bot should run tests against the core version the issue is set to..

klonos’s picture

Alan D.’s picture

My 2 cents...

Having multiple issues for the same issue is messy. It would be nice to be able to combine the status of the bug for 1 or more versions within one issue. Similar to:
--------------------------------------------------------------
Version Status
[8.x-dev] [closed (fixed)]
[7.x-dev] [needs review]
[6.x-dev] [active]
[5.x-dev] [closed (won't fix)]

Track another version [4.7.0] [Add]
--------------------------------------------------------------

David_Rothstein’s picture

So is #4 still RTBC?

@webchick set it back to needs review because she wanted to comment on it, but RTBC normally means "it's time for webchick to comment on it" so I'm confused :)

I don't think anyone's said anything that's particularly opposed to that proposal recently, though I could be wrong. It would be nice to get that settled before this issue starts going off too far into implementation tangents and such.

dww’s picture

@David_Rothstein: No, there have been a couple of objections to #4 -- see #33 and #34 (two of the very top-most core contributors). So, it's not exactly RTBC yet, since we need to address their concerns.

Re: #37 and the general approach of trying to track lots of different versions in the same issue -- ugh! Issues are already too complicated and trying to cover too many things as it is. It can be overwhelming to figure out wtf is going on -- imagine how confusing it's going to be to have multiple versions and status values that people are trying to update with their comments. Plus, it's going to require tons of new code to make a complex UI even busier.

I believe the main objections to multiple issues is how poorly the issue queues handle related issues. If we deployed http://drupal.org/project/pi_related on d.o, I suspect a lot of those objections would go away.

I discussed this in person with Sun at DrupalCon, and agree splitting issues is potentially a worse situation than what we have now, forking discussions etc.

Can y'all be more specific? There's not much there for me or anyone else to respond to. "Forking discussions" can be a feature, not a bug, depending on how it's managed and handled -- I'd rather see a few coherent threads for discussing different approaches to the same underlying bug that are appropriate for each version of core than to try to untangle that whole mess inside a giant monster issue.

From a purely GTD-esque approach, it's better to have lots of smaller issues that can actually be understood and completed than to have a single "todo" item that can't be resolved without breaking it down into smaller chunks. We need something to organize the bigger groupings, but there should be nothing wrong with lots of smaller sub-issues related to a larger effort. I'd rather fix the tools to better support grouping lots of smaller related issues together than to try to keep everything happening in a single monster UI that tries to be all things to all people.

claar’s picture

@#39 It appears to me that Catch in #34 is supporting #4, not objecting to it:

#4: Maybe instead, when we find a bug that affects D7 but which we'd like to do a more far-reaching fix for in D8, we'd open up two D8 issues. The first one would focus on the minimal fix with the intention of getting it quickly committed and backported to D7. The second issue could be for everything else, including non-critical followups, etc.

#34: However I'd be very up for splitting stop the bleeding fixes from perfection, with that latter worked on and committed to both versions unless there's a really good reason not to.

catch’s picture

I'm re-reading this, I think I may have mis-remembered the actual proposal and possibly told sun the mis-remembered version at DrupalCon, but not sure at this point.

Either way claar is right in #40, I'm agree with a 'fast track' issue for things that have a chance of being backported, alongside more sweeping changes for the same code that can't in a separate issue.

lotyrin’s picture

Re #37. If we get inter-issue dependency working, with a proper UI, then we'd basically get the same thing as what you've described there. Except that each of those versions/statuses has a separate comment history, which I at least think would be a good thing.

Otherwise you've got several discussions (although on a largely shared subject) going on, all overlapping each other.

Also, multiple versions with statuses only solves one portion of the problem. If a solution exists across two projects (some kind of inter-dependency), do we also add multiple projects? That doesn't make much sense to me.

catch’s picture

Version: 7.x-dev » 8.x-dev

Since this issue is more or less stalled until either Dries or webchick chimes in, I moved all current D7 issues that need to be applied to D8 to http://drupal.org/project/issues/search/drupal?text=&assigned=&submitted...

Moving this to 8.x too.

wojtha’s picture

Subscribing

catch’s picture

We also need to try to figure out what to do about updates - http://drupal.org/node/761648#comment-4303900 is a good example.

donSchoe’s picture

sub

arlinsandbulte’s picture

catch:
Maybe I am playing the devil's advocate here, but what is the purpose of assigning critical D7 bugs to D8 with a tag to be back-ported to D7?

*IF* an issue is *TRULY* critical, shouldn't the D7 fix get highest priority, since it is actively used in production environments?
For instance, if there was some sort of bug that resulted in a WSOD in a clean core D7 instal, wouldn't it be prudent to get the D7 version fixed ASAP and then forward-port to D8? (an extreme example, I know).

However, if the issue is NOT critical, fixing in D8 and then back-porting to D7 is totally reasonable, IMO.

ceardach’s picture

I think that 7.x should be regularly merged into 8.x. Bug fixes should go into the lowest known version where the bug occurred, and then they'll travel back up by merging the lower branches into the higher branches.

tstoeckler’s picture

Fix in the latest dev version, then backport.
That's how the Drupal development process works.
If you want to change that read up on some documentation, chat with some people in IRC and, if you still want to change it, open a new issue. Thanks!

ceardach’s picture

The backport process made a lot of sense when CVS was so critical to the workflow. Now that Git is involved, our previous assumptions are open to be re-evaluated. A sophisticated merging process is now on the table.

I'm just throwing the idea out there so it isn't missed.

David_Rothstein’s picture

I would agree that in extreme cases (e.g., where a bug is so critical that we would want to do a new point release of Drupal 7 immediately after it's committed) it makes sense to commit to Drupal 7 first. That's actually the procedure that's already in place for security issues. I wouldn't set the bar a whole lot lower than that, though.

I don't think this is a technical issue of CVS vs Git, but rather a community issue. I think we discussed this a bit above, but consider:

  1. If a patch gets committed to D7 and there's then a long delay forward-porting it to D8, so that the patch eventually goes stale, months more pass before it's rerolled, so on and so forth, and eventually it never makes it in before the D8 release...... that means we have introduced a regression. That's bad, and contradicts the goal of having tests (which prevent regressions).
  2. Someone needs to do the work to port the patch forward to D8. What is their incentive to work on boring bugfixes for a piece of software that no one is using in production yet? I worry about people showing up to help fix bugs that affect them personally in D7 production, and then leaving it to "someone else" to do the thankless work of making sure it gets into D8.
Lars Toomre’s picture

I can well understand the need to get "fix" patches applied against the most current version under development. I also am keenly aware of the need to fix the software that people are currently using (and probably will be for some time given the long time between major Drupal releases). Finally, I appreciate how "boring" it is to forward/back port a fix from one Drupal version to another. Thus, there is no clear/best solution to how "bug" fixes should be implemented (absent clear leadership from the community leaders).

Imagine what might happen if Dries/Webchick were to refuse to apply a "fix" patch unless both a D8 and D7 fix could simultaneously applied. I would suggest if the fix is not available to both the development and production code, it is not worthy of being included in either since it is not truly yet a complete "fix". Hence, people can work on resolving production problems or they can work to make sure that it the problem is eradicated from the development code. However, the "community" will not accept the fix unless it works against both the production and development code.

The same thought process also should apply to the issues that we have had with a lack of tests and/or documentation. Why are we applying patches that are recognized as being incomplete? If it is truly critical to get a fix in that is incomplete, why are we then applying other patches without first correcting what was incomplete?

catch’s picture

Imagine what might happen if Dries/Webchick were to refuse to apply a "fix" patch unless both a D8 and D7 fix could simultaneously applied. I would suggest if the fix is not available to both the development and production code, it is not worthy of being included in either since it is not truly yet a complete "fix". Hence, people can work on resolving production problems or they can work to make sure that it the problem is eradicated from the development code. However, the "community" will not accept the fix unless it works against both the production and development code.

Right, this would likely make it a lot quicker to get backports in - but it would require webchick to review RTBC Drupal 7 patches which haven't gone into Drupal 8 yet, or for Dries to commit patches to both branches himself simultaneously. Currently neither of these happen most of the time, nor did it happen with Drupal 6. Trying to get at least some kind of process in place for this stuff was the whole point of opening this thread up. I'm a bit disappointed that large patches are already being committed to (and breaking) Drupal 8 before there's been any committer feedback here, and as we approach ten critical issues against D7/8.

In a few months, Drupal 8 will start diverging significantly from Drupal 7, and Drupal 7 will be in a more stable maintenance mode (similar to Drupal 6 for the past two years or so), at that point this is slightly less of an issue - we'll hopefully have less critical/major issues in Drupal 7 than we currently have - but getting to that point requires something relatively smooth that people can understand now.

catch’s picture

Oh one more reason to have all critical issues against Drupal 8 (and part of why I moved them all there this week). In Dries' keynote, he said that he won't allow more than 15 critical issues in D8 at a time (in other words, if we hit 15 issues, we have a mini-code freeze before other patches are allowed in again). During the 'How do we do Drupal 8' core conversation, there was talking of even making this more strict - where we simply don't commit big patches to Drupal 8 that have known critical issues - those would be resolved in feature branches first, prior to commit (but this is going to have to be worked out over time) - so that the canonical repo for Drupal 8 has close to zero critical issues at all time - i.e. only ones that get found once they're already in).

Many, many times during the Drupal 7 cycle, people complained about x bug not getting into Drupal 6, because people were working on Drupal 7 - this is a silly argument most of the time, but has a grain of truth in it, in that we had 3-400 critical issues against Drupal 7 at one point, and there's no easy way to filter out the ones that also need to go into Drupal 6 (apart from the backport tag which isn't consistently applied). If there's only 15 critical issues maximum against Drupal 8, or a code freeze to get the number down, then posting D7 issues against Drupal 8 helps to ensure they get fixed in a somewhat timely manner - there's no way to impose a code freeze on D7 since it was officially frozen nearly two years ago.

rfay’s picture

I chatted with @webchick about this the other day, and she confirmed that the traditional policy is in force: Things have to go into the higher-level Drupal version (D8) before she'll commit them to a lower level (D7).

What does this mean right now? Drupal 7.0 came out quite raw, with hundreds of *major* issues. Some of those have been resolved and have gotten to RTBC. But now they have to be resolved in D8 first. This is discouraging for D7.

What does this mean in practical reality? It means that Dries alone (with no comaintainer appointed) becomes the gatekeeper for D8 AND D7. Every single bug that needs to be fixed in D7 has to go through Dries alone. Whoops. I think something's wrong here.

@Dries, if we're going to keep the traditional policy, please appoint a comaintainer for D8, or at least an arbitrator or temporary comaintainer or something so D7 does not suffer in such a horrendous way. Many of us are quite disoriented that hard-won fixes to bugs that D7 released with have to now be delayed by a visit to the D8 core process, and without a comaintainer at that. We respect your time and we respect how busy you are. Please help us with this either with a comaintainer or a policy innovation, per this conversation.

We'll live through this.

webchick’s picture

Yeah, I've been meaning to reply to this for about a month now but keep getting dragged down on other things. Time to do so, I guess.

For reasons laid out by several people here, we can't have a "fix in D7 (or D6) first, then forward port" because D8 has 0 "real world" users atm. There is therefore no incentive for patching D8 except among the hardcore, and regressions are all but guaranteed. This rule also is what has brought us some of our most prolific core contributors *cough* rfay *cough*. So I don't support changing it, personally.

I also don't support creating parallel issues for the reasons sun alludes to. It forks the discussion and makes things incredibly difficult to follow, not just for core maintainers but also for patch reviewers. A single issue with patches for D6, D7, and D8 is the best way to go. If the D8 solution is going to be widely divergent, then it might make sense to do a separate issue but there would need to be a really good reason. We lose too much back discussion on architectural decisions without centralization of discussion.

D7 is following about 24 hours behind D8 at this point. I think that this is acceptable. What I gather is not acceptable to people is the rate of bug fixing in D8 currently. But we've always had this initial period of Dries being the sole keyholder for core, as major version co-maintainers aren't typically appointed until 6 months in or so (for example, D7 HEAD opened in Feb 2008; I was appointed in August 2008 at DrupalCon Szeged). So this is nothing new; I'm not quite sure why alarm bells are going off this time. To some extent, it's actually helpful for development to be slow initially, because it gives contrib something stable to port against.

But in terms of rate of commits, there are a couple of things that can help with this. The first is that if something's a critical bug fix for D7, mark it critical for D8. In Dries's keynote, he stated that he wanted to keep the number of critical bugs down to 15 or less, and if it crept higher than that he'd stop committing features. (That does not mean mark non-critical fixes critical in D8; if they're non-critical, they can go in via normal issue queue triage.) Another thing is to ensure RTBC actually means RTBC. It's been awhile since I was able to spend a Sunday afternoon plowing through the queue, but each time I do, roughly half of the RTBC patches are missing tests, for example. This needlessly bloats the RTBC queue and takes time away from core maintainers.

As far as updates go, we've never really had a great solution to this problem. If we follow the rules "by the book," we commit the fix to D8, then the fix + update hook to D7, then finally a patch to adjust the "last updated" hook to D8. This always gets messy, but I'm not really sure of a better way.

So... not sure if there was anything earth-shatteringly helpful in there, but those are my current thoughts.

catch’s picture

Just quickly. At the beginning of the d6 cycle there were two permanent core committers, and Dries wasn't running two companies.
Steven was active committing right up until D6 code freeze.

So the first time we've had a single bottleneck like this was Drupal 7. TBH I don't think we had such a defined backport always policy right when D7 opened up, though might be mis-remembering. Even if we did, there were nowhere near as many patches pending against Drupal 6 when it released (iirc. there were only 100 active bugs in the queue as well) as there are now against D7.

Either way, the 2.5 hours in two weeks (which is what Dries recently posted for core patch review time, albeit with a caveat those were busy weeks) isn't enough time to get nearly 300 major + critical issues resolved let alone everything else that will be coming through. So I agree with rfay.

klonos’s picture

@rfay: I agree 99% with what you say Randy. It's all well-thought and they way you put it is structured perfectly. I have to object in a minor detail that actually is quite a big thing though:

...Things(???) have to go into the higher-level Drupal version (D8) before she'll commit them to a lower level (D7).

I understand that this is how it's done or perhaps what Angie and Dries might have chosen as a way to go or even what everyone has agreed is the best practice, but we're missing a couple of facts here:

1) D8 might be the higher *existing* version of drupal, but it is not the higher *supported* version - D7 is. We don't advertise D8, D7 and D6. We (now with D5 obsolete and all) advertise D7 and D6. Period.

What's more, D7 happens to be also the one with higher usage stats compared to D8 (that reports only ~20 sites using it). I obviously am not mentioning this because I want to compare or discuss usage stats. I simply want to point out that D8 is simply a developer-only thing. Which brings me to the second fact:

2) Besides, the word "things" in your phrase drives people to the conclusion that this practice is for *everything*. It should be "new features" go to the higher (dev) version- "fixes" go to the current supported version(s) first. We do want to solve issues for the current user base *first*. Don't we? That's where we get the issues reported from in the first place after all. That's where we'll get feedback from. From tens and hundreds of thousands of actual, live sites - not from a few dozen of dev setups.

Even if I got this whole thing wrong and things are not done the way I thought they were, I strongly believe we need to rethink this D8 -> D7 -> D6 path (at least when it comes to bug fixing).

klonos’s picture

...people actually replied before I hit "submit" to what I was writing. Angie for example mentions in her reply the 0 "real world" users of D8, but she somehow manages to twist this in favor of D8. So...

@webchick: did I actually understand correctly what you say Angie?? (keep in mind that English is not my mama-tongue). Also, you say:

D7 is following about 24 hours behind D8 at this point. I think that this is acceptable.

...well if that is true, it is good news. This usually didn't take much time in D7 -> D6 either. As a matter of fact I've seen people working, posting patches and finally committing to both branches at the same time. The issue though is not how fast "things" get backported from D8 to D7. It is more a matter of how much incentive there is for people working on D8 and how much feedback developers will be getting during working in D8. When things are done this way, I notice a lot of issues reopening because some things/aspects were not considered until before the change/fix hit the large(er) audience - a thing that should have been done in the first place. I know because I've experienced this with -for example- i18n issues more than enough times already.

catch’s picture

@klonos, the title of this issue is "figure out backport process from Drupal 8 to Drupal 7", it is not "change the backport process to a forward port process". Since this comes up every week or so, I've updated the documentation page here, part of which is in response to your post: http://drupal.org/node/767608

catch’s picture

Couple of practical things we can do to improve tracking:

#221510: Update contributors block for 8.x (and minor changes) - we need to update the links to point to 8.x, and I'd propose adding Drupal 7 critical (and maybe major) issues to it - either merged in with the existing links or separate new ones).

Any issue that's being posted against 8.x (or moved to it from 7.x), that is very likely to need to be committed to D7 too, should have the Needs backport to D7 tag applied to it. This is not a new tag, but it's very inconsistently applied at the moment. Following this tag will allow you focus efforts on issues that have some chance of being fixed in D7 (as opposed to entirely new stuff for D8), if that's what you want to do.

pillarsdotnet’s picture

@catch: Does the issue still need a "Needs backport" tag if it already supplies patches for both versions?

Digression: There really needs to be a docs page listing standard issue tags and their intended meanings.

catch’s picture

@pillarsdotnet: IMO it does. The backport is really the applying of the patch to the older version, even if we use that terminology for porting code between version. Patches for older versions can be lost among issue followups (especially long issues - if it's a 250 comment issue, and a 6.x patch was posted in comment 50, who remembers? The tag is present in the issue metadata rather than in an upload field so it's harder to lose). Also it's possible to make a list of all issues with that tag.

pillarsdotnet’s picture

@catch -- so noted; thanks.

plach’s picture

catch’s picture

There are now 107 RTBC issues against 8.x (dereine and I bumped all the RTBC Drupal 7 issues). I probably won't work on any core patches until that number is significantly lower (unless it's for work where I need to work on them anyway).

chx’s picture

I do not see much point on working on core yet indeed. We lost momentum, big time.

sun’s picture

Another thing is to ensure RTBC actually means RTBC. It's been awhile since I was able to spend a Sunday afternoon plowing through the queue, but each time I do, roughly half of the RTBC patches are missing tests, for example. This needlessly bloats the RTBC queue and takes time away from core maintainers.

The primary reason for this, at least from my perspective, is that Code Gardeners

  1. are either still burnt out from D7
  2. don't have a real target with D8
  3. or can't really handle two different major branches at once

I don't even have D8 checked out or installed anywhere on my filesystem. But why should I? As long as production sites are not safe to run on HEAD, I'm not going to have an incentive (except for a purely theoretical) to install or "really" work on D8.

rfay’s picture

[continuing on from twitter :-)]

@webchick, if asked, would you be willing to manage the migration of all these D7 tasks on their way through D8 and back again? There were probably about 200 issues we wished had been resolved before D7 released to make it less edgy. There are something like 250 issues now in CNR or RTBC for D8. I'm not sure how many of those are migrants from D7, but a lot of them. So I'd say we have to shepherd between 150 and 350 issues into D8 and back into D7. A huge job. You're certainly one person who would be capable. It seems unreasonable even to ask, but we certainly would have confidence that you would do it right.

And no, you don't have to answer. But I think this proposal from DamZ would be one with strong community support.

klonos’s picture

@catch: I hear you mate. Thanx for updating that documentation article too! [going through it right now...]

I like to thing I don't nag that much, but given the fact that 8.x issues currently need to go through Dries and Angie *only* -both really busy people- and the estimation that the possibility of having a couple (or more) extra core co-maintainers for D8 won't be realized till after say ...6 months + the number of serious bugs in D7 that got moved to D8 + Real Life® makes me sigh :/

...I'll simply shut up now and follow this issue just to know what is decided. Thanx everybody for your time and efforts.

robertom’s picture

subscribe

AdrianB’s picture

Subscribing.

fago’s picture

So this is nothing new; I'm not quite sure why alarm bells are going off this time. To some extent, it's actually helpful for development to be slow initially, because it gives contrib something stable to port against.

I don't think so. This slow development keeps off fixes for bugs, which contrib developers are facing again and again as long as the issues aren't fixed. Thus, I think d8 development slowing down d7 development is an overall bad thing.

wojtha’s picture

I agree with fago. I feel like we are in a vacuum now. Honestly I was disappointed when Dries created 8.x branch in Chicago. Additional half year working on D7 "major-bugs-free" version, fully concentrated, makes more sense for me. There are still plenty major issues in Drupal 7 which can't be ignored and many of them are now stucked in the process of D7/D8 migration.

I feel odd when I'm creating some patches e.g. for openid which is broken for significant part of providers in D7 (#1076366: OpenID discovery spec violation - fragments are removed from claimed id , #575810: OpenID discovery spec violation - follow redirects., #1120290: Provide transition for accounts with incompletely stored OpenIDs) and which needs to be fixed ASAP - and I need to fix it in D8 first and later backport as much as I can to D7 with the knowledge that openid may not be even included in next version of core (#556380: Remove OpenID from core).

I completely agree with that process in case of feature requests/feature issues, but if there is a issue fixing some serious bug which has also some effects on security and it lasts three, five months or even years #575810: OpenID discovery spec violation - follow redirects. until it is solved, mostly because of lack of feedback, I can't agree.

More capacity is needed if we want to continue in that process or we need to change the process.

rfay’s picture

IMO we have made a mistake that we can avoid next time around.

We had a huge set of very important changes to do for D7, many of them RTBC. Lots of raw-ness in D7. Instead of figuring out how to manage them and push them into D7, we scattered them to the winds by throwing them into D8 with very poor tracking (just a silly tag, sometimes missing). And now they're almost indistinguishable from D8 things. But the bottom line is that a large number of things that would have stabilized D7 have been delayed and in some cases lost, and that's not for the better.

Right now we have what we have, and I think webchick will probably organize this a bit in her new role.

But next time, let's do better. Let's make sure that we don't sacrifice the stability of the current release for the process of the future one. This might mean that we can't open the new branch until the "majors" are taken care of. Might mean a different approach to where things get committed. Might mean issue queue improvements. But let's not do this again.

Kars-T’s picture

subscribe

rfay’s picture

Status: Needs review » Fixed

I'm going to mark this fixed, as the core maintainers have clearly expressed their strategy for this time around. If you disagree, reopen it.

Let's do it better next time, and figure out how to not lose control of bugs that are ready to go into the recently-released version.

However, IMO conversations like this are best held over at Groups.drupal.org, and should start and finish there before becoming action items here.

wojtha’s picture

I'm going to mark this fixed, as the core maintainers have clearly expressed their strategy for this time around.

@rfay could you please paste here the followup links where is this strategy expressed?

rfay’s picture

#55 and #56.

Dries also tweeted it (can't find the tweet) and confirmed in private correspondence.

Neither webchick nor Dries has wavered on the traditional policy at any time.

wojtha’s picture

@rfay ok, thanks

David_Rothstein’s picture

Status: Fixed » Needs work

I think the current status of this issue is that @catch made a bunch of really nice additions to http://drupal.org/node/767608 to document the current policy, and now we're discussing possible small tweaks to it (not a major overhaul).

I'll try to find some time to take the current proposal in this issue and rephrase it as something like a bullet point that could then be added to the above documentation page.

Also, what we're primarily talking about here isn't really something that's specifically up to core maintainers to decide, right? ... It's more about how we as a community choose to generally manage the issue queue (e.g. use of the "needs backport to D7" tag and what kind of patches get written on an issue when it has that tag, etc).

sun’s picture

Also, what we're primarily talking about here isn't really something that's specifically up to core maintainers to decide, right?

Yes. Whatever "rules" we come up with, we're basically overruling Dries' freedom of core branch management decisions. Though not exactly overruling, but rather limiting. But anyway, learning from our mistakes as a community has precedence over that freedom.

David_Rothstein’s picture

Status: Needs work » Needs review

So the proposal here is to add text to http://drupal.org/node/767608 containing something like the following:

Bugs that occur in a stable branch of Drupal core (Drupal 6 or 7) but also occur in the development branch should be posted to the Drupal 8 issue queue with the needs backport to D6 and/or needs backport to D7 issue tags applied. When working on a Drupal 8 issue with one of these tags, the focus should be on finding a minimal fix that can be quickly committed to Drupal 8 and backported to the stable releases.

Sometimes, the ideal way to fix the bug in Drupal 8 may involve far-reaching changes or refactoring that would not be appropriate to backport. In those cases, it is best to create a separate Drupal 8 issue for the refactoring and cross-link it to the original issue. This allows the minimal fix to be committed to Drupal 8 and backported as quickly as possible, while work on the more far-reaching fix for Drupal 8 can continue at its own pace.

wojtha’s picture

@David_Rothstein #83 Agree.

For this issue management will be very handy to include "Project Issue Related" http://drupal.org/project/pi_related (see #24, #25, #26) to the Drupal.org, Using taxonomy tags and referencing related issues from the comments is weak. Freetagging is great for for general description but I will personally not even design some important process based on freetagging.

+1 for the related issue module is also that you need to scroll down by decades or even hundreds of browser's viewports to see what is related why and so. And for the same reason it will be also nice to have possibility to change the main issue text (with revisions of course). When we allow to change the title of the issue, why we don't allow to change the text? Sometimes the issue evolves to something completely different or went split to several issues, but you notice it after reading comment number 293. But this is may be a topic for a separate issue. I'm voting for "Make The Issue Queue Suck Less Initiative" :-)

I'm using Redmine for 3 years and I really love how the issue queue is handled there. I feel like we could learn and implement project management (PM) things from Redmine/Chilliproject as much as we can.

Related issues support instead of relying on the fuzzy taxonomy tags is a must have to implement David's proposed change to the Backport policy IMO.

David_Rothstein’s picture

Well, those tags are already heavily in use, so there's certainly no reason not to document them. If a better solution comes along later, we can change the documentation then...

I don't think I'm suggesting any change in policy that's specifically related to the use of those tags; the actual policy change I'm suggesting here is in the second paragraph (although even that we already follow to some extent; I just think it would be good to formalize it). In any case, it's independent of whatever method we happen to use to label backportable issues.

sun’s picture

I like @David_Rothstein's take in #83. However, I'd really love to enforce a solution to the pain outlined by @rfay in #75 on us.

If the rule for that solution happens to be "No new major branch before Drupal x.1.", then let it be it. Much better than the horrible pain of crap we are currently doing. This early post release cycle, it simply doesn't make sense at all to have to fix crucial bugs for two different branches.

fietserwin’s picture

The problem I see with patching the development version first is the fact that it creates an additional barrier for people to contribute patches.

I develop my sites in the current stable release (=D7 currently). If I encounter an error in my daily work, contrib or core, I debug it, I solve it (most of the times) and I will test it on the site I'm currently working on. Having done that, I am more than willing to contribute the patch back, albeit for the reason to lower the risk of regressions on my clients sites. So I go the extra mile to create a patch, post it in the issue queue, etc.

But the proposed process requires me to download a rather imaginary release (at least for me right now at at this moment), i.e. the development release (=D8 currently) to create patches for. This does create another barrier and I'm afraid that might be too much for many people to contribute anything at all.

Having said so, I do understand that the way I work might not be ideal for the community as a whole (e.g. not creating tests, not patching the development version = danger of regression, no documentation updates). However, it is still more than most people contribute (the 1-9-90 rule) and we risk loosing these (imperfect) contributions at all.

pillarsdotnet’s picture

Nothing prevents one person from contributing a 7.x patch and another person re-rolling it for 8.x. I have re-rolled lots of patches that I didn't contribute.

lotyrin’s picture

There are a lot of things involved in getting a patch committed

You have to:

  • Come up with a good solution
  • Come up with a solution that scales
  • Document your solution
  • Create a patch against the dev branch to prevent regressions
  • Write tests and make sure they pass
  • Make sure your change doesn't break any other tests

I agree that these are barriers to contribution, but sacrificing any one of them reduces the quality of the code. Conforming to the back-port policy is only one component (and not the most difficult in my opinion).

catch’s picture

Right, you can roll the patch against 7.x. If you're lucky it will apply to 8.x, if not you can mark it 'Needs work' and it may or may not get picked up by someone else for re-roll. There is no requirement that you personally do all the work to get a patch committed, just that the work eventually gets done by someone.

People who haven't read these discussions tend to do this anyway in the 7.x queue, then if they're very lucky, someone is looking through the 7.x queue, sees it's misplaced, and moves it to 8.x for them - if they normally don't follow the core development process, and don't read documentation on core patch processes before rolling the patch (both very likely for extremely long tail contributors), then this will continue to happen.

catch’s picture

Issue tags: +Needs committer feedback

I've added David's suggestion to from #83 to http://drupal.org/node/767608

Note I documented this as a 'current convention, still under discussion' - which is the best we can do here, this is about issue management and documentation as much or more than the actual policy - but getting those right is the only way we can avoid regressions and massive confusion (if those can be avoided at all).

Since people appear to be consistently using the tags at this point, and I think we have some agreement on the issue workflow (neither were the case in February when I opened this issue), I'm going to mark this RTBC again.

The exchange on #935066: theme_table() should not output the no_striping option as an html attribute suggests that this could still use review from Dries though, so I'm marking RTBC and adding the 'needs committer feedback' tag.

fietserwin’s picture

Don't misunderstand me, I do agree with most arguments given here, and this remains a difficult subject. But I am afraid that the current guidelines will delay patching errors in the current stable and supported version(s).

So we have a brave citizen who finds an error in D7.0 and writes a patch for it, only to find out, 6 months later when he upgrades his system to D7.4 that his patch is still not committed because it lacks a D8-dev variant patch. I agree, this is not a perfect citizen, because he did not write a D8 patch, but then, who is perfect?

Moreover, we do commit new features without proper documentation by tagging them as "needs documentation", but we don't commit (easy) fixes to D7 because it needs a D8 patch.???

catch’s picture

So we have a brave citizen who finds an error in D7.0 and writes a patch for it, only to find out, 6 months later when he upgrades his system to D7.4 that his patch is still not committed because it lacks a D8-dev variant patch. I agree, this is not a perfect citizen, because he did not write a D8 patch, but then, who is perfect?

While this isn't ideal, it's better than the same person upgrading to Drupal 8 in two years to find the bug was never fixed there, after it was committed to 7.x then forgotten about.

The one place where we circumvent this process, security issues, we had several known, critical, security issues that had been fixed in 6.x, but were left in 7.x for several months in some cases right through to beta - it took a lot of effort to go through and verify that all of these were eventually committed to 7.x, and that is a tiny subset of high priority issues.

mlncn’s picture

Status: Needs review » Reviewed & tested by the community

Made a few minor typo corrections to the Backport Policy page (including linking to the security team instead of the security module...) and marked it as no known problems.

Marking this issue RTBC as Catch noted in #91.

Dries has hinted that non-API-breaking backports could be allowed, which would give a lot more people a lot more incentive to try to get fixes and improvements into the current dev version of Drupal; i would love to see that confirmed with committer feedback and become stated as part of the official policy.

catch’s picture

Yes that was one of the original things I opened this issue for! But it got lost in the usual circular discussion about the overall process and I'd forgotten about that entirely at this point. It would indeed be good to get more of an idea of what is likely to be eligible for backport vs. not, it will be case by case to some extent, but it'd help a bit before things get to that point.

pillarsdotnet’s picture

Slashdot recently posted an Interesting article that I believe is relevant to this discussion.

rfay’s picture

@pillarsdotnet, Ironically, I think the problem with our chosen path here is *lack* of process. We've taken carefully vetted patches for D7, hundreds of them, and thrown them into D8 with no definitive (or certainly no expedited) process to get them back. Meanwhile, D7 is significantly immature and starving for many of those patches. If you're involved with some of these it's downright embarrassing that they're now languishing in the D8 queue instead of solving the key issues they were built for.

fietserwin’s picture

To me this (= the shocking number of patches mentioned by rfay in #97) clearly indicates that (as sun already said in #86) we should wait for 7.1 or 7.2 before officially opening D8. That will make the number of patches to port far and far less. I think that git is good enough at merging to not hinder new development in isolated branches before 7.2 is released

catch’s picture

@fietserwin, yes I specifically suggested to Dries to wait until there were no 7.x critical issues before opening the 8.x branch (0 critical issues against 7.x around DrupalCon probably would have meant 7.1 would have come out then). The whole point of opening this issue on Feb 4th was to get the overall process for 7.x fixes thrashed out /before/ the branch was opened.

However neither of these things happened, 8.x was opened anyway, and here we are months later still arguing about it.

markabur’s picture

Speaking for myself, I enjoyed being a part of the final 7.0 push last year, but with the switch to 8.x I have to say I've lost interest. With 7.0 there were specific goals -- get the number of criticals down to zero, reduce the number of majors toward zero, etc., but now that those bugs are part of 8.x, those goals are lost. It's ok for d8 to have criticals, but it's not ok for d7 to have them; the number of 7.x bugs should be going down these days, but the number of 8.x bugs is going to fluctuate. Which is totally fine for d8 but which makes d7 feel like it's not getting closer to the finish line. The process of refining and polishing and fixing (big bugs at first, then small bugs, then irritations toward the very end) is something that I find gratifying, but putting that kind of effort into d8 feels premature -- like waxing a car that is still due for some major bodywork.

klonos’s picture

...now that you've mention it Mark, I seem to recall that the whole tendency has always been to first fix bugs in current released/supported versions and then port to next versions. New features used to go to next (dev) branches and then backported. The latter though seems to be logical only for new features and not for bug fixes that should be fixed asap where possible IMO.

Lars Toomre’s picture

Currently the situation with D7 bug patches is a big mess. I first commented on this issue in #22. As I said then,

I appreciate the need/desire to keep the current development environment current for all known bugs. However, given the long lead time before anyone will ever use D8, I think that it is also incredibly important that bug patches also be prepared for D7.

My focus here is on bug-fixes, not code enhancement. Let me ask a rhetorical question: if a "bug-fix" patch (no API change or core enhancement) passes all D7 tests (including those that now have been enhanced to catch the bug), why should it not be applied to D7 right away? Such "bug-fix" patches then can be marked to be included in the D8 development code. (Core enhancements would remain applied to D8 first and then possibly back-ported to D7.)

The general technical community within Drupal benefits by a quicker way to fix bugs, much better adoption of test coverage (since the only way a bug patch gets in directly is with tests that capture the problem), and no need to right away deal with function and constant renames in the development code base.

[As an aside, as technical person hypothetically coming to D7 (from D6 or some other CMS), why should that person even need to care in writing a D7 "bug-fix" patch that in D8 the entity_level() function already has changed its profile or that the WATCHDOG_* already constants have been eliminated, etc? (Those are the first two entries in the D7->D8 changes guide with many more no doubt to come.)]

We want to encourage patch submissions from all. To fix identified "bugs", why from a process basis do we require possibly two patches against two different code bases to fix production code problems? The current process is incredibly frustrating, especially when valid bugs will occur with greater frequency than with mature production code. We need to rethink and somehow adjust our bug-fix process, especially during the early period after a major release.

Alan D.’s picture

I think that the core developers have spoken sadly: To keep the status quo, (Backport policy), and to find better ways in tracking the issues that need back-porting, which is the main reason this thread has not been closed yet.

I think that there needs to be much better handling of the release policy, (and branch policy), that will prevent the negative feedback being seen here.

  • PLANNING!

    What are the major expectations for the next release? What architectural changes are planned for the core API? What modules are to be refactored for the next release? I do not even know if these core questions have been answered yet for Drupal 8. No branching should occur before these have been assigned and officially documented somewhere. See http://groups.drupal.org/unofficial-drupal-roadmap.

    There has never been any official road-map for Drupal AFAIK, and this is the phase that is the most important in any development cycle. IMHO, this is more important than unit test, coding conventions, or any other of the important coding policies that have been implemented by the community at large.

    Trigger: A finalized list of the requirements. The branch would be created and an issue for each item in the road map should be created and assigned with critical priority (for major core changes) or major priority.

  • Early developmental stage

    Early stages in the Drupal 8.x branch - no public releases. This is where there are major refactoring to the core system. Drupal 7.x examples were the addition of the field system and the refactoring of the database system. API changes are freely applied.

    Trigger: The critical issues from the road map are closed or nearing completion.

  • Developmental stage

    Consolation of the early major changes and less dramatic UI changes, performance enhancements, bug fixes. Drupal 7.x examples were the addition of the overlay module and the refactoring of the taxonomic system. API changes are allowed. Release of a public developmental snapshot. Recommendations to module developers that depend on the core API's to port their work to the latest version.

    Trigger: The remaining issues from the road map are closed or nearing completion.

  • Late developmental stage

    Further consolation of the earlier changes. Focus is on UI testing and bug fixing. Freeze to major API changes and string freeze. Release alpha version. Pick up on any of the existing issues from the queue as examples. Minimal API changes only. Recommendations for the remaining module developers to port their work to the latest version.

    Trigger: Completion of the road map issues and other critical issues. This could include the postponement of any road map issue to move it to the next major branch, so that this feature does not delay the existing development branch.

  • Early release cycle

    Focus is purely on bug fixes. Release beta version. Complete freeze to any API changes. Main focus is to drop the major issue queue to near zero, reduce the normal issue queue and to give non-core developers a window to finalize porting contributed modules.

    Trigger: Stable critical / major issue queue.

  • Release

The trick is to minimize the number of items in the first stage to ensure that we have a fast developmental cycle that would allow more time nearing the end of the cycle for testing and bug fixes. So where are the minutes from the Core Developer Summit from April?

Sorry for the additional unrelated noise....

catch’s picture

Title: Figure out backport process from Drupal 8 to Drupal 7 » Figure out backport workflow from Drupal 8 to Drupal 7
Status: Reviewed & tested by the community » Needs review

I'm changing the title to reflect the original purpose of this issue. I wanted this issue done and closed before 8.x opened up, it is now four months since opening it, 8.x has been open for two of those months, still no feedback from Dries, and it is getting progressively more derailed as time goes on. I'm going to contact him personally (for at least the third time since February) to ask him to read this issue and the linked threads and follow up. Until there is something in place that has been agreed on by Dries and is being implemented, I am basically done with the core queue, since it's impossible to get any work done right now (and the one day sprint on Thursday barely made a dent in it - this needs a long term strategy, not a few hours every few months).

So here's an attempt at an issue summary, I am also going to propose specific workflow changes that I think may satisfy the 'D7 first' crowd.

First of all, the issues that prompted me to open this issue in the first place, #895014: All fields of a node type are lost on module disable and #1017672: D6 to D7 update process permanently deletes comment bodies and other data, and throws fatal SQL errors, remain a pristine example of everything that is wrong with the current workflow. If you are reading this and you haven't read those issues end to end, please do so now. Specifically note the dates when the two issues were opened up (hint, the second one is about three days after Drupal 7 was released). Those issues show several breakdowns in the core development process, specifically around getting high priority bugs fixed in a timely way, and tracking bugs that affect multiple branches. If you think that might be an unfortunate one-off, then read #999004: user_save() relies on $edit instead of $account where the same thing happened.

Specifically:

* We completely failed to accurately track the progress of issues as they get backported from the development version to previous versions, or to ensure that process happens in a timely manner.

* There is no mechanism in place for issues in the latest stable version to block changes in the development version - in this case a Drupal 6 patch was supposed to block the 7.0 release itself, and still isn't fixed 9 months after it is opened. Not only this but the Drupal 7 workaround I came up is still not committed to Drupal 7, due to 8.x patches being committed that conflicted with it (and broke the test bot, and contained regressions).

* The reason we have so many major and critical issues against 7 and 8, was because we let this get completely out of hand while 7.x was in development (over 400 critical issues at one point). Dries stated at DrupalCon that he would not allow Drupal 8 to have more than 15 critical issues at any one time (over that focus would shift getting the critical queue back under 15), however this is neither sufficient, nor actually being enforced at all.

* The fact that 8.x was opened with all of this outstanding was a very serious error, one which I tried to prevent.

* It is a ridiculous situation that webchick as the 7.x branch maintainer is unable to commit patches to Drupal 7, the only patches RTBC against Drupal 7 are ones that Dries committed locally but forgot to push as far as I can tell. Right now when the code bases have not diverged that much, and 7.x is in such a state, is precisely when webchick is most needed up front in the queue.

* Also, more than once, Dries has committed an 8.x patch which has broken the test bot (for whatever reason), then disappeared from the queue for days (three days here: #1097100: Remove all 7xxx update functions and tests (D6 to D7 upgrade path) and that's not the only time it's happened), leaving absolutely no-one who's got access to get things working.

NB. An '8.x' only patch is one that changes APIs, moves code around, refactors code, and will definitely not be backported to Drupal 7. A 7.x patch is one against 8.x but tagged with 'Needs backport to D7', or an RTBC issue against 7.x itself. So when I say "7.x" patch, that might well be posted against the 8.x queue, please bear that in mind when reading.

So, concrete proposal:

1. If there is any RTBC 7.x issue, that issue should be committed before any 8.x only issue. This will stop API changes, 'clean up' etc. breaking pending/finished Drupal 7 bugfixes - which is the absolute most frustrating thing about trying to get things fixed in 7.x right now.

It is currently impossible to produce a single link of all issues that affect Drupal 7, that are also RTBC, however these two links will do:

http://drupal.org/project/issues/search/drupal?status%5B%5D=14&version%5...

http://drupal.org/project/issues/search/drupal?status%5B%5D=14&version%5...

As you can see, this list stands at around 40 issues right now. There may be additional RTBC issues that aren't tagged yet too.

2. If you add 8.x critical issues and 7.x critical issues together and get more than 15 (currently 21), then no 8.x only issues should be committed until the number goes under 15. This gives people working on those critical issues a chance to get the patches ready and committed without having the code base change under their feet. This is more or less what Dries already agreed to at DrupalCon, just incorporating D7 issues too (right now, there by definition cannot be critical issues in D7 that aren't also in Drupal 8, even upgrade path bugs affect D6 sites that will eventually need to upgrade to D8 when it comes out, so it's not much of a change at all really).

3. If there are more than 100 major bugs (specifically not including tasks) against 8.x and 7.x combined, no API changes, cleanup, refactoring (basically 8.x-only patches again) should be committed until that queue is back under control. (there are currently over 300 of these, although probably some could legitimately be downgraded to 'normal'). If they don't fit on two pages, then it's impossible to keep track of that list.

4. Exceptions to this would be security fixes (which are forward ported to 8.x), test bot breakage (has to be fixed before other patches can be committed), and possibly serious regressions introduced into the 8.x branch (although ideally we should fix serious regressions by rolling back the patch and put it back in when it's ready).

5. To enable this to be cleared in a timely fashion, it needs to be possible for the 7.x branch maintainer to commit 7.x patches, so webchick should have commit access to 8.x (or alternatively, just delete/mothball the 8.x branch and start it up again when the above conditions are met).

6. The order in which patches are committed should as closely as possible match the issue priorities - critical -> major -> normal - > minor, and bugs before tasks.

What this will change compared to now:

- if there are any RTBC issues in the 7.x queues, then 8.x doesn't get any new API changes. Right this minute 8.x has only a few minor API changes compared to 7.x (this is still too many, but they're already in there, we could fix that by deleting/mothballing the branch). So if you disagree with the backport policy altogether, all you need to do to stop 8.x steaming ahead with API changes and holding up 7.x patches, is review 7.x patches and get them to RTBC.

- For people who really want to get going with 8.x patches (and ironically I'm one of these people too, although I'm not exactly motivated to do any work on it right now), this is not actually going to stall 8.x, because all bug fixes against 7.x are bugs in 8.x too. There is also git now, which lets you have local and sandbox feature branches etc, so if you're working on a large change, the tools are a lot better. If 8.x gets completely stalled because the queue is full of critical and major bugs, then the way to get it going again is helping out with critical and major bugs. Bug fixes that affect 8.x (which all 7.x issues currently do) have to get committed at some point, keeping them under control during the process will allow us to get 8.x out without an 18 month code freeze at the end again. By the time 7.x is somewhat stable, there will be less activity in that queue so more chance for major 8.x-only refactoring, new features etc. to move ahead at a faster pace.

- For those wondering why none of this applies to Drupal 6 too. The difference is the existence of major issues (a new introduction during D7) and that it's been out so long. The reason we could release Drupal 7 at all was that any 'critical' issue that wasn't an out and out release blocker was downgraded to major (not all major issues are like this, but several are). D7 has a lot of these almost-criticals but the process hasn't changed at all to accommodate them (and there's too many right now to be able to get an overview of what's in there). Also while Drupal 6 will be supported until at least 8.x is out, the code base is very different from Drupal 7, and the criteria for backports being committed to Drupal 6 is much stricter (since it has been a stable release for years now and more than 400,000 sites are already running it). Having said all this, if patches get committed to D7 faster, that will help them get back to D6 too.

If people have specific, practical alternatives or tweaks to this, I'd be really happy to hear them, however I am mainly concerned that we have something in place that's better than the current mess - and if there are problems with that then we can discuss it too.

webchick’s picture

Wow. Wow wow wow. Excellent summary, catch.

I'm in agreement with basically all of that. Commit access to 8.x is obviously Dries's call. At the moment, we're dealing with this with a tag which basically means "I spent 20 minutes looking into this, here's the gist, you should be able to more or less commit it blindly the next time you get a couple of hours to spend on the core queue." We'll see how that goes.

Just so everyone's on the same page regarding expectations: This week I'm on the road (as is Dries for the first couple of days), so this will initially kick off a bit slowly. (Not because we don't care, but because of the bounds of physics. Stupid physics.) But my goal is to spend at least an hour per evening this week caring for the issues in the queues you specifically called out there. Next week+, though, then this will become my primary focus, so we can get D7 into a good place, and then get D8 rolling full-speed ahead.

Lars Toomre’s picture

A very big +1 to catch in #104. Well put and I very much agree with proposal points 1) and 5).

Anonymous’s picture

catch++

keithm’s picture

subscribe

Shadlington’s picture

Subbing

Dries’s picture

Sorry I didn't get to this issue sooner.

Early on in the Drupal 8 process, I decided to focus my time on changing our development model and launching the core initiatives (configuration management, design, web services, multi-lingual). Frankly, this has taken more time than anticipated. I plan to launch the last initiative this week; that will be the HTML5 initiative. To give you a sense; I probably spent 10 hours talking to different people to launch the HTML5 initiative. Once I launched the HTML5 initiative, I'm probably done launching initiatives -- at least for a while. I hope to launch more of initiative once we're on top of the issue backlog.

In addition to launching the initiatives itself, I've spent quite some time working with the Initiative Leads and various Drupal project managers to figure out the how our "gates" should work, what reporting structure to put in place, how to communicate with the world, etc. We're slowly getting a handle on that as well, but have more work to do.

Long story short, because all the initiative work, I had to scale down my issue queue involvement -- that obviously caused some pain for people involved with the issue queue work. On hindsight, I should have better communicated or balanced my priorities.

Now, I agree that I've not always been focused on reviewing the right patches. I "corrected" that behavior after being asked to. I've been focused on critical and major issues last week and this weekend. I hope you appreciate some of the progress made as I spent quite some time catching up. I obviously have more work to do.

Scaling yourself is not easy -- and actually takes time. To help me scale, I hired webchick; that obviously didn't happen overnight. I hope to hire someone else in the Office of the CTO at Acquia to help me scale further so I can be more focused on Drupal 8 -- this might take another 2 months to materialize but I promise you'll continue to see improvements over time.

Either way, webchick will be primarily focused on Drupal 7, and help me with Drupal 8 project management, as well as finding efficiencies in our tools and processes. I just spent 2-3 days planning with her -- I expect we start to see real progress once she gets back from the UX sprint. Webchick and I talked about many of the current paint points. For example, we have agreed to try and release Drupal 7.1 in 2 weeks, and to do regular, predictable releases from then on. I'm also open to give her some level of commit access if the community is comfortable with that.

That being said, I'm pretty much done launching Drupal 8 initiatives so that will free me up some more. I hope to clear more of the backlog and to help resolve some of the pending discussions soon. I'll continue to make changes to the way we work/collaborate in the best interest of the project.

Once again, sorry for causing frustration -- it obviously is never my intention.

I know this comment doesn't specifically reply to your proposal, but I'll work on a more specific reply this week. For now, I wanted to give people background on what I've been up to. I'd also like to sit down with webchick and catch once webchick is back from the UX sprint to brainstorm about the proposal on IRC or Skype. At first glance, it seems to have a lot of good suggestions. Thanks for taking the time to write up a proposal, and stay tuned for more feedback on it, or concrete actions to start implementing some of these suggestions. I'll be thinking about it this week.

sun’s picture

Some conclusions to draw:

  1. Still seeing core committer follow-ups on the current critical/major RTBC 7.x issues along the lines of "This looks acceptable for D7, but we might solve this more cleanly for D8." (» needs work) only means that the understanding of the problem space hasn't found its way through everyone's synapses. In fact, I'll start yelling the next time this happens.
  2. I'm not sure who came up with the idea and whether it was planned in any way, but the mere fact that we have at least one D7 maintainer "approved" patch for D8 is a crystal clear sign of a development process running ad absurdum. Good intentions and all, but it's entirely impossible to make any sense of out of that.
  3. In fact, it may be time to revisit our change management strategy. The ultimate cause for all of this is our "Fix HEAD first" policy. Combined with D8's new <16 criticals policy, there'd actually be an easy way out. One, that major contributors apparently did not attempt to discuss for real: Fix HEAD last. Simply bump all committed fixes up as critical. Regardless of whether they were critical before.
catch’s picture

@sun:

#1 - agreed, I have found it quite unsettling seeing comments like this as well, from Dries' response so far it looks like it's not that he necessarily disagrees with what's been posted here, but hasn't actually read everything yet, which might explain some of the disconnect.

#2 - yes it is equivalent to having the lead striker in a soccer team leaving footballs in the penalty area with post-it notes on them, so that the manager can walk over from pitch side to shoot them later in the game. Hopefully it's a stop-gap that won't last longer than a week or two.

#3 - that sounds like a variation on my "don't commit any Drupal 8-only patch while there is a patch that affects Drupal 7 ready to go in", or your "delete the 8.x branch and start over" - they are going to amount to the same thing for about 2-3 months at least.

@Dries:

I look forward to seeing your response to the proposal itself and the other issues raised here, happy to discuss in real time next week if that helps.

claar’s picture

+1 to all of #104 -- thanks, catch.

In particular, the 7.x maintainer having commit access to 8.x (for 7.x-targeted patches at the very least) is just what we need to get 7.x development moving again, and makes perfect sense until 8.x has matured more and/or an 8.x maintainer is named.

Dries’s picture

I'll talk with @webchick about this once she gets back from the UX sprint. Probably on Monday.

tamsoftware’s picture

sub

catch’s picture

Here's another example

#1007830: Nested transactions throw exceptions when they got out of scope was RTBC'ed 19th April.

webchick added the 'webchick approved' tag 4am May 25th, which basically means "I'd commit this if I could", this was part of the last pass over the RTBC queue prior to starting the tagging/release process - patches were still being committed 3-4 hours later. Drupal 7.2 was released 13 hours later without the patch applied.

There are some other RTBC patches in the queue that didn't make it into 7.2, but that's fine - it's not always possible to get to everything and you have to draw a line, but this particular patch didn't make it only because of the current workflow.

fietserwin’s picture

Some tweaks to catch's excellent suggestions, but those will need changes to the issue module.

* instead of raising regressions to critical, add a category "regression" and handle these as critical (>15 regressions is halting any other activity). This because it is my experience that priorities are changed quite often by participants and so it will be easy to loose regressions.

* instead of the field version, we might need 2 (multi-select) fields "version(s) affected" (initially filled in with the "version reported") and "version(s) solved" as it looks like the same issue will be used to patch multiple versions. This will also give a better insight if an error applies to the current version people are using right now and in what version they may expect the fix. Before reporting an issue I try to search through the queue. to reduce the number of results I tend to filter by version. Thus if an error is already solved I might miss it because the version field has been changed to the "version solved".
(Note: this will probably affect automated testing as the testbot may not know anymore what version to test)

catch’s picture

@fietserwin: regressions could just be marked critical, people do change status/priority around a lot, but that would happen with bugs marked 'regression' too.

Posting issues against multiple versions there's an issue at #66484: Allow issues to be filed against multiple versions/branches..

Dries’s picture

A first reply at http://buytaert.net/fast-tracking-drupal-7-bug-fixes.

Summary: I granted @webchick commit rights for bug fixes in Drupal 8, in order to help ease maintenance of Drupal 7 and speed along the stabilization process.

I'll obviously continue to review and commit bugfixes myself as well. I'll also continue my search for a Drupal 8 co-maintainer.

dww’s picture

This is excellent news! Thanks to webchick for agreeing to take on the new responsibility, and to Dries for an important shift in policy at a critical time for the momentum of the project. Yay!

catch’s picture

Dries, commit rights for webchick is great news (possibly not for Angie but definitely for the rest of us!).

I'm still concerned about our ability to triage/track major/critical issues between releases effectively, but that's not only a core maintainer issue (although feedback on that would obviously be really welcome).

webchick’s picture

Well. :) There goes all my free time. :)

Catch, what further feedback is needed in order to mark this issue resolved? The bit about not committing feature patches to D8 while the cumulative total of critical bugs between D7 && D8 > 15? A signed-off on process for "needs backport to XX" tags? Or?

catch’s picture

So this is just what I'd like, while it looks like some people agree with me, I don't know if everyone does and I obviously can't unilaterally impose it. However even if the solutions aren't agreed, I'd like to see the issues they try to solve addressed (up to and including people saying "that's really not a problem").

1. Total critical bugs against D7 + D8 not allowed to go over 15 like you say.

2. RTBC 7.x patches first: the past week there was a focus on clearing the 7.x RTBC queue before any 8.x only patches, previously there hadn't been. I would like to see the 7.x (+ backport) RTBC queue completely empty before any 8.x-only patches are committed on a permanent basis - since the worst thing about trying to fix urgent (or even non-urgent) bugs in 7.x, is when a 'finished' patch is invalidated by a sweeping API change or clean-up patch - that by definition has a longer window when it could go in and only needs to be rolled against one branch. I say this as someone who has posted 1,000+ line 8.x patches in the past week that would get broken by one line bug fixes to 7.x - that's completely fine with me since it is more fun working on 1,000 line 8.x patches than it is 7.x bugfixes, so I mind less about re-rolling them. Without this it becomes increasingly hard to justify the 'backport first' policy to people who flat out disagree with it.

3. Sanity limit for major bugs: As well as criticals, I'd like to add some kind of sanity limit for major bugs too (not major tasks) - suggested 100 above, but anything would be better than infinite. We are at over 300 at the moment, it goes up every day, and it's impossible at that number to differentiate the 'almost critical' from "wtf this is stupid". We risk getting to > 500 major bugs, having no idea what's in there, and discovering down the line that some of them are release blockers and should have been marked critical all along, leading to another very long, cold, freeze. Again - when the number goes over 100 (or whatever), feature/cleanup patches stop going in until it's back under control. 100 might not be realistic, but again something is better than nothing and it happens to fit on two pages. Some of the major bugs at the moment are really tasks, some might not be major, some might be critical - but with over 300 it is anybody's guess what's in there.

4. Commit necessary workarounds to D8 if there's no other way to fix it in 7 then followup with a better solution later. Several 7.x issues with less than perfect solutions due to inability to change APIs have been marked work by Dries with "let's fix this properly in D8 first" as recently as last week. There might be cases where a better Drupal 7 fix could emerge from this, but sometimes we have to commit workarounds to stable releases if it's the only way to fix something. If we want to both fix bugs in the stable release promptly and keep branches in sync, this is going to mean committing ugly workarounds to 8.x too then following up with a second patch to clean it up properly. Not doing this for some ugly security fixes in 7.x led to some, very, very old critical security issues that had no 7.x patch for months and ended up in the final release blocker queue of 30-odd (these were forward ports but the same principle applies - we should not block working bug fixes to 7.x on vapourware if the vapourware can never be committed to 7.x anyway).

So we don't necessarily need all of these in place, or there might be a #5, #6 and #7 I forgot, but they are not dealt with just by giving you commit access (even if it might make some easier in practice).

dww’s picture

catch++ (again). I'm happy to see webchick with permission to commit D7 fixes to D8 and D7 (and wanted to quickly express that last night when I saw this), but yes, that's only one aspect of the bigger problems here. Once again, catch has done a great job of summarizing the problems and his proposed solutions. I, too, can't just unilaterally impose them, but I strongly support them.

Something that's a bit sad is that for one of his points he's basically arguing "we should do what Dries said we were going to do when he announced the D8 initiative proposal and the concept of the no-more-than-15-criticals gate", and we still haven't gotten an explicit agreement to that. ;) Yes, we understand Dries wants to see D8 get off to a solid start and to build momentum, so he's excited to be committing D8-only patches so it feels like "D8 is going somewhere." But you can't have your cake and eat it, too. Priorities are priorities, and keeping D8 releasable means stabilizing D7 ASAP. Otherwise, we're just shooting ourselves in our own feet by repeating the same mistakes of the D7 dev-cycle-of-doom. ;)

Anyway, thanks to everyone who's grappling with this issue -- these are very hard problems, and there are a lot of different things everyone is trying to balance and compromise on. I'm sure we'll find a solution in the end.

catch’s picture

Oh just in general, back on February 4th when I posted this, in the back of my mind what I wanted us to do was a post-mortem on what went wrong with the D7 cycle (primarily the permanently broken upgrade path and the 18 month freeze). We hadn't done that on Feb 4th, but it seemed like a good time to start discussing it. Three months later I still don't think this has really happened. If we don't seriously confront the issues that led to D7 taking about a year longer to get out the door than anyone expected, then we'll repeat the same for Drupal 8.

sun’s picture

Status: Needs review » Reviewed & tested by the community

I'm feeling bold today and so I RTBC the full and entire proposal in #123, including the stated number proposals.

Thanks, @catch, for nailing it.

plach’s picture

+ 1 from me, awesome work, catch!

wojtha’s picture

++catch, brilliant, again

Crell’s picture

Catch's proposal sounds reasonable to me, too. Especially for issues where "we know this part of D8 is going to change later anyway, so it doesn't have to be a great fix, just a working fix". There's a fair number of those at this point.

Anonymous’s picture

Sub. I'm a bystander, but this discussion seems critical to the future of Drupal, and I've been following it all along without officially subscribing.

As a refugee from Joomla, where the 1.5 core development process was totally opaque, but obviously severely broken, I'm glad to see these difficult issues being hashed out in public. The months of no motion on this issue, and on Drupal 7.1, were worrisome; it's a relief to see the logjam broken.

JohnAlbin’s picture

If we want to both fix bugs in the stable [7.x] release promptly and keep branches in sync, this is going to mean committing ugly workarounds to 8.x too then following up with a second patch to clean it up properly.

I completely agree. To make sure we track those issues better, shall we use the tag “needs forward port to D8” or something else? (There's already a “needs forward port to D7” tag.)

I'm also in total agreement with all of catch's other points in #123. :-)

pillarsdotnet’s picture

webchick’s picture

Just a report form the field on my first chance to use my newfound powers, as it relates to this proposal.

I spent about 2 hours today working my way through this list: http://drupal.org/project/issues/search/drupal?order=priority&sort=desc&...

This was enough time to give thorough review to about 5 issues in the "majors" list. I was only able to commit one fix, accidentally from the "needs review" queue instead (not sure how I managed to do that), and that made me super discouraged so I decided to commit a minor clean-up one as well, to lift my spirits. :P

So my initial impression is focusing only on that list and working from top to bottom is going to result in big morale problems, both on the patch contributor side, and on the committer side. While it was good to make progress on some of the majors, it's also good to show that smaller bugs are getting taken care of in a fairly timely manner. It still makes sense to prioritize from top to bottom, but not allowing minor/normal fixes until all major/criticals are taken a look at is not really realistic. Sometimes I only have 30 mins, which might not be enough for a single major but might be enough for 4-5 normals/minors.

But I think as a general guideline, "Look at criticals/majors first" makes a lot of sense. But I'm not sure how that's much different from our general guidelines now.

webchick’s picture

Sorry, I guess that was more of an aside observation than it was specifically related to catch's proposal in #123 (I wrote that in a hurry).

chx’s picture

Yeah #123 is more than just major/critical first, let me sum it up

  1. Total critical bugs against D7 + D8 not allowed to go over 15.
  2. RTBC 7.x patches first
  3. Sanity limit for major bugs -- if major bugs go above this then halt features until these get down.
  4. Commit necessary workarounds to D8 if there's no other way to fix it in 7 then followup with a better solution later.
pwolanin’s picture

Would it be reasonable (in terms of timing and likely participants) to have some post-mortem + forward looking discussion of the 7.x and 8.x dev cycles as part of the core conversations track in London?

Anonymous’s picture

That sounds like a very good idea to me
.

bryancasler’s picture

subscribe

Crell’s picture

Peter's idea sounds like exactly what the CC track is for, and I nominate Peter to moderate it. :-) (Or baring him, Angie.)

Dries’s picture

It sounds like the fundamental thing at issue here is a balance between making sure the latest stable release is taken care of, while at the same time allowing for innovation to happen in the new development release. Drupal 7 was released with 0 criticals, but still with hundreds of major bugs, and when Drupal 8 improvements are committed that break backwards compatibility, it sharply increases the cost of fixing bugs in the stable release that people are using today. For this reason, a number of people who would normally be driving Drupal 8 development are holding back, because they don’t want to see their important bug fix patches broken. But at the same time, we can’t push off major Drupal 8 improvements until Drupal 7 is perfect (and it will never be perfect), or we risk Drupal falling behind the innovation curve, as well as alienating contributors who want to drive Drupal forward.

Reviewing catch’s proposal, here are my thoughts:

  • It makes a lot of sense to have the 15 critical bug cap before new features can be committed span both the 7.x and 8.x releases. This will help to ensure that things aren’t getting out of hand, and that the really bad bug fixes aren’t held back by less urgent patches. I can formally announce that if there is general agreement.
  • 100 major bugs across both 7.x and 8.x blocking new features in D8 is also a good target goal. However, I don’t think we can reasonably set that as a hard cap right now, with the number up around 300, because it will just send D8 core developers into contrib and elsewhere until “someone” cleans up the mess.

    What if we instead approached that goal gradually, and did something like take a bi-weekly count of the majors and set that as the new cap, with the goal to push it down to 100 over the coming months? For example, we could set the cap currently at 300. If the count gets to 302, no new features. But if in two weeks we have knocked this number down to 275, that becomes the new cap. The number of majors can never go up until we arrive at a number less than 100. That will enforce the number to go down, but it doesn't halt innovation in Drupal 8.

  • In terms of prioritizing Drupal 7 fixes over Drupal 8 features, I believe this is to a large extent taken care of by granting @webchick commit access to Drupal 8. She has been steadily chipping away at the Drupal 7/”Needs backport to D7” RTBC queue, and is also in touch with both the core developers and myself to help make sure that important fixes get prioritized and dealt with quickly. Let's see how that evolves over the next couple of weeks.
  • In terms of committing workarounds to Drupal 8 for the benefit of Drupal 7, I think we need to treat this as an exception, rather than a rule. I’m reasonably okay with it if it’s a bad enough bug that’s actively harming contributed modules or production sites, but this situation definitely needs to be handled on a case-by-case basis -- at least for now. Maybe we could come up with a tag like “revisit fix in D8” or "don't wait forD8 fix" to identify these issues, and webchick and myself can review these in our talks and make a call.

So that’s my current thinking. These are all just suggestions, I’m also open to other ideas. Let’s discuss.

Anonymous’s picture

100 major bugs across both 7.x and 8.x blocking new features in D8 is also a good target goal. However, I don’t think we can reasonably set that as a hard cap right now, with the number up around 300, because it will just send D8 core developers into contrib and elsewhere until “someone” cleans up the mess.
What if we instead approached that goal gradually, and did something like take a bi-weekly count of the majors and set that as the new cap, with the goal to push it down to 100 over the coming months? For example, we could set the cap currently at 300. If the count gets to 302, no new features. But if in two weeks we have knocked this number down to 275, that becomes the new cap. The number of majors can never go up until we arrive at a number less than 100. That will enforce the number to go down, but it doesn't halt innovation in Drupal 8.

how about we add to this a time-boxed drive on reducing the major count? it would be scary to commit to a target with no end in sight, buti think we need to do more than just commit to "the target will reduce over time".

so how about we commit to a 6(?) week push, during which its all hands on deck for the stabilisation push, then we move on to "it will not go up but only down" commitment?

dww’s picture

Based on Dries's comment (thanks for that!), I don't think he's going to support what amounts to a 6? week D8 code freeze just to get stability under control. He's (rightfully) trying to balance stability with D8 progress. At the same time, I agree we can't just assume the major count will naturally decrease unless that's somehow a priority, and 300 is a pretty high number given the criteria for major.

How about we say something like every 2 weeks, the major count limit drops another 10 issues until we hit 100. So, if we're at 300 now, to drop to 100 would take 40 weeks. That means for most of the next year, we have to at least conquer 1 major issue every "work day" (not including new majors). That's actually a pretty ambitious goal, and I'm not sure if it's realistic. Maybe we have to say 10 issues every month. But, regardless of the specific number, the basic approach of a limit that we gradually push downward on a time-boxed way seems like the way to go. That doesn't mandate an "all hands on D7 deck" code freeze for D8 (which Dries and others will veto with good reason), but it doesn't let D8 get too far ahead without at least *some* focus on the majors backlog.

David_Rothstein’s picture

In terms of committing workarounds to Drupal 8 for the benefit of Drupal 7, I think we need to treat this as an exception, rather than a rule. I’m reasonably okay with it if it’s a bad enough bug that’s actively harming contributed modules or production sites, but this situation definitely needs to be handled on a case-by-case basis -- at least for now. Maybe we could come up with a tag like “revisit fix in D8” or "don't wait forD8 fix" to identify these issues, and webchick and myself can review these in our talks and make a call.

I think in practice, pretty much any time this actually comes up, it will be a bug that is actively harming contributed modules or production sites. If it didn't, people wouldn't be eager to have it backported quickly in the first place :) Certainly the times it has come up so far (at least ones I can remember) were cases like that.

We could add a new tag if it helps, but more important, I think, is to make sure the separate "fix this better in D8" issue is actually created (and cross-linked). Because the idea is that once we have two issues, different people can work on them simultaneously. Neither the "mediocre but backportable to D7" nor the "amazing D8" solution has to get held up on the account of the other.

This is all nicely written up already at http://drupal.org/node/767608 (@catch and I wrote it based on consensus earlier in this issue), so see if the policy as written there makes sense.

catch’s picture

So major bugs, both Dries' dww's post actually shows part of the problem, the same confusion was in irc.

The count isn't actually 300 - the quick counts on the contributors block show bugs + tasks, we're only discussing bugs here. Having seen this issue, quicksketch pre-emptively started triaging the major queue, chx and I have also joined in a bit this week. There were probably 300 issues classified as major bugs a week or two a go, but last count it is 222 - that's at least 50, if not 100 less. Just opened #1177356: Show total 'major bugs' count for 8.x and 7.x in contributor links + general tidy-up which would help to make this more transparent.

The reason it can drop by 75 (for the sake of argument) in one week, is because people open major bug reports every time they fart. I've been at least as guilty as this of anyone else over the past couple of years - for example there is no way that #1120928: Move "history" table into separate History module should ever have been posted as a bug, there are countless examples, many of which are still in the queue.

Having a sanity limit will help to enforce better triaging of issues in general, we have docs for issue priorities at http://drupal.org/node/45111, the same for categories might help a bit, I don't mind drafting something or modifying an existing handbook page.

However once issues have been triaged, that doesn't get around the fact that there are still quite a lot of very nasty bugs in the major queue.

So... I'd suggest the following:

- A time-boxed sprint to triage the major bugs queue - so that we know that the issues in there are actually major bugs, and aren't multiple duplicates (like the three separate major bug reports I found about image derivative directory permissions yesterday), or already fixed (like at least two issues chx found). If we don't have such a sprint first, then the number of bugs could go down 10 a week for months more or less just from me banging my head against the keyboard. Just for triaging, we only need 2-3 weeks - especially if a few people chip in to give things a once over.

- Only when that is done, will we know how many real, actual, major bugs there are in 7/8, my guess would be somewhere between 100 and 180. As part of the triaging, we can identify how many of these really need urgent backport to D7 (i.e. site hosing or contrib blocking), and how many are bizarre edge cases that can only be fixed with a rewrite of a subsystem or whatever. Based on that we might want to bump some of the issues to critical (i.e. an issue that blocks a stable D7 release of views or ctools is probably critical - regression from D6 or whatever), some might be quick fixes etc. - when we have the actual data, then we can re-open the discussion, or go with the 'no higher than' + targets idea, or set a different target.

I'm not opposed to a rolling total, that also gives us the flexibility to eventually keep it at a number below 100 as well, but without an opportunity to triage the current mess first it will be meaningless. Also there are still more than 15 critical issues so it won't even block anything (and I won't be helping on critical issues while that's the only sanity limit we have).

This is already getting long again but on the last two:

7.x patch priority is not fixed by webchick having commit access. If there are 10 RTBC major bugs that affect Drupal 7 in the queue, and Dries commits a megabyte+ #22336: Move all core Drupal files under a /core folder to improve usability and upgrades or 300kb+ patch like #1018602: Move entity system to a module then all or most of those RTBC major bugs need to be re-rolled. In the same way that #1097100: Remove all 7xxx update functions and tests (D6 to D7 upgrade path) stalled #1017672: D6 to D7 update process permanently deletes comment bodies and other data, and throws fatal SQL errors for weeks. If I'm working on hundred line patches to Drupal 8, we can be relatively sure I have at least passing familiarity with git and can figure out how to deal with conflicts, the same is not necessarily true for people who just want x bug fixed in D7.

If there is no distinction made, then it turns into a running race between webchick committing bugfixes and Dries committing clean-up and features - webchick might be around more but any feature or clean-up patch can move the goalposts. Without some discipline on this, I am going to start agreeing with all the people on this issue who say working in the development branch actually blocks stability in the stable release, because they'll have been right.

I agree with david on the 7.x/8.x hacks thing - it doesn't mean committing any old crap to 7.x, it means if a fix is good enough for 7.x, it's good enough for 8.x too until the 8.x change is superceded by something better.

Dries’s picture

The follow-up suggestions are interesting. I'm mulling over them. It is a hard problem, for sure.

Will reply shortly -- until we have set some guidelines, I'm going to hold back committing big 8.x patches. That allows us to make progress on 7.x or to triage the major issues.

Stay tuned, and thanks for all the suggestions. It really helps me think through this. :)

catch’s picture

@Dries - thanks for holding off feature patches until we have this sorted out.

I had a look through the major bug queue today for an hour or so, and found several dud or duplicate issues still. Current count is 198 major bug reports between both versions.

I've not tried to go through the D8 major bugs queue to add the "needs backport to D7" tag to issues that need it, nor fixed status/tagging on 7.x issues, and I probably won't do that in favour of vbo integration for project-issue. But, if someone has the time and energy to try doing that, it would help as well as triaging the content of the actual issues - that needs to happen for anything that's not an upgrade path issue and is still assigned to 7.x pretty much.

For the 198 count, I used this link:

http://drupal.org/project/issues/search/drupal?status[0]=1&status[1]=13&...

For triaging, http://drupal.org/project/issues/search/drupal?order=comment_count&sort=... is all issues ordered by least number of replies.

#1177356: Show total 'major bugs' count for 8.x and 7.x in contributor links + general tidy-up would put a link/counts for that search on the dashboard as well as some other cleanup of the block, would be great to get reviews of the proposed changes and/or the patch there.

anrikun’s picture

Subscribing.

A concrete case:
#955848: When editing an existing node with a link, the link itself is listed in "Parent item": menu_parent_options needs some rewriting
1. The provided patch is good for 7.x as it doesn't break the current API (function menu_parent_options).
2. But it is not that good for 8.x as it would be better to change API so that the way to use menu_parent_options becomes clearer.

Anyway, the patch is currently marked as for 8.x with "needs backport to D7".
In such a case, wouldn't it be better to mark the issue as for 7.x and to create a parallel issue for 8.x, adding a tag such as "needs API change on 8.x"?

sun’s picture

Not done yet. Anyway, conclusions so far:

  1. Many performance optimization related issues filed as major bug reports. I know that @catch's intention is to make Pressflow obsolete, so I didn't touch those. But strictly technically speaking, these are not bug reports. I'd recommend to change all of these to tasks.
  2. A solid range of major bugs against Postgres and SQLite. I've to admit that I have no clue what our current stance on "and we also support ... databases" is. I mean, with regard to critical/major bugs.
  3. The File API / file system is not in a good shape, stopped counting issues. Looks like sub-system maintainers could use some help. After committing all patches in the queue, we'll have replaced every effin' native file system method of PHP with a custom Drupal wrapper. Despite early expectations, our challenge turns out to not be remote files, but the very local file system at hand. I hate examples, but here's a simple one, tightly along the lines of many other patches in the queue.
  4. I might have missed it, but I don't think we have a rule for whether D7 upgrade path patches should target D8 first (like any other patches). Again, an example. I'd recommend to treat everything the same way. (Especially, since I'd really love to drop the dropping of Drupal core updates ASAP.)
catch’s picture

fwiw I'm fine with most major bug report performance issues being moved to major tasks, although it should be case by case. For example some performance issues were unintended regressions - any straight regression should be considered a bug, some we lived with for a long time and can probably continue to live with for a while, some I've personally witnessed taking sites down, there are still multiple reports out there of people not being able to install Drupal 7 at all due to memory limit or timeouts.

Time to write/update the issue category docs page, and I'll try to go through the performance major queue and move ones that really aren't bugs to tasks.

D7 upgrade hooks were already ripped out, so many issues that only add new updates can only have Drupal 7 patches now. webchick has said she wants to do all the hook_update_last_removed() some time just before release in Drupal 8, I have given up trying to come up with a single sane approach to this since it can always go wrong (a last rush on hook_update_last_removed() is probably the least worst of the current options), but #1119094: Add a test to verify the schema matches after a database update would be a start on enforcing what we have - that started off as a meta-issue discussing the overall problem.

Postgres, sqlite and file API are indeed the focus of many major bug reports.

After sun's (partial?) pass through we're down to 151 filed major bug reports. Unless we were to decide that sqlite/postgres-only issues don't get to be major (I'm not planning to argue either way on that one), I'd guess there are probably between 140 and 150 genuine major bugs in the queue now (i.e. they need to be fixed rather than triaged).

I think we're close to having the queue mostly triaged now, probably by the end of the week that'll be done, that then leaves us with where we can have a time-boxed period to try to get the queue down further as Justin suggested, and at what level and how to set the cap once that's done - although there are still more than 15 criticals so that ought to be a moot question while this is the case.

catch’s picture

Did another pass through, it is now at 131.

There are undoubtedly still some issues in there that are wrongly categorised, but also I'm sure that some will get bumped back to 'major bug' when and if people disagree with the category or priority changes.

However that means we're probably looking at a post-triage list somewhere between 115 and 140 give or take.

At that rough level, 100 feels pretty doable to me within a reasonable amount of time, weeks rather than months.

Also bear in mind we do not have an eventual target of zero like we do with criticals, we just need to get 30-40 of the easiest (or most urgent) bugs in the queue fixed, and not add a load of new major bugs in their place. So the main question now for me is whether we set the limit now and just work down to it, set a fixed time box as Justin suggested and try to get there then decide what happens if we don't, or set a sliding limit (possibly with some kind of deadline?), or something else.

Some rough stats on what's left:

Out of 131:

Twenty eight major bugs against D8 do not have the 'Needs backport to D7' tag, some are missing the tag, some are 8.x-only issues. A lot of 7.x bugs should be against 8.x, but they're going to stay on the wrong version until we have vbo on drupal.org, do a manual update, or someone bites the bullet and goes through manually.

31 issues are active.

39 issues are needs work.

59 issues are needs review (but many patches will be stale since we have no re-testing right now).

2 issues are RTBC.

Anonymous’s picture

thanks to all who've done triage work :-)

i'm ok with either an X week sprint or a target of a 100. both will give us something to rally people around, which is almost as important as actually resolving these issues IMHO.

sun’s picture

We're down to 102 now. No further conclusions.

webchick’s picture

GREAT work, guys!!!! :D

catch’s picture

I added some basic documentation for issue categories here: http://drupal.org/node/1181250

So to close this one, IMO we need the following:

- is there anything else preventing a commitment to the "No more than 100 major bugs" limit - since we're hovering somewhere just over that now? Even if it goes up again after the triage sprint, it should be possible to actually fix enough to get it back down again within a reasonable amount of time. If there's a commitment to that, it'll also need publicising/documenting somewhere.

- #147 and previous comments still have points which IMO haven't been dealt with yet - i.e. that fast-tracking 7.x patches means not committing 8.x feature/cleanup patches while there is any backlog of RTBC patches that apply to 7.x - to avoid a running race. If you're not prepared to do this, then please explicitly say that - I personally will stop defending the backport policy if this happens though, since it is indefensible as it stands.

- I don't think there is clear agreement on 'ugly but necessary' fixes to 7.x yet, or maybe there is, but details of that question are in #143.

catch’s picture

Out of interest I took a look at the 'normal bugs' queue, it is about 30 issues short of 2,000 posted against Drupal 7 and 8 combined.

I am not going to suggest a low hard limit to get that down to, even the triaging would take months, but would Dries' rolling number work for this maybe? Say set it at 1950 with the same consequences as major/critical if it goes over, and then eventually try to bring it down over time (by setting lower ceilings).

With these it is not going to necessarily help get the bugs fixed any faster at all, since there are probably hundreds of issues in there that should just be closed, but it would help keep to eventually keep the queue triaged.

In irc webchick mentioned trying to get together a 'bug squad' for core, I don't think that'll overly help with the critical/major issues - those are as much about vertical scaling (making it easier for those who might already work on getting them fixed to do so) as it is about getting more people involved, although hopefully we can get more people involved too. But for the queue in general it could help a lot.

fietserwin’s picture

Great work. However, I do agree with catch that a queue of normal/minor (±2300) bugs is simply too long to be manageable and that some steering on that would be welcome. An healthy issue queue with not to much pollution and duplication is normally a sign of a good project. So are short response and resolve times are highly appreciated. But these are nearly impossible with such a long queue (1200 waiting for someone else to review, 75 to be committed or ultimately rejected).

This probably comes down to activating more people to get really involved besides creating new issues and subscribing to already reported ones.

Note: the numbers on the statistics page (http://drupal.org/project/issues/statistics/drupal) differ a bit from the numbers in the (right) side bar (e.g. http://drupal.org/project/drupal). So somewhere some incorrect counting is done. Though the totals do not differ that much, individual counts may be more off, depending on what causes the incorrect counting.

bleen’s picture

I think something like this:
bleen18 | drupal.org

will be a HUGE help
a) spread the word to contributors old and new
b) help keep everyone informed about why their pet patch is being delayed
c) encourage people to say to themselves "if I want my pet patch to get in, I better help get that red number down"

Just a thought....

BTW: Thanks Dries, Webchick & especially Catch for tackling this issue and coming up with some really good solutions

Dries’s picture

First off, incredible job on reducing the major issues. :-)

Brainstormed about this more and what about setting the following requirements:

  1. CB D7 + CB D8 < 15
  2. MB D7 + MB D8 < 100
  3. TH D8 < 15

where CB stands for 'critical bugs', MB stands for 'major bugs' and TH stands for 'temporary hacks'.

The TH one would allow us to make a limited set of temporary compromises without painting ourselves in a corner. Temporary hacks are things we know we should fix better in Drupal 8, but which we don't want to right now in order to make progress with Drupal 7.

Put otherwise:

 if ((CBD7 + CBD8 < 15) && (MBD7 + MBD8 < 100) && (THD8 < 15)) {
   add_features_or_fix_bugs();
 }
 else {
   fix_bugs_only();
 }

Thoughts?

pillarsdotnet’s picture

Is there a "Temporary Hack" tag?

Dries’s picture

By the way, it is not every day that I get to write code. ;-)

Anonymous’s picture

re #158, i really like this.

re #160, acquia needs:

if (number_of_days_since_dries_last_wrote_code() > 7) {
  figure_out_wtf_is_wrong_at_acquia();
}
catch’s picture

#158 sounds great to me too.

sun’s picture

I can't really become fond of the third limit for temporary hacks. If I interpret it correctly (perhaps not?), then we would stop committing other things, because we fixed some critical and major bugs in a stop-gap way, and didn't fix them by radically changing D8 just because we can. Unless I'm missing something, properly_fix(THD8) is the same as add_features_or_fix_bugs().

As long as no other API changes require to fix things differently in D8, and as long as we're able to say "what's good enough for D7 is good enough for D8" for a particular patch, I think we have to commit it in order to make progress, and move the entire discussion about perfectionism into a new issue, a feature request/task for D8.

Alan D.’s picture

If I read this correctly, this should not hold up proper fixes to D8, it will simply add a temporary road-bump.

$count = CBD7 + CBD8 + THD8;
If (is_fixed(CBD7) && is_temp_hack(CBD7))
  commit THD7;
  commit THD8;
  reassign THD8;
}
$count2 = CBD7 + CBD8 + THD8;
if ($count == $count2) {
  print "keep core developers happy by forcing fixes to THD8";
}
else {
  die(shouldn't happen);
}
catch’s picture

I think the temporary hacks idea (correct me if I'm wrong..) is more about dealing with incurred code debt from things that are fundamentally broken in D7 and we could only workaround with a kludge.

So if there are 90 major bugs, 10 critical issues, but 17 temporary hacks, then it'd mean you're still allowed to continue to get the bugs fixed, but the only new development happening is refactoring/clean up to bring down the temporary hacks.

That could be a small API change (say adding a new argument to a function rather than overloading an existing one), or it could be refactoring something that is causing lots of related issues.

Possibly a good example would be #534594: [meta] system info cache writing and registry rebuilding is broken without a full bootstrap. I don't think there is going to be a pretty solution to that one for Drupal 7, because that issue is the intersection of many things that are wrong in core (i.e. that the entire system from the very lowest bootstrap levels, the installer, the class registry, more or less everything, depends on system module - which should not ever be loaded until full bootstrap if it was just a module).

However this does bring up a question. Say that issue gets committed as is, that is definitely a 'temporary hack'. How do we determine that this has been properly fixed in D8? There are multiple potential routes to removing the specific hack from D8 (make the registry only available after full bootstrap, decouple the module system from system module, all kinds of things) - but if we tag every issue that could lead to a possible fix as 'temporary hack' it is going to get counted more than once. And a proper decoupling of system module from everything else is a full release cycle task.

On the plus side, nothing got tagged as a temporary hack yet, so at least there is a head start with 15 to go for the moment.

webchick’s picture

catch is correct. This is about placing a limit on incurred technical debt.

I think the way to handle that scenario is to mark the issue where we put in the workaround solution as a "temporary hack", then after fixing it, mark it to 8.x, tag it, and make it "postponed". We could then use further comments as a "meta" issue to track related issues where better ideas are being hashed out. When one of those ideas gets committed, we'd go back and mark the issue fixed.

Make sense?

David_Rothstein’s picture

I agree with @sun. It sounds like with this plan we'd be prioritizing certain features/tasks in D8 over others (including some that might be very difficult to fix compared to the benefit gained), for no particular reason.

There is technical debt in Drupal that dates back to, say, 2003... Why is technical debt newly added in 2011 more important to fix than that?

Maybe this could work if we went back and tagged lots of historical issues also (to have a complete sense of the total "technical debt" in all of Drupal) and then set a limit on that, but offhand it sounds like too much procedure to me.

catch’s picture

Makes sense to me, having one meta issue to track works.

So one last thing here, I hesitate typing this although I did mention it in irc a week or so ago.

There are currently 2408 open bug reports against Drupal 7/8 in total (excluding 'postponed' it is 2332).

Probably even more so than the major queue, this number is so high due to lack of triage (only ~1800 of those were even updated in the past year), however there are undoubtedly plenty of real bug reports in there lost among the cruft.

2400 is a very different number from 300, no-one's looking through that queue in a couple of weeks, but we could maybe set a limit on 2450. If we used Dries' idea from a few posts ago, and brought that down by say 25/month, we'd be at zero in around 8 years.

It is a meaningless total in terms of actual bugs fixed, but it would help to stop it getting any worse than it already is. Normal and minor bugs are more like a mortgage than credit cards, but the mortgage is looking a bit negative equity to me.

catch’s picture

Crossposted with David, that's a good point as well - although if a critical/major bug fix has to be fixed by a kludge in D7, then I'd say the technical debt was incurred before the kludge went in - otherwise it'd be possible to fix it more or less cleanly. Whether that is sufficient criteria to prioritise those issues I dunno though.

klonos’s picture

Just a quick question: Are we going to enforce those limits once decided what they will finally be?

...I mean if the limit -whatever it might be- is reached, the "Priority" drop-down will not offer the critical/major options anymore or something? Is that the plan?

bleen’s picture

re #170: If a critical error is found users must be able to report it no matter how many others exists ... These limits are meant to be policies that the core committers will follow. I think its safe to say that we will not be enforcing these limits by disallowing people from reporting bugs.

In terms of practical "enforcement", I maintain my suggestion from #157 above.

catch’s picture

@klonos it's better to think of these as thresholds than limits, if we go over it triggers a mini code freeze. Doesn't actually stop new issues being added to the list if they get found.

klonos’s picture

Thank you both for taking the time to reply.

The possible/probable need to file critical & major issues past their respective thresholds was exactly the reason why I asked in the first place. I've been following this issue, but for so long that I've lost track after some time. Thanx once again.

catch’s picture

@bleen I started a patch at #1177356: Show total 'major bugs' count for 8.x and 7.x in contributor links + general tidy-up to make the counts/links in the contributor's block reflect the discussion in this issue, some kind of extra indication (and/or direct links to documentation) makes sense.

catch’s picture

A candidate for retrospective 'code debt' tagging if we wanted to go that way would be #608894: Resolve the conflict between the fieldable users UI and the profile module. There is no way that patch would ever have been committed during code thaw, but the issue was sufficiently severe that it couldn't be left as it was for a release either. We're months into code thaw now, and there have been no updates to any of the core profile issues (although profile2 seems to be coming along, haven't used it personally but http://drupal.org/project/usage/profile2).

One more possible idea for TH tracking:

If an issue is impossible to fix in Drupal 7 without a temporary hack (and sufficiently broken that we can't leave it broken), then it is probably already in the major or critical bug queue.

So a possible workflow would be commit the TH patch to Drupal 8 and 7, then move it back to Drupal 8 to work on the wider fix - but keep it at least major status when there to resolve properly.

This means you'd only have the 100 major / 15 critical bug thresholds, but some issues in that pool would be resurrected issues. Issues designated as TH would not actually get those counts down until they were fixed the second time in D8 with a proper fix (insert zombie reference here).

This would also allow for tagging with 'code debt' more generally and potentially tracking that statistically and/or with its own threshold later.

You could potentially end up with 115 major/critical bugs that are all tagged as code debt/temporary hacks in that scenario, but this seems unlikely.

sun’s picture

@catch: If you edit your comment to add an idea, no one is aware of it. ;) So, everyone, there's a new idea in #175.

Regardless of whether we are going to do that tracking of temporary hacks, I'd like to throw out a friendly reminder that tags like "D8MI" or "WCSSI" or "CMI" are meaningless cryptic garbage for newcomers, impossible to remember, and not self-descriptive at all. I'd strongly recommend to rename those via taxonomy admin UI. Initiative owners may want to agree on a consistent pattern though; e.g., "D8 Configuration initiative" ("D8 [short-topic-noun] initiative").

I'm mentioning this, because a "code debt" tag would be similarly cryptic. If we'll go with a tag, then make it follow the existing 'major' tags; i.e., "Needs proper resolution" or similar.

Regarding the idea in #175 to move issues after committing stop-gap fixes back to D8 and keeping them major, I don't think it is a good idea to "re-open" issues this way - the original issue was resolved, perhaps not in a perfect way, but the issue can be closed. Revamping the whole thing to resolve it properly should happen in a new issue that has a proper summary right form the start, explaining what stop-gap fix has been applied and why it's ugly, and in which you don't have to scroll down to comment #76 to find out what the issue is actually about now.

Overall, however, I still think that any follow-up issue for a temporary hack should have the same priority as any other improvement issue in the queue. While #534594 and #608894 are indeed two good examples for hacks, I can provide you a whole queue of other wonky hacks, bad designs, and inconsistencies in Drupal core, and I don't see why new ones should get more attention than old ones.

catch’s picture

Again, I don't think this is at all to do with new bad design - it's to do with existing bad design that caused a major or critical bug in core (albeit recently because we'd be starting the process fresh) - and the design did not allow us to do a clean bug fix in Drupal 7.

If I look at the API clean-up tag, I can see quite a few in there that would have fallen into this category. For example #550124: Remove prepared statement caching led to #679112: Time for system.module and most of includes to commit seppuku being opened, and the same bad design/circular dependency is the focus of #534594: [meta] system info cache writing and registry rebuilding is broken without a full bootstrap.

A new issue with a new summary sounds good, I need to re-read what me and David documented again...

Another possibility - what we're talking about closely corresponds to the kinds of issues found in the major/critical tasks queue - that is usually crap APIs, bad performance, some usability and accessibility issues - nearly always about refactoring existing functionality rather than adding brand new stuff.

Looked through the existing list of major/critical tasks with sun. It is not actually that big. 134 major tasks, 2 critical tasks across both versions - this is before anyone has gone through to triage, quite a lot of these are likely needs tests/needs docs from D7 too, some will be feature requests.

If we used major/critical tasks for this, it could look like this:

if ($rtbc_backportable_issues > 0) {
  review_and_commit_rtbc_backportable_issues();
}
if ($critical_bugs > 15 || $major_bugs > 100) {
  fix_bugs();
}
elseif ($critical_tasks > 15 || $major_tasks > 100) {
  fix_bugs();
  refactor();
}
else {
  fix_normal_bugs();
  refactor_normal();
  add_features();
  free_for_all();
}

I think that'd help David and sun's concerns about prioritising certain refactoring issues over others just due to what got partially fixed recently, while also providing a motivator not to put too many quick fixes into D7 just to push the bug count down.

It's possible to come up with major task candidates just from having a casual read through the code base, so it'd have to be carefully monitored if that number started to skyrocket, but given it's somewhat under control now that seems like something to watch out for rather than an imminent problem.

sun’s picture

Just a note: if the patch for project module in #1177356: Show total 'major bugs' count for 8.x and 7.x in contributor links + general tidy-up would also add a menu callback that returns the current counts as JSON, then I could add some magic to Dreditor that doesn't allow @Dries and @webchick to set a currently irrelevant/unqualified issue to fixed, or not even allowing to look at it... ;) Ahem, just kidding. :) Though, it could indeed show a warning, perhaps even for everyone.

webchick’s picture

Just playing devil's advocate, but why is it such a big deal to make the introduction of new temporary hacks a measurement of its own, separate from existing cruft? The idea is not to diminish the importance of other refactoring, but to make sure we don't accrue a bunch of new technical debt in our quest to get the bug count down. I think this is a worthy goal.

Catch's algorithm in #177 makes sense, but is really starting to get scary and complex, and hard to explain to new people (or anyone else not involved in this thread). It also requires people to understand the nuanced difference between "task" and "feature request," which in my experience even top 20 core developers don't have alignment on 100% of the time (or even 70% of the time).

catch’s picture

I'd argue that if the only way to fix a bug is a temporary hack, then the technical debt is already there - otherwise it would be possible to fix the bug cleanly without API changes or hundreds of lines of refactoring. You might make the argument that the temporary hack is compounding technical debt already accrued in the same function or subsystem (at least it probably won't make it any better), but we are usually talking about 5-10 line fixes (plus tests) in a sea of pre-existing spaghetti in these cases.

To take the debt analogy a bit further, major and critical bugs are like having a loan from a loan shark (high interest rate and they may break your legs if you don't pay on time) or a credit card debt. Refactoring is more like savings (or paying back a loan from family member maybe). If you take money out of a 2% interest savings account to pay off a a credit card debt charged at 27% then that is a good trade-off - although not everyone does that in reality. If you have some combination of all of these debts, it is pretty inadvisable to have a swimming pool installed in your back garden, but it is not a bad idea to continue to pay off the credit card debts and loan sharks even if the savings account looks a bit thin.

Or in other words - a 15 limit only on newly incurred technical debt is going to mean that perfectly good patches to refactor horrible ugly code in core elsewhere will be blocked on other issues that may not have clear fixes even. Simply because those other issues are related to recent, less than perfect, bug fixes. If I try hard enough I can probably relate any of my clean-up patches to recent bug fixes on some level, Drupal is currently a mass of interdependencies and a lot of nasty bugs in D7 stem from that, but a lot of people have less patience than I do defending their issues.

On categorisation, we haven't really had alignment on bugs vs. tasks for most of the past 3-4 years (and before), this only started to kick in during the slow, painful process of getting ~400 'critical bugs' down to 0, and due to meta-discussion in issues like this. The fact that the major bugs queue has managed to stay around ~100 (currently 105) rather than shoot back up again suggests there is not massive disconnect there any more though. Bojhan was concerned about usability issues being downgraded/recategorised, but looking through the queue with him in irc we couldn't actually find many examples where it went wrong, so even where the lines are blurrier it has mostly worked. Sun was concerned about downgrading/recategorising performance issues that I'd opened as bugs (also blurry in some cases), but in that case I recategorised several myself (although a couple are still in the bugs queue)

If we give refactoring tasks priority over new features same as bugs, then we'll be able to refine the differences between tasks and feature requests a bit more as well. A lot of the subtext to this issue is that the core queue is really, really badly triaged, so there is no way to have an overview of what's in it any more.

I will try to make a start on docs for the major bugs / critical bugs thresholds - in pirate pad most likely. And will try to add a section on tasks too to see if it's explainable.

There are new docs on issue categories at http://drupal.org/node/1181250, and the priorities of issues were recently updated at http://drupal.org/node/45111

David_Rothstein’s picture

I like the proposal in #177.

The confusion between "task" and "feature request" is already there, and the nice thing about #177 is that it begins to define an actual meaningful distinction between them. Whereas the alternative of introducing a new "temporary hack" tag or likewise and giving that meaning seems like it would just add a new layer of process that people have to figure out.

More important, #177 definitely addresses the concern I had before. It lets us fix hacks based on a combination of how important they are and how easy they are to fix, rather than an arbitrary prioritization. Keep in mind that under the plan we're talking about, any hacks introduced by recent bug fixes to D8 are going to be more difficult to remove than average, almost by definition. Because if the proper D8 fix for a bug involves an easy API change, the hack wouldn't get committed to D8 in the first place (instead, we'd commit the proper fix to D8 right away and the temporary hack to D7 only). So the cases where temporary hacks get committed to D8 to fix bugs are going to tend to be the "we need to rewrite Drupal to fix it properly" kind of issues, and we really don't want to get stuck with a rule that says we must fix those at the expense of anything else and regardless of what their actual priority is.

webchick’s picture

Ok, so then to re-summarize, what's being counter-proposed is this:

---
- Critical bugs: cap of 15
- Major bugs: cap of 100
(everyone seems on board with that)

In the case of temporary hacks required to get a bug fixed in D7, what would happen is a separate major or critical "task" issue would get created in D8, focused on refactoring the thing we just had to do terrible things to.

To avoid "things that need refactoring" getting out of hand, we would also therefore create the following caps:
- Critical tasks: cap of 15
- Major tasks: cap of 100

This would cover both new and legacy "temporary hacks" so that neither is prioritized over another.
---

Now. My only concern with this proposal is that there are major/critical tasks that are not at all related to API refactoring. For example, #1164760: [meta] New users universally did NOT understand that you can extend Drupal and #538904: D8UX: Redesign Modules Page. I actually think that these issues are accurately prioritized and categorized, currently (critical and major task, respectively). But what's inevitably going to happen (because this is exactly what happened when the major/critical bugs were triaged) is that UX stuff is going to get shoved aside and kicked to "features" or "normal" by core developers who want to work on D8 features. Or, as we inch closer and closer to the cap number, core developers may resist logging legitimate refactoring tickets because they want to work on D8 features. Do you have thoughts on these concerns?

Also, catch proposed adding caps to the "normal" bug count, but I think we really need to watch how much burden we're putting on the core development team, here. My experience so far with these caps is that it's taken days away from top 20 core contributors toggling issue statuses back and forth. Hopefully this will only have to happen in smaller spurts from now on to keep major/critical inline. But if we expand it to the ever-growing pile of normals, we're going to serious sap away at core's development powers. (Additionally, there's only so much forcing core contributors to fix bugs you can do before they say "You know what? Eff this. I'm headed back to contrib.")

Ideally, there would form a separate "bug triage team" who are not top 20 core developers would handle things like triaging the normal bug queue (this might actually be a real possibility with the new "Issues needing triage" link that catch is proposing to add). If such a team were to materialize, I think we could maybe revisit this suggestion in the future, but for now I'd really rather keep it off the table. It's clear that at the moment issue triage sucks time/resources/energy away from key people on the core dev team, and there are few enough of them as it is.

catch’s picture

In the case of temporary hacks required to get a bug fixed in D7, what would happen is a separate major or critical "task" issue would get created in D8, focused on refactoring the thing we just had to do terrible things to.

To avoid "things that need refactoring" getting out of hand, we would also therefore create the following caps:
- Critical tasks: cap of 15
- Major tasks: cap of 100

This looks right. There might be situations where we have an existing issue opening trying to refactor the same code, we could cross-link and/or bump status in those cases rather than a new issue, otherwise 100% agreed.

The two UX issues:

#1164760: [meta] New users universally did NOT understand that you can extend Drupal - that one is split between quick fix of documenting what we have better, that's kind of user interface text refactoring isn't it? D7 had tonnes of interface text cleanup that was essentially code refactoring (as much or more so than some docs and coder patches are).

The other part of it is the module/theme browser stuff - that is definitely a feature (a cool feature, but a feature nonetheless). However, to implement that feature, is going to require work on the existing update.inc system we added in D7 (and probably update status too for release xml parsing/storage etc.). That work is all firmly refactoring and bug fixing, and in fact has major bugs and tasks related to it already that are not at all at risk of getting downgraded. So stopping features in general from being committed, won't necessarily slow these particular features down - because it may help to focus attention on pre-requisites to getting them in. It might not, but I don't think it'll make the situation worse.

#538904: D8UX: Redesign Modules Page: D8UX: Redesign Modules Page. Again that pages has a tonne of code that no-one likes, and there are wonky APIs associated with it like package groups. Then there's all the issues about required + disabled + uninstalling modules which are filling up the critical bugs queue at the moment. So again while the new interface is a feature, there is a lot of groundwork leading up to that feature which is refactoring and firmly in the task category. Had the dashboard been added on the foundation of a solid block API, instead of the 4.6 era one we have now, then we may have had less lingering bugs with that one too (or at least more fun to fix).

Some UX issues got kicked to normal or features during the major bugs sprint, but that was not at all a higher proportion of issues than everything else (and there are definitely still major UX bugs around in the queue). I found one 'major UX bug' that was 99% duplicate of another issue, unless you were using IE6 or IE7 - this was just old, stale and outdated like so much of the other crap in there, it's not getting picked on because it's UX.

I agree we need to keep an eye on that (and performance and accessibility too), but generally people are pretty careful - or no worse than any other issue.

Ideally, there would form a separate "bug triage team" who are not top 20 core developers would handle things like triaging the normal bug queue (this might actually be a real possibility with the new "Issues needing triage" link that catch is proposing to add). If such a team were to materialize, I think we could maybe revisit this suggestion in the future, but for now I'd really rather keep it off the table.

That's fine with me, I'd add to this that it took 4-5 of us spending a couple of weeks getting the major queue down before people started to take the 100 cap seriously, if we'd just added the cap without the triage first it might have been at 200 or more. However the fact that we had a good chance of capping it once it was down, was a motivator to do the triage, so the same might happen if we can assemble a bug team. But doing this in a new issue in a few months is completely fine with me (although I may open that issue sooner rather than later if things continue to get rapidly worse with that queue).

If I open a piratepad, can we try to thrash out docs in irc once there's something in there to work with?

webchick’s picture

Ok, that's true. My viewpoint might've been skewed because I was naturally subscribed to most of those UMN UX issues and saw this disproportionately happening there in comparison to other places. We have a completely separate problem, which is that issues aren't the best place at all to have these brainstorming discussions, but that's Prairie material. :)

Totally down with hacking on a piratepad over IRC, but about to sign off for the night so maybe tomorrow instead. I also shot Dries an e-mail with this final counter-proposal, so we'll see what he thinks. :)

Alan D.’s picture

Related to the "bug triage team", has there been a policy change on what to do with old bugs?

There are now only 3 active Drupal 5 bugs, and activity flagging a lot of the Drupal 6 ones to postponed.

Firstly, it is great to see these queues cleaned up, but I think that this is a bad move tagging these as "closed (won't fix)" as there are still some users running these older versions and it will now be nearly impossible to find fixes & patches in these queues when we find issues supporting these older clients.

A new flag should have been created first, that would allow filtering. "closed (unsupported version)"

catch’s picture

Here's some proposed documentation for this, if Dries is happy with the current proposal, we can get this into a handbook page, publicise it a bit then mark it fixed. Thanks for webchick for reviewing the text in piratepad/irc. Hopefully this documents both what the proposal/counter-proposal will mean for the purposes of a final ruling on this issue, as well as for documentation if that's what we decide on - if we need to tweak again, hopefully the docs will just need tweaking too.

Code freeze, code thaw, issue count thresholds.

Drupal core release cycles are divided into three main periods: code thaw, code slush and code freeze.

Two issues were identified during the course of working on Drupal 7:
- the difficulty of tracking backports from the development branch to stable branches.
- code debt (in the form of bugs or API deficiencies) incurred due to the addition of new features, which led to a much longer desired code freeze for Drupal 7.

To assist in the resolution of these issues, the following measures have been put in place:

Fast tracking Drupal 7 backports

Patches marked "reviewed and tested by the community" and also tagged for backport to Drupal 7, will normally be given higher priority than patches that affect only Drupal 8. webchick, the Drupal 7 branch maintainer, has access to commit patches to Drupal 8 to assist with this process.

Issue count thresholds

To avoid the situation in Drupal 7 where we froze the code and then had to spend over a year stabilizing the release, issue count thresholds have also been added during the Drupal 8 release cycle:

- Critical bugs, across D7 and D8: no more than 15
- Major bugs, across D7 and D8: no more than 100
- Critical tasks, across D7 and D8: no more than 15
- Major tasks, across D7 and D8: no more than 100

If these thresholds are exceeded, a mini "code slush" will be instituted, meaning the following:

- New features and functionality will not be committed while the thresholds are exceeded. Focus will be solely on bug fixes until the queues are back within the thresholds.
- Relatively minor, far-reaching patches that touch a lot of files (e.g. coding standards clean-up)—even if they are legitimate bug fixes—will also not be committed until thresholds are met, to ensure that more urgent bug fixes are able to target a relatively stable code base.

Frequently asked questions

What about normal and minor priority bug fixes?

No thresholds have been added for normal and minor priority issues, however the 'mini code slushes' will allow normal and minor bug fixes to be committed while working towards getting the major/critical queues below threshold.

What about Drupal 6?

These thresholds do not apply to Drupal 6. There are significant changes between Drupal 6 and Drupal 7, and Drupal 6 does not have many automated tests. This makes the process of backporting even bug fixes more involved than from Drupal 8 to Drupal 7.

Can I add new major or critical issues when the counts are over the threshold?

Yes! These are thresholds rather than limits - and only affect which patches will be committed, not what can be reported or worked on during code slushes. One thing to bear in mind is that having the thresholds is likely to lead to closer triage of the core issue queue, and more of a distinction made between bugs, tasks and feature requests. You may find your issue is triaged to a different priority or category. Issues can be better triaged if there is a clear explanation of the problem (including steps to reproduce) and/or a clear explanation of the proposed solution.

Dries’s picture

The proposal in #186 looks great to me. I'll try to announce it on my blog this week. Feel free to go ahead and put it in the handbook as well.

arlinsandbulte’s picture

#186 looks great, but perhaps it could be more generalized to not specify the Drupal version (6, 7, or 8 for example). Thus, when D8 is released, this doc would not need to be rewritten just to update version #s.

Could #186 be re-written to say something like:
Current Drupal Stable Branch or Release (D7 at this moment)
Previous Drupal Stable Branch or Release (D6 at this moment)
Next Drupal Development Branch or Release (D8 at this moment)

or something like that... not a deal-breaker, but it might make it easier in the future...

webchick’s picture

Status: Reviewed & tested by the community » Fixed

Awesome! :D

Ok, I have impersonated catch and created a handbook page on his behalf at http://drupal.org/node/1201874.

Probably the next logical thing to do is update the contributor dashboard block at #1177356: Show total 'major bugs' count for 8.x and 7.x in contributor links + general tidy-up to reflect this, so it's more visible.

Marking this sucker FIXED at long last. Yay for one critical task down! :)

webchick’s picture

arlinsandbulte: Sure, that sounds good. Go ahead and edit the handbook page directly to clarify a bit.

klonos’s picture

Awesome work everybody! Thanx for all your hard work and time spent on this one.

catch’s picture

Dries, that's great, really happy to see this one marked fixed.

@webchick, thanks for posting the handbook page.

I have updated the patch at #1177356: Show total 'major bugs' count for 8.x and 7.x in contributor links + general tidy-up.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

David_Rothstein’s picture

I noticed that the Backports Policy (http://drupal.org/node/767608) still said the policy was not finalized and was under active discussion here.

Since that's not true any more, I edited the page to remove that note and also to clarify what the final policy wound up being.

If anyone wants to review the changes I made, they are here:
http://drupal.org/node/767608/revisions/view/1484808/1695908

xjm’s picture

Issue summary: View changes

Updated issue summary.

silverwing’s picture

Issue summary: View changes

Updated issue summary.

xjm’s picture

Issue tags: -Needs committer feedback