Support unstable2unstable upgrades? [policy, no patch]

I think it's worth discussing the pros and cons of this.

Pros:
- It saves time, both on the development and review side, not worrying about the upgrade path until later.
- It allows us to avoid "upgrade function bloat." For example, the /core directory problem is easy to fix with a simple empty update function, but we might later in the release obviate the need for it by fixing other bugs.
- It gives us the flexibility to refactor the upgrade path before release, consolidating similar upgrade functions or functions affecting the same parts of Drupal, for better performance.
- It avoids the risk of spending a lot of time on an upgrade path for small change A, only for large change B to re-write some entire subsystem, requiring the re-writing of those upgrade paths.

Cons:
- It doesn't actually save time. It only postpones the pain. And it postpones it until well after code freeze, when we lose more than half of our active core development team.
- This policy drastically reduces the number of people who are able/willing to test against the latest version of Drupal to only the very most adventurous of folks.
- It also far reduces the quality of testing we get, since no one's going to bother making any "real" sites when they might be told at any given time "Sorry, drop your database and start over." We lucked out in Drupal 7 that Drupal Gardens, Examiner, and a handful of other "real" sites were brave enough (and had enough core developers on staff) to chase HEAD. That situation is by no means guaranteed to repeat ever again.
- The policy results in colossal wastes of energy like http://drupal.org/project/head2head, where people who needed to solve these problems anyway had to solve them off in contrib land, instead of in core where the solutions would've benefited far more people.

And then in response to the Pros above not directly refuted by Cons:

- Honestly, who gives a crap about "upgrade function bloat" and a clean, refactored upgrade path? No one looks at these functions ever again unless they're causing a problem, and they're the first thing ripped out of a Drupal N+1 release. Let's go for function here, rather than style.
- People are far more motivated to fix upgrade path bugs when it's tied to getting in a feature they want to see. When it becomes "just" a bug fix, months after the fact, unless it's actively biting them it might as well not exist. I can count on many hands the number of people who added feature patches to core, but I can count on only one (maybe one and a half) the number of people who contributed significantly to fixing the upgrade path so we could release.

So I think the current policy is absolutely terrible and really ought to be scrapped, personally.

Ok, something like that I could see. HEAD2HEAD with an asterisk. ;)

Another possible option is moving to time-based (or however you want to do it) unstable releases of Drupal 8, and only support upgrades between those. Food for thought.

This issue definitely needs chiming in from chx and DamZ, who wrote the upgrade path tests that allowed us to release 7 in the first place. chx is also familiar with the pain we saw in Drupal 7.6(?) when we introduced a one-line adjustment to an upgrade function that "should be harmless" and ended up destroying node bodies for localized D6 sites. It took us a good month to figure that problem out. BTMash and Boobaa would be other great voices to hear here as well.

Maybe a discussion worth announcing at http://groups.drupal.org/core ?

I do not readily see why a D7->D8 code upgrade path properly tested wouldnt caught the bugs introduced by the Exodus patch but even if not, as catch says this is completely ridiculous with such a moving target.

We would need to embrace branching and cultivate UNSTABLE release branches with mini freezes of a few days without blocking HEAD to make this possible.

Guess most know it already, but I'm going to state it nonetheless: A huge +1 for bringing back head2head updates!

• It allows people to run simple(r) production sites on Drupal 8 while it is being developed.
• It's a huge support and motivation for initiatives like Snowman.
• It allows us to identify unnoticed larger problems through actual usage, and fix them prior to release.
• It's a huge motivation for contributed modules and themes to start porting to Drupal 8 early. (This motivation is non-existing currently, 'cos it simply doesn't make sense at all to provide a module for sites that can't be used.)

Limiting to unstable releases (possibly following the monthly point release schedule?) sounds fine to me as well.

I like the compromise of introducing more frequent unstables and supporting upgrade path between them. It gives us real targets and avoids some of the issues catch described with doing upgrade paths as we go along. I know for myself it would probably be enormously helpful to have the motivation to write upgrade path for CMI as we push the patches in (more painful and tedious for sure, but I can't deny the benfit to the release.)

Unlike stated in the issue opener, we did support head2head updates in Drupal 6 at least. I know we did not do in Drupal 7. A considerable issue with that was that we cannot go back and optimize/change earlier update functions, and we needed to go hoops to skip certain update functions if you ran them on sites which did not yet run them, when we had a more performant or more encompassing update done. So head2head updates ended up with mostly poorly performing (slow running, repetitive) updates in the final product. Except that I'm all for head2head updates as much as possible. I know some big changes we have make it pretty hard to keep even everything working all the time, let alone keep them in an updated state though.

However I do not want us to get ourselves into a situation where we have to support UNSTABLE-1 to UNSTABLE-26 and every combination of releases in-between.

So we'd basically say: "You should be able to update from UNSTABLE-1 to UNSTABLE-26, but you have to incrementally update to every intermediate step in between, since a direct jump to 26 is not supported." ?

(which doesn't mean it won't work, but there's a chance it might not) Not sure whether that makes sense.

I can see the argument of not wanting to open up the possibility of additional update path and upgrade path "branches" (starting points), and retaining the option to improve or rewrite updates at a later point. However, ultimately, I think this entire idea only makes sense if there is a certain level of reliability involved. As soon as there's a relatively high chance of your site being hosed, the entire idea and effort is going to collapse. That's I think what we learned out of early UNSTABLE+ releases in D7 - good idea, but kinda wasted effort, since no one was able to actually do something with them in a reliable manner.

Brainstorming:

- We might need different official support stances on D7->D8 upgrades vs. pure head2head updates?

- What if (some) updates would live in an actual head2head.module (in core)? (Only enabled on sites that started on D8, and to be removed entirely prior to release?)

I don't understand what the problem is here: in most commercial software development that I've done, if I introduce a change that requires some sort of database or configuration update, I'm expected to write that update and commit it as part of the patch that introduced the change.

Why can't we do the same thing with Core? If a patch requires an update function, add it. Could somebody clarify why that would/wouldn't work?

Trying to support one UNSTABLE release to the next allows developers who have limited resources to realistically choose to use the in-development version of Drupal for an experimental, personal, or beta product. Agaric would. I think this is huge for engaging the larger Drupal community in core development.

@cweagans: in Drupal 7, if you had an update to write that iterated on all nodes like the previous update function, you did not need to write the whole thing again, you can just go in and develop further that function. That has the advantage of not needing to duplicate more code and speedier updates eventually (no need to reiterate on the same nodes multiple times lets say). However, it pretty much rules out head2head updates. So writing the update code is not in itself enough to support head2head updates.

In response to #13, we already do that. But that update function in D7 was only expected to upgrade you from whatever the latest stable release of D6 was to the latest version of D7.

What's being suggested here is that update functions in D8 would need to work for *both* the latest stable version of D7 *and* the D8 unstable release from <= a month ago. Both paths would need to be tested and coded for.

Gábor's point is a good one, btw. That it's really sucky to have to do something really computationally expensive, like loop through all of the nodes/users on the entire site and save them all, more than once during an update.php run. Consolidating those types of updates into single update functions that perform both task A *and* task B, which wasn't discovered until later, while looping through all those records and saving them is a nice thing to be able to do.

However, in such instances, it seems like we ought to be able to do something like:

function system_update_8002() {
  // Moved to system_update_8012() so we can update both the foo and baz columns at once.
}

...

function system_update_8012() {
  foreach ($users as $user) {
    if (that one table column you're about to update already has been done) {
      // Skip this part.
    }
    else {
      // Update the table column.
    }
  }
}

These update functions can be as ugly as we need them to be, IMO. They're only around for the life of one release.

@webchick: yes, we did that a lot in Drupal 6, and those update functions get pretty ugly sometimes. Tradeoffs :)

This is progressing nicely however I want to state that I am not the upgrade maintainer for Drupal 8, it was a conscious decision on my end.

Now, what catch says about not being able to upgrade UNSTABLE-1 to UNSTABLE-3 just makes sense because the API the upgrade from -1 to -2 simply might not be there and that's why we really can't support that.

So if I'm understanding this "latest unstable only" concept, we're saying that if you're on unstable-3, then to get to unstable-5 you need to not just run updates, but actually grab the unstable-4 tag and run its updates, and then the unstable-5 tag and run its updates.

That means that we *are* free to go back and mess with update functions and not care that they break the -2->-3 upgrade when working on unstable-5, as long as D7->D8-latest still works.

That is, for those chasing head "A man who falls behind is left behind." :-)

Is that what we're suggesting?

I don't see anything wrong with that. If someone's chasing head, they're likely to be on top of it anyways.

Even though it was my patch that broke everything in our first HEAD2HEAD break in D8 (I think) in #1327958: D8 upgrade path before /core move is completely hosed, I'd be behind this also. As webchick pointed out in that issue, preventing a breakage would have been trivial just by adding an empty update function. But the patch didn't include one just because HEAD2HEAD updates are not currently our policy.

#20 is what I suggest yes.

Just to clarify, with the current update testing we would need to keep a Drupal 7 database dump around (7.0? latest 7.x? I am not sure) and also a latest UNSTABLE-N DB dump and run the tests on both. Not too bad but we need someone to redo the dumps every month if that's the unstable frequency and that someone, as I stated above is not me.

After chatting a bit with webchick and catch in IRC, I'd like to volunteer to create the database dumps at some specified interval. The previous comments mention ~1 month; webchick/catch say ~2 months in IRC. Either way works for me.

Sorry, but I don't think that "require to chase HEAD" can ultimately work out, because:

1. You are on UNSTABLE-12.
2. UNSTABLE-13 is released, you run updates, and they succeed, but they wreak havoc.
3. That critical update path bug gets fixed. Only way to fix it obviously is to adjust the existing update.
4. UNSTABLE-14 is released, there are no updates to run.

And so this process and result will occur regardless of whether you're closely chasing HEAD or whether you iterate over the UNSTABLE releases to catch up later on.

An awkward way to work around this would be to allow git commits to UNSTABLE tags.

But then again, the situation can very easily get worse, say, in case that critical update path bug is fixed in UNSTABLE-15 only.

Sleeping over this, the above mentioned idea sounds more and more appealing to me:

What if (some) updates would live in an actual head2head.module (in core)? (Only enabled on sites that started on D8, and to be removed entirely prior to release?)

You roll back to your backup of UNSTABLE-12. It's not called unstable for nothing.

So much I get :P But how do you get past UNSTABLE-13, if UNSTABLE-13 contains a bogus update, combined with the constraint that you have to run updates of UNSTABLE-13, as proposed here? :)

I strongly support doing head 2 head upgrades. If we don't want to do it with hook_update_N() would could use hook_update_alpha_N() or something like that and get the best of both worlds. But there's no reason to continually break everybody's environment all the time.

I hadn't seen the issue (webchick brought it up to me after thinking about the potential un-usefulness of #1182290: Add boilerplate upgrade path tests if the dumps used for it aren't kept up to date for testing upgrades). @cweagans, let me know if you need help as I'd also like to volunteer.

Just wanted to chime in here. If we got through the process of writing updates for core more frequently and in less pressured circumstance than WE NEED THIS NOW SO WE CAN GET IT OUT THE DOOR then it could bring novice Drupal devs like myself along.

I'd love to know more about everything that it takes to get a major version of Drupal out the door so that you guys don't have to do it all, all the time.

So if the work that it takes to get minor versions of Drupal to upgrade "Head to Head" is easier than trying to upgrade D7 to D8. Maybe someone of my experience can help out.

One thing I'm wondering is if the tests could somehow be hosted on the d.o infrastructure. I'm thinking we could have some scripts on the server that generate these dumps automatically (and could be on a more frequent and timed basis) and when it comes time for testing, the end user gets these dumps. I'm not sure on what this would entail, however and the possible ramifications of such a move.

It seems like we have relative consensus here. Assigning Dries and marking RTBC.

Summary (feel free to correct if I got any of this wrong):

- Supporting HEAD 2 HEAD upgrades during the development cycle is generally seen as a very good thing, for enabling better testing not only within the Drupal core development team, but for outside, "slightly adventurous" site builders as well. This is really valuable for fleshing out problems while we still have some flexibility to do something about them.

- However, the idea of supporting any past version of D8 to any later version of D8 is sheer and utter madness, so we should support unstable-to-unstable only, 1 step at a time. (Unstable 1 => Unstable 2 => Unstable N => alpha1 => alpha2, etc.) Unstable releases could either be rolled on a predictable schedule (say, a week or so after D7's releases), or could just be rolled ad-hoc whenever a major patch lands and it seems like a good idea; doesn't really matter, and we could figure that out elsewhere in a non-critical task.

- We will need some folks to help with re-rolling the new upgrade path database dumps each unstable release. Both BTMash and cweagans have offered. I know BTMash has done this before, but we could use some docs on how to do that in case he's unavailable. We can handle that in a non-critical task.

Good to go?

cool, @webchick you can count me on the list of people who will help with the rerolling of updates from one target to the next.

Sorry to be so late to the party, but there is something that I really don't like in here. It is critical that we allow ourselves to refactor the upgrade path as we go. The upgrade path has been a lot of work in Drupal 7, not only because the upgrade path stayed broken for a long time, but also because we changed our mind about stuff a lot along the way. We ended up renaming some tables a couple of times each, adding and then removing indexes, etc.

The upgrade path definitely needs to stay:

• clean
• testable
• efficient

... or large sites are never going to upgrade.

So here is what I suggest:

• We support HEAD to HEAD upgrade,
• But we only support upgrade from UNSTABLE(n-1) to UNSTABLE(n). Everything before UNSTABLE(n-1) is fair game for refactoring: we can remove update functions, merge update functions together, etc.

The application of this suggested policy is simple:

• In each .install file, we separately keep track of UNSTABLE(n-1) to UNSTABLE(n) upgrades and of the UNSTABLE(n) to UNSTABLE(n+1) ones (by putting them in different docgroup, the same way we currently track post-release updates
• After each UNSTABLE release:
  • We remove the UNSTABLE(n-1) to UNSTABLE(n) group, those update functions become fair-game for refactoring
  • We open a new docgroup for UNSTABLE(n+1) to UNSTABLE(n+2)

Hm, that's an interesting idea. Would that still preserve the ability to move from Unstable LastN to Alpha1? If so, then I think that'd be fine.

Would that still preserve the ability to move from Unstable LastN to Alpha1? If so, then I think that'd be fine.

Yep. But only to Alpha1, not Unstable LastN to Alpha2 for example.

Cool! That works for me. Anyone else have opinions?

Ok, so just so I understand, can we walk through a sample scenario?

Right now, locale.install has 3 update_N() functions.

The one in the middle, locale_update_8001(), gathers up a bunch of columns (plurals, formula, javascript) from the languages table and saves the results to the variables table, and then drops the columns from the table.

Let's say that UNSTABLE-1 is rolled next week, with the current state of things.

Now let's imagine that after UNSTABLE-1, but before UNSTABLE-2, we decide that we didn't actually want to drop the 'javascript' column after all. Oops. Now we need to do two things:

a) In the UNTABLE-1 to UNSTABLE-2 upgrade path, we need to add a "javascript" column to the table, get the value of the locale_translation_javascript variable, parse it out, and save the results to the "new" column.

b) In the D7 => D8 upgrade path, however, we just need to remove the three lines from locale_update_8001() that deal with the JS column.

So how would we deal with this?

I think how we would deal with this is we add a locale_update_8003() that checked for the existence of that variable, and if found, would do a), and then we'd further adjust locale_update_8001() to do b). We roll UNSTABLE-2 and call it good.

But now between UNSTABLE-2 and UNSTABLE-3, what happens? Do we drop locale_update_8003? And if so, how do we reconcile the schema differences between an unstable upgrade path and a normal upgrade path? Do we empty out the function body and say "// Moved to locale_update_8001()"? Or would we do as rfay suggests and make a "locale_update_unstable1_8001()" and call it from locale_update_8001(), then know to remove all of the _unstable1_ functions upon rolling UNSTABLE-3?

Hopefully that made sense. :D

Let's suppose that UNSTABLE-1 is released today. It contains the following upgrade path:

// From 7.x to 8.x-UNSTABLE-1
locale_update_8000
locale_update_8001

Now let's suppose we decide to revert the change in 8001, we add a 8002 that is going to be in the UNSTABLE-1 to UNSTABLE-2 upgrade path. Let's suppose we release UNSTABLE-2 right after that, it will contain:

// From 7.x to 8.x-UNSTABLE-1
locale_update_8000
locale_update_8001
// From 8.x-UNSTABLE-1 to 8.x-UNSTABLE-2
locale_update_8002

At this point, we are free to refactor 8000 and 8001 if we want (but it would probably not make a lot of sense to do so).

Let's suppose then we make another change, 8003 after UNSTABLE-2.

When tagging UNSTABLE-3, we don't support UNSTABLE-1 to UNSTABLE-3 anymore, so UNSTABLE-3 contains the following upgrade path:

// From 7.x to 8.x-UNSTABLE-2
locale_update_8000
locale_update_8001
locale_update_8002
// From 8.x-UNSTABLE-2 to 8.x-UNSTABLE-3
locale_update_8003

At this point, we are free to refactor 8000, 8001 and 8002, so we can probably just drop them. When we drop an upgrade function, we keep a ghost of it so that numbering still matches (because the schema version of UNSTABLE(N) has to be > UNSTABLE(N-1)).

Ok cool, I follow all of that except:

"At this point, we are free to refactor 8000, 8001 and 8002, so we can probably just drop them."

But don't you still need 8000-8002 for the D7 => D8 upgrade path? So we can't just drop them, correct? Or would we be moving the entire upgrade path from D7 => D8 *and* UNSTABLE-2 -> UNSTABLE-3 into locale_update_8003()? Basically, under what circumstances would we drop a previous update function?

However, the general philosophy of "New update functions for additions / adjustments to the upgrade path, leave stub functions in for previously obliterated updates" (which is what I think you are saying?) sounds good to me!

It has proven to be invaluable to get people to adopt development snapshots of Drupal 8. At the same time, having to support head-to-head upgrades has proven to drive more than one core developer insane. Unstable-to-unstable upgrades, one step at a time, with the ability to refactor older unstable-to-unstable upgrades seems like a great consensus to me. I'm on totally board with that.

Hm, I think this is reasonable, but with some important caveats...

1. This should be a guideline, rather than a hard policy.

I'm basing that on the experience of having spent a lot of time working on the head2head module for Drupal 7 (and similar custom code). Most of that work was painful and time-consuming. I have no interest in doing it again, and definitely don't want to see all core developers forced into doing it (there are much better, much more productive things that can be done with one's limited core development time).

So it's reasonable to make a best effort to support these upgrades (i.e. in cases where it's trivial, such as some of the examples discussed above) and also prioritize avoiding things like whitescreens and massive data loss, etc. But if there are patches that will require a lot of extra work to support an UNSTABLE-X to UNSTABLE-X+1 upgrade path, that work simply isn't worth it, and I don't see why issues should be held up over it. Someone running an unstable version of Drupal core on a live site ought to expect to occasionally need to do some manual tweaking during upgrades if they want their site to remain perfect.

2. We also need to be honest; this policy by itself (even if it were to be a hard and fast policy) is not going to significantly change the kinds of sites that can actually try to run off UNSTABLE releases.

Again speaking from experience, I can say that head2head upgrades were only one of the things we worried about during the lovely time period when we were running production Drupal Gardens websites off of D7 unstable code. Other fun things included:
(a) extremely nasty, critical bugs that were just out there waiting for us to be the first live sites to hit ("all uploaded files get automatically deleted after 6 hours" was a fun one),
(b) the possibility of new features that our users were actually depending on later getting removed from Drupal core and us having to figure out what to do about it (a scenario for which it's obviously impossible for Drupal core itself to ever provide an "upgrade path" for),
(c) the possibility that a critical core security issue would get (justifiably) reported in the public issue queue with no immediate fix available and with our sites vulnerable to it.

End result: You are still going to need a team of experienced people with a lot of time on their hands in order to adopt these releases on any kind of meaningful website.

3. Instead, if we want to improve the testing we get before Drupal 8.0 is released, I'd suggest we look more at the development cycle.

Drupal 7 was unusual in that it had a very long "code freeze" (whatever that meant exactly) followed by a relatively short beta. However, a beta is actually the perfect time to require the upgrade path to work and to expect people to start testing the software in more realistic situations. So instead of basing our policy off the history of Drupal 7 and focusing on what we will support in unstable releases, I'd suggest we aim for Drupal 8 to simply have a shorter code freeze and a longer beta, and focus on getting people to start adopting (and heavily testing) the software during the beta.

A couple things:

1. While I can acknowledge that once in a while it might be ridiculously hard to do an upgrade path for a change and we could possibly make an exception i'm not sure a guideline actually buys us anything. But i would ask, Wasn't most of the pain in maintaining head2head updates because features were being introduced willy-nilly under your feet and you had to go back, as someone uninvolved in that patch, deduce what had happened, then pick apart the changes, then provide an upgrade path, then test it, etc? If that work were instead distributed to actual feature authors as part of the condition of getting their changes in in the first place (the point where they are both motivated and knowledgeable about the change) rather than a separate body having to play "clean up crew" wouldn't that help mitigate some of these concerns?

2. Fair points. Can't argue with them, other than I see representation here from many top-tier developers who are saying they need this in order to use D8 seriously before beta.

3. Do usage stats actually back up that a huge influx of testers came in at beta? IMO that only happens at final release. :( I'm on my phone ATM but will check when back on my laptop.

Yay, fun with Excel! Here's the usage stats for D7 piped into a chart:

Here's just the first part of that chart, before release, since the detail gets lost in the overall picture:

So basically, while we do see roughly a doubling of testers each time we introduce a new "phase" in release (alpha/beta/RC), the real jump comes in when we put out a stable release. Some people are early adopters, most people aren't. If we make it easier for early adopters to adopt D8, then we should get more and better testing from them while we can still do something about it, which we can't anymore at beta.

Here's the data, too, if someone else wants to play. It's only up to October of this year since it's from my d.o sandbox.

Love the Excel fun. Data is always helpful so thanks for collecting it.

Looking at the first graph, one could conclude that it is not really worthwhile to guarantee head-to-head upgrades until after the first beta release.

Maybe the rule should be: "we only worry about head-to-head upgrades after beta 1".

Maybe that should be our definition of a beta release?

Yeah, that's what we did for D7. But it creates a number of problems:

1) You literally can't even run a simple blog like buytaert.net on HEAD, let alone a large-scale project like Drupal Gardens or Examiner.com (unless you employ a gaggle of core developers, that is), while we don't support upgrade paths. Even for small changes, our standard response in the issue queue to "Help, I git pulled and now my site broken!" is "Blow away your DB and start again." This gives us much less useful test results during the development phase.

2) Then, by the time we reach Beta, we are done with API changes, done with schema changes, done with UI and text changes, and everything else that would be remotely useful to be able to change when "real world" sites are using it and providing feedback. We're firmly into "let's fix bugs and get it out the door" by that point.

3) Not fixing the upgrade path as you go along leaves a huge glut of technical debt clean-up to the very end of the cycle, LONG after people who broke it remember what the heck they were doing, and long after we have the traditional falling-off of core developers following the end of code thaw. So you're left with a skeleton crew (literally; there were only about 3-5 people doing the bulk of the fixes to the upgrade path in D7) of extremely fried people who're doing nothing but fixing bugs, now fixing upgrade path problems for features from other people who've long since left the core development team for shinier pursuits.

4) People point at a lot of things for how long the D7 cycle took, but I'm going to point my finger of BLAAAME at the upgrade path. ;) You can see from the graphs that it took 10 months to get from alpha to beta, because we could not roll a beta release until the upgrade path was working. Then once that was ready, the rest of it (beta1 -> final release) only took about 3 months.

So yeah, the point of this issue is that we'd really rather not repeat that again if we can possibly help it. :D

The point is to

1. Allow people to use the current state - while we can still change it.
2. Implement and resolve the upgrade path when performing changes - not when we want to release.
3. Allow people to use what they have contributed - without having to wait years for it.
4. Encourage contrib developers and eco-system to adopt and work with the new core earlier - to identify problems and regressions before it is too late.

The statistical data of D7 is misleading. We had unstable releases, but there was no upgrade path in core. head2head.module existed, but you had to know of it, and it was too complex for a single person to consider.

If there's any point in statistical data for this issue, then at maximum, we could compare usage stats of the last core cycle that had head2head upgrades built-in (D5?) and somehow anyhow try to put them into relation to our current design parameters.

Hm. Sadly, our usage stats only go back as far as June 2008. :( (And download stats only since Jan. 2009.) So we don't have data on either the D5 or D6 release cycles for that.

But yep, agreed with those points. sun nicely sums up the benefits of this change, and that we should highly suspect that the stats seen here are a result of the current status quo regarding our upgrade path policy, not an innate human propensity to holding off on taking something moderately seriously before beta. :)

2) Then, by the time we reach Beta, we are done with API changes, done with schema changes, done with UI and text changes, and everything else that would be remotely useful to be able to change when "real world" sites are using it and providing feedback.

Right, so I forgot about the fact that Drupal currently defines a "beta" differently than any other software project in the world :) So, revised proposal: The point at which we should start guaranteeing an update path is the "polish phase" (as defined at http://drupal.org/node/935558). Most software projects would call that a beta, but whatever. The point is, the right time to start guaranteeing people an upgrade path is when we're done totally revamping features and ready to start polishing and fixing what we have. I don't see how it makes sense to do it during an "unstable" phase (but again, best effort is reasonable during an unstable phase and I do think best effort would go a long way).

But i would ask, Wasn't most of the pain in maintaining head2head updates because features were being introduced willy-nilly under your feet and you had to go back, as someone uninvolved in that patch, deduce what had happened, then pick apart the changes, then provide an upgrade path, then test it, etc?

Sometimes, maybe, but really not always and especially not in the hardest cases. Upgrade path issues can be much more complicated than the original issue, and many of them aren't going to be easy for the original patch author to deal with either.

I looked through the head2head queue and tried pulling out some issues that I think illustrate the problem. Here are some examples:

#906092: #812688: Update existing image field instances to use formatter settings
#818530: Head2Head update for numeric list fields no longer being unsigned
#824932: Update for user signature_format

All these are cases where the original core patch added or changed a feature that was new to Drupal 7, so it didn't have to think about upgrading existing data because it could assume there was none. Once you start assuming there is data things get a lot more complicated, and that's what we dealt with in head2head. Early in the D8 cycle, I don't think it's productive to waste time on those kinds of things; if people's test sites' user signatures are broken (or similar), I think it's fine to ask them to fix it manually.

In terms of sheer numbers, most of the issues in the head2head queue aren't like the above, though; I think they were more dealing with core update function renumbering or small schema changes that core didn't write a new update function for, annoying things like that that would have been easy to fix in core. So by doing "we make a best effort to support upgrades" we would still avoid a large number of head2head issues anyway.

3) Not fixing the upgrade path as you go along leaves a huge glut of technical debt clean-up to the very end of the cycle, LONG after people who broke it remember what the heck they were doing, and long after we have the traditional falling-off of core developers following the end of code thaw.

Well, we always want to support the D7-to-D8 upgrade path as we go, right? I didn't think that was up for debate in this issue, only whether we also want to support D8-to-D8.

This is a very immediate question for http://drupal.org/node/1301040#comment-5307336, which is to remove about 4 update functions and move all those code to update.inc to fix the bootstrap. For that patch, answering whether it should introduce no-op update functions like Drupal 6 used to do or remove and renumber update functions like Drupal 7 did would be important. I don't have a good understanding of the current suggested approach, but would need to apply that for the patch in question :)

Interesting discussion. There are a host of tradeoffs here. I'll just state my motivations for wanting this:

1. I want D8 to be our highest quality release ever. We need live sites running it before release to shake out bugs and misfeatures.
2. It takes substantially more work to write patches that have to deal with two upgrade paths (D7 => D8 and D8 => D8). One is bad enough. We don't want development to get *harder* or less fun.
3. I don't see much value in preserving dev sites as d8 progresses. use drush site-install whenever you need it

Given those motivations, I am in agreement with #49 by Dries that we do this only after beta1. I also like David's suggestion that this is a "This should be a guideline, rather than a hard policy.". Maybe it becomes policy for D9. We already have few developers minding the upgrade path. This is just one more pain in the arse.

Sorry for my late reply here - was on vacation.

Based on David's comments in #46 and #55, how about we:
- Support latest 7.x tag to latest 8.x-unstable tag.
- At some point in the code slush / polish phase, release an unstableN, and communicate that from that point forward, we start supporting unstableN -> unstableN+1 updates, but not unstableN -> unstableN+2 (as per #37).

As per #46.2, are we really expecting anyone to be running real sites on D8 during code thaw without the staff needed to deal with all of the difficulties that entails? However, we do want some brave folks running real sites on D8 well before beta1, as per #51.2 and #52.4.

Re #6:

It's a huge support and motivation for initiatives like Snowman.

How does Snowman benefit from early upgrade path support? Why can't they reinstall fresh sites like all other developers?

FWIW, this is a 1/0 decision for me. The effort of writing proper updates is only worth it, if we actually support production sites running on D8. Otherwise, the net effect, usage, and adoption rate will be close to zero (as with D7) for many reasons that have already been stated.

I do feel that we are slightly over-stating the required amount of work for writing and maintaining the updates. Sure, there are always edge-cases, but most of them were caused exactly by not caring for updates in the past, and in terms of overall volume of updates, I think it's safe to say that the easy ones easily outweigh the hard ones. Additionally, the head2head attempt in contrib was curated by very few people only, who were not necessarily involved in the original issues that caused the changes.

A properly written D8->D8 update should also work properly as D7->D8 update in most cases.

Lastly — this obviously depends on how much progress there will be on migrate-upgrades — but FWIW, I'd even be happy with supporting D8->D8 only - leaving the major upgrade path to migrate (if existent) or to the end of the release cycle. That would still leave the option to rewrite the updates at a pre-determined point in time (e.g., release an alpha100 that everyone on D8 needs to update to before updating to beta1).

Trying to reconcile #60 with #46, seems like one area of disagreement is the estimate of the burden of maintaining D8->D8 update functions during code thaw. Though even there, it seems like there's agreement that it's usually not a barrier: perhaps just a question about how to handle the edge cases where it is.

if we actually support production sites running on D8

As per #46.2, what does this mean? During code thaw, we can remove features and change APIs. During alphas, betas, and rcs, people report and discuss security bugs publicly.

Ok, so if what we're talking about is enabling personal blogs of Drupal developers, then what about David's suggestion in #46.1 about supporting 8.x-unstableN -> 8.x-unstableN+1 a guideline/goal, but not an absolute requirement? Again, anyone running an unstable ought to have different expectations of reliability than running a released version, so if there's one or two things that require manual intervention after an update, that doesn't seem so bad to me. Just like we have a guideline that patches that need tests should include them, but sometimes, they get committed without tests, which isn't ideal, but sometimes is justifiable.

webchick in #47.1 and sun in #60 are concerned about the guideline being too weak, so perhaps we can give it some teeth by saying that when we feel that it's better to get a patch committed timely rather than holding it up on a difficult update function, then after committing it without the update function, we leave the issue open and set priority to major.

Sounds mostly good to me. I'm mainly concerned with making sure that potentially-complex but relatively low-impact D8-upgrade-only issues like some of the ones I listed in #55 don't block patches from being committed (in the case where they're discovered before a patch is committed) and aren't given any sort of serious priority in the issue queue (in the case where they're discovered after). Basically, I think our threshold for what constitutes a "real" (or especially "major"/"critical") upgrade issue should be lower for D7->D8 than for an issue that only affects D8->D8, but the threshold can still be there.

If someone running their blog off of Drupal 8-dev comes across one of those and wants to deal with the upgrade problem in the issue queue (rather than just doing that manual fix for what is likely a very small amount of content on their own site) I guess they can feel free to... I just don't want anyone else to have to feel obligated to look at it also :)

I've read everyone's comments, and in the end I do not support this. I think head2head updates are good for beta or alpha periods, but not earlier.

The threshold for contributing core has gotten so high. Patch authors already have to deal with simpletest, D7 => D8 upgrade path, and painful issue queue consensus building. I think it is unwise to add another upgrade path to that tower of pain (even a minor one). I see that folks think that this won't affect many patches and that the issues here will be minor. Maybe so, but I also think the benefit is minor. A half dozen blogs running HEAD doesn't do all that much for us either, IMO.

We really don't need to discourage core contribution any further.

Actually using D8, in any capacity, may be the biggest incentive to contribute.

We really need to get to a consensus here, as we're currently removing even the most simple updates from all D8 patches that are backported to D7. Thus, D7->D7 gets fixed, but D8->D8 is not.

I can't really make sense of that at all.

Furthermore, the D7->D8 upgrade path is hardly tested by anyone. A very recent change only introduced code for the D7->D8 upgrade, whereas quickly adjusting that code into a simple update for D8->D8 revealed a critical bug.

That's sort of a prime example for what I already mentioned earlier: A D8->D8 update gets immediately tested by everyone working on core, whereas the D7->D8 update is not within the next ~2 years. There's almost no difference in terms of actual code, but the updates are actually executed in various environments shortly after commit. That's a big bang for the buck.

I'll see your "we should do this" and raise you "I volunteer to help testing the upgrade path throughout the D8 development cycle". All I need is some help to find the issues that need either hook_update_N functions written or need the upgrade path to be manually tested.

Can we flag all such issues with "upgrade path" or something? Call it whatever you want just let me know what it is and I'll prioritize those issues.

I'll also join cosmicdreams and help with testing the upgrade path going forward.

Well, it's not just about a commitment to testing the upgrade path. It's also about the additional burden this would place on people trying to get their patches into Drupal 8.

I was pretty gung-ho about this idea back in the day, but a lot less so now. Drupal 8's velocity has been struggling along, with still basically no real major features landing since DrupalCon Paris. Requiring people trying to add cool, crazy stuff to not only solve the upgrade path from 7 to 8 (an actual release blocker), but also from arbitrary points along Drupal 8's development cycle (which may or may not lead to more people testing D8 before release) means yet another enormous barrier of entry on top of our pre-existing pile of spikey, shark-laced barriers.

Take something like CMI. It's vitally important to get something there committed as soon as possible, because it's actively holding back lots of other work on D8. If we don't care for head2head upgrades, CMI folks have the capability to change course later on as they convert more subsystems and run into unforseen snags. For example, currently, the file format chosen is XML, but conceivably that could change again before release. Since D7 doesn't have a configuration management system, the upgrade path from D7 => D8 would remain the same; just call the API functions to convert stuff in the variables table to files on the file system, they're just a different type now. Except if they have to support head2head as well, now they need to adjust for the fact that sites will already have dozens of XML files laying about that need to be cleaned up and cared for. Something like that would add weeks onto the patch time, and make people a lot more skittish about committing anything in the first place until it was "perfect," and thus further stagnating the code base.

And in the end, I'm not confident at all that we'd gain more than a couple of dozen new test sites doing this—maybe—and of those, almost all would be from existing core contributors anyway. Is that worth further stagnating the progress of D8 and throwing more barriers in front of patch contributors? No, I really don't think it is.

If you're interested in helping with this, moreso than a commitment to testing the upgrade path, what we really need is an "head2head upgrade path clean-up crew" that offers to actually write and maintain the head2head upgrade path, either in core or in head2head module, to allow feature developers to keep trucking along making D8 awesome (note I did not say "D7 => D8 upgrade path clean-up crew." Been there, done that. Ugh.).

"head2head updates" doesn't automatically mean "perfect."

Contrary to that, not caring for updates at all automatically and inherently means that we'll have to adjust the announced D8 timeline and add a very long period of very painful upgrade path work.

Regardless of whether that's going to be based on a migrate approach or not, it will mean to revisit every single API changing issue to find out about any possible nitty-gritty changes that may have to be applied to stored data and configuration. And that, at a point in time, where it's guaranteed that no one, not even the original authors, will be able to remember all details and edge-cases of the change. (A migrate approach requires this, too - believing that one can spot all detailed changes, such as constraints, renames, and whatsoever, by visually comparing a D7 and D8 site/database would be a tad naive.)

"we'll have to adjust the announced D8 timeline and add a very long period of very painful upgrade path work."

That I don't get. The D7 => D8 upgrade path has test coverage, and that is being written as we go along.

Here is a very lively practical example from right now, so we don't need to talk theoretics at all :)

1. We removed 4 locale module update functions in November and moved them to update.inc (see above). Then we "recycled" those numbers and used them again.
2. Now @catch rolled back #1272840: Add upgrade path for language domains and validation which was at the end of the update function list. There are again two other RTBC issues wanting to add an update function to locale.install at the moment.
3. This sounds like we might end up with the same update function number used for 3 very different routines in just 4 months.

What should we do if an upgrade function was backed out? Should we reserve its number and not put in anything else? If people updated when it was backed out, they'd still need to run it later, so we cannot add it back with its previous (now reserved) number. So we add the function back with a bigger number later? Now that update function needs to be written to accommodate that it might have run, right? Do we write upgrade tests for the case for the original update function run and when not run separately?

Should we keep doing whatever and decide here later or is figuring this out *now* more important then making progress with changes in core *now*?

@catch points out over at #1272840: Add upgrade path for language domains and validation that recycling functions is a go inbetween UNSTABLE releases even in light of this discussion and since we have no UNSTABLE release yet, its a go still too.

(Also he underlined that this issue is about UNSTABLE to UNSTABLE releases despite the title. Retitling for that to make it clearer for people.)

So in a world where we're trying to support, for example, an upgrade path of UNSTABLE-1 to UNSTABLE-2, we may have an "end-of-sprint" push for upgrade path stability and quality. That sounds like a professional-grade standard to me. That makes each unstable release is own waterfall-based project with it's own QA round. Makes sense for a project this size.

The process could be like this:

1. Feature-Freeze UNSTABLE-N | Branch for UNSTABLE-(N+1) new feature work continues here
2. QA UNSTABLE-N with merging patches to UNSTABLE-(N+1) when necessary
3. Release UNSTABLE-N | Feature Freeze UNSTABLE(N+1) and send to QA | Branch for UNSTABLE (N+2) and continue to add features

The "Send to QA" and "Freeze Feature work" actions don't have to happen at the exact same time but should be close to each other.

If this workflow sounds familiar then you might be a fan of Google Chrome / Chromium http://en.wikipedia.org/wiki/Google_Chrome#Release_history

Independent fixes regardless of this issue: #1867972: Make HEAD2HEAD updates work

I've just filed #2014623: system_update_8057() failes when no action module enabled. to learn we do no head2head. I somehow expected always have hook_updates working while using 8.x-dev. Especially now we have 8.0-alpha1 ... not sure anymore. (subscribing)

Too late to make this change for D8. A good discussion to have for 9.x though.

Note that for this happen you need someone who is not chx to be your upgrade maintainer. As long as I am bearing that particular burden, this issue is an absolute nonstarter.

Also, no need to be assigned to Dries.