[META] Requirements for tagging Drupal 9.0.0-alpha1

final 8.x LTS

Are you assuming more than one 8.x LTS? If so, has that been discussed anywhere?

I like the proposal of getting the latest 8.x development branch as ready as possible before we open 9.x.

I think there is a agreement that we want to get rid of simpletest.module, so we need to convert all web tests to browser tests before we open 9.x. That alone could take a while, probably 1 year?

OK, opened #2735005: Convert all Simpletest web tests to BrowserTestBase (or UnitTestBase/KernelTestBase).

I've been thinking about this quite a bit and I like this proposal. I think we should declare this to be our desire, but allow ourselves to change our mind (e.g. maybe when modules don't continuously upgrade to the latest APIs, making Drupal 9 a "big bang" release after all).

Is there an issue to roll back to routing system yet? Would be interesting to see a compilation of problems we have with it and what the alternative would be. Looking forward to https://events.drupal.org/dublin2016/sessions/routing-drupal-9-and-lesso... but the summary does not give any hints yet :(

Well, I guess that core conversation will be popular ;-)

@chx - can you clarify here (or in another issue) what it would mean to "remove" the router?

A lot of the code came from D7 in terms of actually doing it efficiently.

> @chx - can you clarify here (or in another issue) what it would mean to "remove" the router?

Remove all APIs and code depending on route names and for incoming, find the one possible route with a database query. I believe that is possible. For outgoing just use path where needed. Most of the time it's an entity link anyways.

I feel the biggest pre-requisite would be, do people have time.

A Drupal 9 branch would allow us to drop (or "reset") the templates, CSS, etc, in the Stable (and Classy?) themes.

The one unaddressed point of feedback so far is:

I feel the biggest pre-requisite would be, do people have time.

I don't think this is actually a concern -- we can keep releasing minors until the necessary work is complete. Honestly the work defined here is only a fraction of the work we did for D8's development. Given the new minor version cycle, the one risk associated with not releasing major branch is the degradation of performance and maintainability due to accumulated BC layers. But the thing is, to fix that and release 9.0.0, we still need to provide a working migration path, clean removal of deprecated code, etc. The main difference is that we do everything we can before opening the branch, rather than opening the branch and then letting things spiral out of control for the next two years after that.

Regarding the points in the summary:

• Timing -- I mostly agree, but I'd say "The earliest possible LTS of 8.x is 8.5.0" rather than 8.6.0 released before 9.x is opened. That means 3 more minors / 18 more months from now. Less than that is probably infeasible; longer than that in the future feels a bit magic eight ball. We can always shift the expectation later since it is "at least".
• Migration path -- I agree with all this.
• API changes -- Agree with all of 1-4. Not sure about point 5 though; I would not block opening 9.x on having RTBC BC breaks for non-deprecated code. Such patches/feature branches would be really difficult to maintain. I think such changes are ready when they're ready, and if they don't make 9.x, they don't make 9.x.
• Feature removals -- not sure about this one, for the same reason as with API changes point 5.
• Bugs -- absolutely agree. So far we have a good track record over the past year for fixing post-release criticals. We are about 25% of the way through triaging the major queue.
• Feature additions -- I think the first three points are all things that should be reviewed as part of the checklist for opening 9.x, but I'm not sure whether or not they should block it. I think we can also discuss these things as we get closer to that point. For the fourth point, I think we should actually do what Symfony does and avoid additions in 9.0.x entirely. They should go into 9.1.x.

At DrupalCon Dublin, the framework and release managers met with Nicolas Grekas (one of the Symfony core committers) to discuss how they handle BC and deprecations. (See #2575081-33: [policy, no patch] Use E_USER_DEPRECATED in Drupal 8 minor releases.) Adopting those practices will make a lot of the recommendations here much easier to implement, so I would in turn add adopting those practices as a prerequisite for opening 9.x. I would also add the explicit documentation indicated in #2550249: [meta] Document @internal APIs both explicitly in phpdoc and implicitly in d.o documentation since that is really entangled with what BC means and what we need to deprecate.

For me, this is close to RTBC, minus the few points above I'm unsure of.

Remove stable features

Should this be scoped down to just "stable features that have already been documented somewhere as deprecated"?

+1 to #36

Tagging for Drupal 9.

What exactly "opening Drupal9" means? Having the drupal9 branch created and actively maintained?

Because if that is the case, then based on all these requirements, we are already late to release Drupal 9 on time. Contrib needs to have a year to test against D9 before it is released.

I proposed a title update to make that clear :)

Created #3007166: [META] Stabilise and/or remove experimental modules as appropriate in/before Drupal 9 and will use it as a collection issue for experimental module stuff.

Added child issue for item 3 as well.

The only way to prove that #2093143: [meta] Remove calls to @deprecated and "backwards compatibility" procedural functions from core is outdated is to finish #2856742: [meta] Adopt trigger_error() for deprecation messages where it is missing :-)

$ grep -lr @trigger_error core/ | wc -l
     267
$ grep -lr @deprecated core/ | wc -l
     344

Added #3007300: [META] Release Drupal 9 on June 3 2020 as an interim parent issue, and made that a child of #2615502: [meta] Plan the core development and release process for 8.0.1 and beyond (that this issue used to be a child of). Since we are dealing with what needs to be done before Drupal 9 is branched, what needs to be done between then and Drupal 9 is released would be the realm of that issue. So that issue will be the master starting point for the work on Drupal 9.

Maybe contemplate cleaning up the components before the 9 branch: #1826054: [Meta] Expose Drupal Components outside of Drupal

Broke up 1 to 1 and 2, because they were two separate things.

Enhanced point 4 for with the [or] case of how version handling is supported.

Updated issue summary based on #3010983: Deprecate Drupal 6 and Drupal 7 migrations and move to contrib and #2607524-42: [meta] Migrations from Drupal 8 to Drupal 8 and Drupal 8 to Drupal 9.

@Mile23, #60:

That would be great to have before opening the D9 branch, but why would it be a pre-requisite? I had a look at the issue summary and it seems the missing steps are only additive, they don't seem to imply breaking changes. Are you suggesting this to make sure that we can expose 8.x components as well or am I missing something?

Added #3013276: [META] Remove deprecated modules on the Drupal 9 branch.

Retitle for more specific issue organization.

Moving back the experimental modules item since all that is to be done in 8.x.x and mainly before 9.x is branched.

Further reformatting of issue summary, made it an actual list.

Based on Slack convo w/ xjm and Gabor, adding #3031198: [META] Add classy_dev and minor-branch theme snapshots to allow fixing markup bugs within minor releases without front-end disruption (now marked a child of this) to the "should have" list.

whoops.

#3015625: [META] Prepare patches to be committed to the Drupal 9 branch once it is open is probably a "should-have" before we open the branch. We need to gather data, adopt a policy, and announce it soon.

Broken keyboard. #2917655: [9.4.x only] Drop official PHP 7.3 support in Drupal 9.4

Thanks! Fixed minor typos.

All deprecated code usages must be removed and deprecations properly documented

Why must the branch be delayed until all of these are done? Wouldn't the branch still be useful before then, in allowing the removal of the >90% of deprecations that are already properly documented and not used?

We should have a policy that allows fixing markup bugs without front-end disruption... Frankly, we should have that before 8.7.0 is released, but we definitely should have it before we open up the 9.0.x branch.

The last few words of that are inconsistent with this being listed in the "Prior to tagging Drupal 9 beta 1" section.

@catch and I also discussed and agreed back in July that if we get to October 11 (the weekend before the 8.8 alpha deadline and the normal point at which we create the new development branch) and the must-haves are close but not-quite-done, we will still open 8.9.x and 9.0.x at the same time. This will unblock the 9.0.x-only work without adding a lot of commit overhead or confusion about the purpose of the branch. However, we won't start tagging alphas until the first three points are complete.

Thanks @xjm for the update. I guess copy-paste issue here.

#2313917: Core version key in module's .info.yml doesn't respect core semantic versioning was committed which allowed closing #2807145: [policy, no patch] Allow contrib projects to specify multiple major core branches

These are the 2 issues under Must-haves prior to tagging Drupal 9.0.0-alpha1 item #3 in the issue summary 🎉

Just to point out that #3057420: [meta] How to deprecate Simpletest with minimal disruption continues apace. In that issue, we talked about deprecation of pieces of the testing infrastructure that would allow the simpletest module to be a contrib module.

In order for that to happen, someone needs to volunteer to be the maintainer of the contrib module. That person will not be me.

The contrib module has a project page, left over from the days of yore: https://www.drupal.org/project/simpletest

Retitling since this is about branch opening AND alpha requirements as those go somewhat hand in hand for first alpha (unless the date is met earlier).

Also moved the beta requirements to #3079680: [META] Requirements for tagging Drupal 9.0.0-beta1 and parented its respective issues to that.

So all the things that are part of the beta-stability requirements for 9.0.x as a release are _also_ requirements for beta. Including having all deprecated code removed, etc. So we might want to do a little further issue tree refactoring.

@catch and I also discussed this morning that bumping the PHP version minimum to 7.2 will also be a 9.0.0-alphaa requirement, as will removing deprecated dependencies (like jQuery UI components, removed polyfills, etc.) Alphas will indicate in bold text that not all deprecated APIs have been removed and that any use of remaining deprecated code _will_ cause fatal errors by beta1. And that beta1 must be tested, etc.

Needs an issue summary update for the changed alpha requirement then.

@catch, php7.2 is security only from nov. 2019 to nov. 2020, then EOL, why not bumping to php7.3, especially after reading tweets like this one https://twitter.com/SaraMG/status/1167593974737133569 ? I'm asking this question naively I don't know if there's rules in Drupal dev regarding php support.

@pounard better to discuss it in #2917655: [9.4.x only] Drop official PHP 7.3 support in Drupal 9.4
PS the thread is slightly related to https://ctankersley.com/2019/09/01/the-false-promise-of-lts-releases/

@andypost @catch thanks !

Adding #3072702: Core extensions should not need to specify the new core_version_requirement in *.info.yml files so that 9.0.x can be installed at the top of the list, which we need to fix even before PHP 7.2. ;)

Also, as 9.0.x itself is open, moving this issue to 9.0.x. (Issues with actual patches should stay against 8.9.x for now, even if they're 9.0.x-only changes, because DrupalCI doesn't work yet for 9.0.x.)

Also I don't think this is NW and the IS reflects the requirements AFAIK.

The branch is already open, so we are tracking alpha1 requirements here.

Updated some text to apply to the current scenario. Did not change any requirements.

Typo.

Adding #3087685: Remove deprecated jQuery UI components and fork remaining source code into core and #3070401: Install profiles do not support multiple core branch compatibility.

This meta is complete: #3057420: [meta] How to deprecate Simpletest with minimal disruption

It's unclear whether we're going to actually remove the simpletest module from D9 core, whether there will be a stub module (to allow an update path), or whether simpletest will remain in core as-is for D9.

If anyone wants to take the lead on that, I can help out, or if it's beyond all the necessary deadlines, then Oh Well. :-)

9.0.x can install and tests pass.

Step 3 (#3079791: Bump Drupal's minimum PHP version to 7.2 as soon as 9.0.x is branched (a higher version may be required later)) also completed!

I wonder if having an equivalent of :

• core/modules/system/tests/fixtures/update/drupal-8.bare.standard.php.gz
• core/modules/system/tests/fixtures/update/drupal-8.filled.standard.php.gz
• core/modules/system/tests/fixtures/update/drupal-8.6.0-minimal-with-warm-caches.sql.gz

Is necessary?

I wondered that too.

We have three deprecated polyfill libraries to remove from core in addition to forking jQuery UI; adding the meta to the list.

Is there an existing issue for a Composer update of 9.0.x core? It would be good to treat such an issue as a plan issue and spin out any components that introduces disruptive changes or especially things that are major version upgrades.

That also links to #2864037: [META] Update core PHP dependencies which is an ongoing issue that @Jibran keeps up with :)

Swapping in the real twig2 issue.

Symfony 4.4 done (for now) :)

#3016471: Make localization file download major version agnostic needs to be committed for ongoing compatibility with localize.drupal.org.

Should we add #3013276: [META] Remove deprecated modules on the Drupal 9 branch to IS?

@Gábor Hojtsy: Symfony 4.4 is a big milestone and the community worked hard to make this happen.

But what is the plan for Symfony 5? I know that this is still in development, but they target a release November 2019.

Would alpha or beta require Symfony 5? Or is Drupal 10 that is the target for Symfony 5?

#3064049: Replace jQuery UI sortable with Sortable js landed earlier today. #3087685: Remove deprecated jQuery UI components and fork remaining source code into core is currently being rerolled. AFAICT that is the last blocker 🤓🥳

We currently do not have plans to jump from Symfony 3 in Drupal 8 to Symfony 5 in Drupal 9, because it may make it harder for contrib modules to be compatible with both versions.

At some point not jumping to Symfony 5 will make it harder for contrib modules to support both Drupal 9 and Drupal 10. Symfony 6 will be released in two years which means Drupal 9 will fall at least two major versions behind upstream.

...there's also an issue here about changing our composer.json settings so that it's easier to switch between Symfony major versions.

Supporting both Symfony 4 (by default) and 5 (modifying composer.json) in Drupal 9 would be great along the run (in the future) in later minor version (e.g. Drupal 9.1 or 9.2).

At some point not jumping to Symfony 5 will make it harder for contrib modules to support both Drupal 9 and Drupal 10. Symfony 6 will be released in two years which means Drupal 9 will fall at least two major versions behind upstream.

You are thinking of the minor releases, not the LTS releases that we need to use. Symfony's non-LTS support cycles have been dropped to 8 months, while ours are 12 months, so we can't realistically use anything other than an LTS release. The current Symfony LTS release is 3.4, which Drupal is using. Then next LTS, 4.4 releases in December and the pre-release is already used in Drupal 9.

Symfony 5.4 LTS won't be out for 2 more years, and we will release Drupal 10 on it. Symfony 6.4 LTS is 4 years out.

@mikelutz
As I get it, the whole point of sticking to Symfony LTS releases is that we cannot update Symfony major version in Drupal minor release.
Because it is considered as a BC break. But is it really true? I know Drush had troubles (#2874827: Drush 8.x doesn't install Drupal 8.4.x and Drush master doesn't install Drupal 8.3.x) when Drupal 8 moved to Symfony 3 but nowadays both Drush and Drupal Console are installed as local Composer packages. So it shouldn't be a problem anymore.
Being able to update dependencies within release cycle would make Drupal more vendor independent and extend Drupal 9 lifetime. That's not only about Symfony but any other dependencies that have more frequent release cycle, i.e. PhpUnit.

The relevant semver discussion.
https://github.com/semver/semver/issues/148

Another option to consider would be dropping our non-LTS support cycles same way as Symfony did.

> Because it is considered as a BC break. But is it really true?

You're literally talking to the person that spent *days and weeks* to get Drupal 8 running on Symfony 4.4 as well as testing for Symfony 5. We've contributed upstream to fix problems and regressions and it is *hard* to make it work. It was practically impossible to get Drupal 8 to run on Symfony 4.3, only with some of our contributed changes and fixes were we able to make it work on 4.4. That's why Drupal 9 is now on a development snapshot of 4.4 and not 4.3.

So yes, it is very true, there are a lot of changes between Symfony major versions. Symfony 3 was an exception, we've communicated from the start (8.0) that we'll have to update and that some BC breaks will be unavoidable.

The current strategy is based on learnings over the last years to make this process work. Exactly to avoid that things like the Symfony 3 update. Drush and console still runs the Drupal code, so it needs to work with the Symfony version that Drupal ships with. Updating Drupal 9 to Symfony 4 broke Drush again and I think it was even questionable whether it will be possible at all to make Drush 9/10 even work with Drupal 9.

@Berdir That makes sense. Thanks.

questionable whether it will be possible at all to make Drush 9/10 even work with Drupal 9

Drush 10 does fully support Drupal 9 in its current state.

We need to get #3094146: Update Drupal 8 to the latest stable/compatibe Symfony 3.x versions done before tagging the alpha.

Now that Symfony 4.4.0 is out, we should block alpha on it.

#3074993: Use "current" in update URLs instead of CORE_COMPATIBILITY to retrieve all future updates is necessary for Drupal 9 to use updates.drupal.org.

There is no https://updates.drupal.org/release-history/drupal/9.x and https://updates.drupal.org/release-history/drupal/8.x does not contain 9.x releases.

Once we open up contrib to use semver, see #2681459: Support contrib semver releases, those will only be in https://updates.drupal.org/release-history/…/current.

Issue summary update. Some issues have been done.

Issue summary update. Drupal 9 now uses the stable Symfony 4.4 release.

Include Drupal in title for clarity. (Similar to beta issue).

Seems like only #3074993: Use "current" in update URLs instead of CORE_COMPATIBILITY to retrieve all future updates remains from the requirements, reviews welcome :)

Re #144, amazing that we're down to the final must-have!!

Which means, now is also a good time to re-evaluate our should-have list.

Why is #3075954: Remove duplicate scaffold files a should-have? It was added in #99. I don't really see any downside at all to it being done after alpha1, but a should-have implies that there'd be some downside to that, so what am I missing?

For #3091418: Update composer dependencies on 9.0.x following PHP 7.2 requirement, I think that's an accurate should-have, meaning that it would definitely be very nice for alpha1 to reflect up-to-date dependencies, but that we could also release alpha1 without that if needed and get those updates into later alphas. However, if that issue isn't easy to solve in one shot, would it make sense to split it into ones that are easy, and at least get those into alpha1, and then let the harder ones come later? Also, it might be nice to try to get the ones that need a major version update (e.g., phpunit from 6 to 7) done before alpha1, and let the libraries that only need a minor version update come in a later alpha.

So #3075954: Remove duplicate scaffold files is also listed as a should have in the beta requirements, so I am not sure if that was intentionally added here. How can it be both a should have of alpha and then also beta?

Adding #3099876: Establish a packaging pipeline queue, because getting 9.0.x releases in the mix doubles the number of releases we have to coordinate with infra and build by hand.

So #3079680: [META] Requirements for tagging Drupal 9.0.0-beta1 is there because back in October in a meeting between core committers and the DA engineering team, the DA indicated the change was important and asked how to track the issue relative to 9.0.0's alpha release. It's not a must-have, so it does not block when or whether the alpha gets released. And it's listed as a should-have for both the alpha and beta because logically speaking all should-haves of the alpha are either should- or must-haves of the beta.

However, the issue currently does not indicate what the implications of not making the change are, so I'll follow up over there.

One of the should-haves was marked as a duplicate of a different issue, so updating with the correct link.

Dunno whether it's for alpha or beta, but IMHO #3093632: Contrib metapackages cannot be installed in D9 should be addressed - contrib modules depending on sub-modules in other contrib modules cannot be installed (and therefore tested) in 9.0.x ATM.

Removing #3075954: Remove duplicate scaffold files based on discussion on that issue.

Composer dependencies got updated :) One more should-have down.

Also the Symfony update was done.

Doctrine is done. 8.9 deprecations policy was announced.

PHP version is now settled, so the only remaining ones are #3099876: Establish a packaging pipeline queue on the infrastructure side and #3074993: Use "current" in update URLs instead of CORE_COMPATIBILITY to retrieve all future updates on the core side, the two of which should make us alpha-ready AFAIS.

#3074993: Use "current" in update URLs instead of CORE_COMPATIBILITY to retrieve all future updates is done.

Tagging that this must be completed before 9.0.0-beta1 can be completed, just to cross-reference it all.

My understanding is this is currently on hold for #3108115: Update packaging to support packaging releases from the local copy of subtree splits, instead of Packagist or a workaround to be found for it.

Adding #3086644: LegacyProject composer templates wrongly reference 8.x + fix test coverage as per @mixologic.

Turns out we cant release 9.x until we have some composer templates that can actually, use 9.x.

Out now! https://twitter.com/DropIsMoving/status/1227312415714533381?s=20

Crediting @xjm, @Mixologic, @drumm, @hestenet for doing the release and coordination.

If we're crediting the folks for release coordination (yay, thanks!), doesn't it also make sense to credit everyone who contributed to this issue, updated lists, triaged, etc? Just blanket credit everyone who participated here?

Cheers,
-Derek

@dww: this issue has a lot of history, especially in the first few years of it, it was a policy issue discussing details and people popping in to ask questions. It felt most appropriate to me to not try to piece that apart. I credited the four people who were actually sitting live on our video meeting yesterday for two hours debugging problems and testing pieces while doing the release. (I did add @hestenet additionally as he was doing super detailed daily reporting and coordination on the DA side in the past weeks to prepare for the alpha and that was not reflected anywhere).

I did not mean this to devalue the contribution of people who helped discuss and tend to the alpha plan. If this makes crediting uneven, I am happy to remove all credits from this issue.

I also added credit for @catch since he also spent hours helping define the alpha requirements, but I agree with the choice in #167. Individual alpha blockers are already credited on their individual issues.

Thanks!

Definitely agreed with adding @catch. That was the big obvious item in the list that was missing a checkbox. ;) Totally fine leaving the rest of us off.

Cheers,
-Derek