[meta] Update JavaScript dependencies for Drupal 10

I filed #3262573: Update our yarn dev dependencies to the extent allowed by current constraints which covers the first step of the scope here and fixes two of the three "high" severity vulns from 10.0.x through 9.3.x as allowed under the current constraints. Maybe worth postponing this issue on that going into the respective branches?

@lauriii and I discussed what to do next after #3262573: Update our yarn dev dependencies to the extent allowed by current constraints, and he suggested it would be best to handle some of the dependencies individually, particularly where there are security updates that require changing constraints. He will file issues for those.

Also see #3238507: Drupal 10 JavaScript dependency plan.

Added child issues to the issue summary and made this a meta issue.

On top of that, to get empty yarn audit output, CKEditor 5 team will have to resolve https://github.com/ckeditor/ckeditor5-dev/pull/695.

#3262573: Update our yarn dev dependencies to the extent allowed by current constraints has been committed.

Technically beta1 is the deadline here, although it would be good to get the major updates especially in sooner.

Well, let's just say this.

Moved issues that are not required for cleaning yarn audit to #3238507: Drupal 10 JavaScript dependency plan.

For accessibility, please spell out acronyms on first use.

We may want to wait until https://github.com/shipshapecode/shepherd/pull/2037 has landed before we update Shepherd.js.

Let's not wait. I have no idea when it'll make it and the change to 10.0 doesn't impact us so it's safe to update

Thanks @nod_! Opened an issue for that.

Updated IS with the current state of things.

I tried to upgrade ESLint but there is something wrong with my local setup, yarn lint:core-js-passing never finishes even if I leave it for 30 mins or more.

maybe we should group all the easy ones into one patch so that we don't have to reroll all the time because of yarn.lock conflicts?

Re #39, upgrading ESLint worked fine on my machine, and after doing so, yarn lint:core-js-passing completed in less than a minute. I opened #3319819: Update eslint to 8.27 with the resulting patch. Currently, that's the last minor-level one. Following that, here's what's left as of today; these are all patch-level:

yarn outdated:

cspell             6.14.0  6.14.1                                   
nightwatch         2.4.1   2.4.2                                                                    
postcss            8.4.16  8.4.18                                                                        
postcss-header     3.0.2   3.0.3                                      
postcss-preset-env 7.8.1   7.8.2
tabbable           6.0.0   6.0.1 

added #3319917: Remove raw-loader dependency so that we have a clean yarn audit.

#3317887: Update PostCSS, postcss-header, and postcss-preset-env landed in 10.0.x, which is the last of the current child issues. However, yarn outdated now shows:

cspell     6.14.0  6.14.1
nightwatch 2.4.1   2.4.2     
tabbable   6.0.0   6.0.1
webpack    5.74.0  5.75.0

I'd suggest opening two new child issues: one for webpack, since that's a minor version update, and one for the other 3, which are all patch-level, unless folks want to suggest a different approach.

Also, what's the best way to find out if we're up-to-date on all our other JS dependencies (the ones not managed by yarn)?

I think we could open one issue for all of the remaining ones. The webpack minor update should be straight forward, but if turns out it isn't, it could be split to its own issue.

Also, what's the best way to find out if we're up-to-date on all our other JS dependencies (the ones not managed by yarn)?

Do we have any JavaScript dependencies not maintained in yarn? I think all of the dependencies are managed by yarn since #3219088: Use package.json to manage third party JS libraries.

After migrating CI to Gitlab core could use https://docs.gitlab.com/ee/user/application_security/dependency_scanning/

Someone already opened #3320370: Update to tabbable v6.0.1 to fix bug with scoped headers

Filed
- #3320515: Update cspell and nightwatch
- #3320518: Update webpack

Removing rc blocker tag and adding stable release blocker, we should do another round of minor/patch level updates before release, but there is little point chasing any more releases before rc1.

As of today, yarn outdated shows the following:

Package            Current Wanted  Latest  Package Type
cspell             6.14.1  6.14.3  6.14.3  devDependencies
eslint             8.27.0  8.28.0  8.28.0  devDependencies                                                              
jsdom              20.0.2  20.0.3  20.0.3  devDependencies      
nightwatch         2.4.2   2.5.1   2.5.1   devDependencies
postcss-preset-env 7.8.2   7.8.3   7.8.3   devDependencies
prettier           2.7.1   2.8.0   2.8.0   devDependencies                               
shepherd.js        10.0.1  10.0.1  11.0.0  devDependencies                                                 
stylelint          14.14.1 14.15.0 14.15.0 devDependencies                                            
webpack-cli        4.10.0  4.10.0  5.0.0   devDependencies

In particular, shepherd.js and webpack-cli have new major releases, so I think we should open either one or two child issues to upgrade those. If it's easier, maybe we should upgrade all of the above in one issue/patch?

shepherd.js has a big breaking change: the underlying library has moved from popperjs to floating-ui; not sure if this is too big a change for us to make now we are in release candidate or not?

The others (including webpack-cli, given that the break is removing support for Webpack 4 and we are already on v5) can likely be done in a single issue.

This patch updates everything except shepherd.js. I ran yarn build, yarn lint:css and yarn lint:core-js-passing and there were no changes and no errors.

Either we were very unlucky or the nightwatch update is breaking all but one nighwatch tests by not being able to connect to the selenium server.

Ordered a retest to be sure what's going on.

Same fails in Nightwatch tests.

This seems the likely culprit to me:

-    selenium-webdriver "4.3.1"
+    selenium-webdriver "4.6.1"

Maybe we should split off the Nightwatch update into a separate child issue and get the other ones in whilst we investigate?

Possibly related, see https://github.com/SeleniumHQ/docker-selenium/issues/1723. This caused FunctionalJavascript tests to fail for me and at least a handful of others. Also affects Drupal Test Traits. Pinning to selenium/standalone-chrome:4.5.2 resolved the issue. I suspect it's the same issue here.

Opened child issue #3323834: (Try to) update Sheperd.js to latest major version

Opened child issue: #3323944: Update JavaScript dependencies for Drupal 10, except Shepherd.js and Nightwatch

Also, put this issue back to Active since there's no valid patch/MR to review any more (and this is a [META]).

Opened child issue #3323988: Update Nightwatch from 2.4.2 to 2.6.19

As of today, yarn outdated on 10.0.x outputs:

Package        Current Wanted Latest Package Type  
nightwatch     2.4.2   2.5.3  2.5.3  devDependencies     
postcss-import 15.0.0  15.0.1 15.0.1 devDependencies 
shepherd.js    10.0.1  10.0.1 11.0.0 devDependencies 

For nightwatch and shepherd.js, I tagged #3323988: Update Nightwatch from 2.4.2 to 2.6.19 and #3323834: (Try to) update Sheperd.js to latest major version themselves as stable release blockers.

It might be nice to get postcss-import updated prior to tagging 10.0.0-rc2, though probably not a big deal if we don't. Either way, we'll still need to check again just prior to the 10.0.0 release.

It might be nice to get postcss-import updated prior to tagging 10.0.0-rc2, though probably not a big deal if we don't. Either way, we'll still need to check again just prior to the 10.0.0 release.

Well, it has to go in anyway, so let's do it now, so RC2 is as up to date as possible: #3324927: Update to the latest postcss-import version (15.0.1)

I know this is an endless chase, but let's try to get RC2 (is that even an official thing yet?) out as up-to-date as possible, and do a final(?) update round on Monday Dec. 12 to get the 10.0.0 release up-to date.

#3325114: Update to the latest cspell, eslint, postcss-import, stylelint. terser and webpack-cli version

As of today, yarn outdated outputs:

Package           Current Wanted Latest                                       
eslint-plugin-yml 1.2.0   1.3.0  1.3.0
nightwatch        2.4.2   2.5.3  2.5.3          
postcss-import    15.0.1  15.1.0 15.1.0
prettier          2.8.0   2.8.1  2.8.1                    
shepherd.js       10.0.1  10.0.1 11.0.0

Upgrading the major version of Shepherd was postponed to Drupal 11 per #3323834-15: (Try to) update Sheperd.js to latest major version.

Upgrading the minor version of Nightwatch is likely postponed to Drupal 10.1 per #3323988-12: Update Nightwatch from 2.4.2 to 2.6.19, considering we don't yet know how to make the new version work on DrupalCI.

If there was a compelling need to update eslint-plugin-yml, postcss-import, or prettier before releasing 10.0.0, we potentially could do those, but if there isn't a compelling need (I'm not aware of one), then I think we're better off not making any further changes to our dependencies in the 10.0.x branch in preparation for a stable release. Therefore, marking this issue Fixed, and let's open a new one for 10.1 when there's a benefit to updating the JS dependencies in that branch.

Saving credits for contributors who helped keep this plan up to date.