Tagging a core release with Composer 2.0.0-alpha3 fails spectacularly [#3163890]

Problem/Motivation

Drupal 10, the Automatic Updates Initiative, etc. will require Composer 2. Composer 2.0.0-alpha1 was released awhile ago and 2.0.0-alpha3 has just been released this week.

Yesterday I accidentally tried to tag the core bugfix releases using the Composer 2.0.0 alpha. It failed dramatically.

Steps to reproduce

These manually follow the automated steps used when creating a core release.

composer self-update 2.0.0-alpha3
Fetch the latest 9.0.x HEAD.
rm -rf vendor (to remove stale vendor created with Composer 1)
composer install --no-progress --no-suggest -n -q

Set the version constant in core/lib/Drupal.php

diff --git a/core/lib/Drupal.php b/core/lib/Drupal.php
index 5f9890ed33..098925e936 100644
--- a/core/lib/Drupal.php
+++ b/core/lib/Drupal.php
@@ -80,7 +80,7 @@ class Drupal {
   /**
    * The current system version.
    */
-  const VERSION = '9.0.4-dev';
+  const VERSION = '9.0.4';

COMPOSER_ROOT_VERSION="9.0.4" composer update drupal/core*

Expected result: Composer updates the metapackages as appropriate for the 9.0.4 release.

Actual result: Composer outputs the following:

$ COMPOSER_ROOT_VERSION="9.0.4" composer update drupal/core*
> Drupal\Composer\Composer::ensureComposerVersion
Loading composer repositories with package information
Updating dependencies
Lock file operations: 0 installs, 3 updates, 0 removals
  - Downgrading drupal/core (9.0.x-dev => 9.0.4)
  - Downgrading drupal/core-project-message (9.0.x-dev => 9.0.4)
  - Downgrading drupal/core-vendor-hardening (9.0.x-dev => 9.0.4)
Writing lock file
Installing dependencies from lock file (including require-dev)
Package operations: 0 installs, 3 updates, 0 removals
  - Downgrading drupal/core-project-message (9.0.x-dev => 9.0.4): Symlinking from composer/Plugin/ProjectMessage
  - Downgrading drupal/core-vendor-hardening (9.0.x-dev => 9.0.4): Symlinking from composer/Plugin/VendorHardening
  - Downgrading drupal/core (9.0.x-dev => 9.0.4): Source already present
Generating autoload files
Could not scan for classes inside "core/lib/Drupal.php" which does not appear to be a file nor a folder
Could not scan for classes inside "core/lib/Drupal/Component/DependencyInjection/Container.php" which does not appear to be a file nor a folder
Could not scan for classes inside "core/lib/Drupal/Component/DependencyInjection/PhpArrayContainer.php" which does not appear to be a file nor a folder
Could not scan for classes inside "core/lib/Drupal/Component/FileCache/FileCacheFactory.php" which does not appear to be a file nor a folder
Could not scan for classes inside "core/lib/Drupal/Component/Utility/Timer.php" which does not appear to be a file nor a folder
Could not scan for classes inside "core/lib/Drupal/Component/Utility/Unicode.php" which does not appear to be a file nor a folder
Could not scan for classes inside "core/lib/Drupal/Core/Cache/Cache.php" which does not appear to be a file nor a folder
Could not scan for classes inside "core/lib/Drupal/Core/Cache/CacheBackendInterface.php" which does not appear to be a file nor a folder
Could not scan for classes inside "core/lib/Drupal/Core/Cache/CacheTagsChecksumInterface.php" which does not appear to be a file nor a folder
Could not scan for classes inside "core/lib/Drupal/Core/Cache/CacheTagsChecksumTrait.php" which does not appear to be a file nor a folder
Could not scan for classes inside "core/lib/Drupal/Core/Cache/CacheTagsInvalidatorInterface.php" which does not appear to be a file nor a folder
Could not scan for classes inside "core/lib/Drupal/Core/Cache/DatabaseBackend.php" which does not appear to be a file nor a folder
Could not scan for classes inside "core/lib/Drupal/Core/Cache/DatabaseCacheTagsChecksum.php" which does not appear to be a file nor a folder
Could not scan for classes inside "core/lib/Drupal/Core/Database/Connection.php" which does not appear to be a file nor a folder
Could not scan for classes inside "core/lib/Drupal/Core/Database/Database.php" which does not appear to be a file nor a folder
Could not scan for classes inside "core/lib/Drupal/Core/Database/Driver/mysql/Connection.php" which does not appear to be a file nor a folder
Could not scan for classes inside "core/lib/Drupal/Core/Database/Driver/pgsql/Connection.php" which does not appear to be a file nor a folder
Could not scan for classes inside "core/lib/Drupal/Core/Database/Driver/sqlite/Connection.php" which does not appear to be a file nor a folder
Could not scan for classes inside "core/lib/Drupal/Core/Database/Statement.php" which does not appear to be a file nor a folder
Could not scan for classes inside "core/lib/Drupal/Core/Database/StatementInterface.php" which does not appear to be a file nor a folder
Could not scan for classes inside "core/lib/Drupal/Core/DependencyInjection/Container.php" which does not appear to be a file nor a folder
Could not scan for classes inside "core/lib/Drupal/Core/DrupalKernel.php" which does not appear to be a file nor a folder
Could not scan for classes inside "core/lib/Drupal/Core/DrupalKernelInterface.php" which does not appear to be a file nor a folder
Could not scan for classes inside "core/lib/Drupal/Core/Installer/InstallerRedirectTrait.php" which does not appear to be a file nor a folder
Could not scan for classes inside "core/lib/Drupal/Core/Site/Settings.php" which does not appear to be a file nor a folder
Class Drupal\Core\Composer\Composer is not autoloadable, can not call pre-autoload-dump script

                                                                               
  [RuntimeException]                                                           
  Could not scan for classes inside "core/lib/Drupal.php" which does not appe  
  ar to be a file nor a folder                                                 
                                                                               

update [--with WITH] [--prefer-source] [--prefer-dist] [--dry-run] [--dev] [--no-dev] [--lock] [--no-install] [--no-autoloader] [--no-scripts] [--no-suggest] [--no-progress] [--with-dependencies] [--with-all-dependencies] [-v|vv|vvv|--verbose] [-o|--optimize-autoloader] [-a|--classmap-authoritative] [--apcu-autoloader] [--ignore-platform-req IGNORE-PLATFORM-REQ] [--ignore-platform-reqs] [--prefer-stable] [--prefer-lowest] [-i|--interactive] [--root-reqs] [--] [<packages>]...

... and also deletes the entire contents of the core directory.

There is no workaround that I could find.

Proposed resolution

Remaining tasks

User interface changes

N/A

API changes

Data model changes

Release notes snippet

TBD

Comments

Comment #1

6 August 2020 at 20:51

xjm created an issue. See original summary.

Comment #2

xjm

she/her

English

CreditAttribution: xjm at Acquia commented 6 August 2020 at 21:54

Issue summary:

View changes

After discussing with @greg.1.anderson I realized it's "only" core/*, but there are also no workarounds for tagging. It happens even with freshly built vendor, correctly specifying COMPOSER_ROOT_VERSION, etc.

Comment #3

xjm

she/her

English

CreditAttribution: xjm at Acquia commented 6 August 2020 at 21:58

I did also test with #3159967: Allow Drupal 9 (including dev builds) to use Composer 2, part 2 committed locally (in case it was removing core because it was considered incompatible with Composer 2, or whatever) but the problem still occurred.

Comment #4

Mixologic

He/Him/His

English

Portland, OR

CreditAttribution: Mixologic at Drupal Association commented 9 August 2020 at 17:19

Slack thread to preserve.

mixologic:drupalcon: 8 minutes ago
I’ve been digging on this in the debugger. So far its a confluence of composer-installers + path repos.

mixologic:drupalcon: 7 minutes ago
During the update, composer gathers all the local filesystem path information.

mixologic:drupalcon: 6 minutes ago
https://github.com/composer/composer/blob/83c64a9d191b64689b408906d40943... Is where it checks to make sure that the source data doesnt already exist in the place where we want to download it to.

src/Composer/Downloader/PathDownloader.php:181
$realUrl = realpath($package->getDistUrl());
composer/composer | Added by GitHub

mixologic:drupalcon: 6 minutes ago
Actually here: https://github.com/composer/composer/blob/83c64a9d191b64689b408906d40943...

src/Composer/Downloader/PathDownloader.php:180-189
{
$realUrl = realpath($package->getDistUrl());

if ($path === $realUrl) {
if ($output) {
$this->io->writeError(" - " . UninstallOperation::format($package).", source is still present in $path");
}

return;
}
Show less
composer/composer | Added by GitHub

mixologic:drupalcon: 5 minutes ago
So, the problem is that composer installers determines the $path based off of the type, in this case drupal-core and sets that value to core

mixologic:drupalcon: 4 minutes ago
but the $realUrl line ends up setting it to the fully qualified path on the filesystem, or /Users/ryan/Documents/Work/Current/DrupalAssociation/drupal_core/drupal_8/core in my case.

mixologic:drupalcon: < 1 minute ago
and since 'core' != '/Users/ryan/Documents/Work/Current/DrupalAssociation/drupal_core/drupal_8/core' it removes the path repo.

Comment #5

Mixologic

He/Him/His

English

Portland, OR

CreditAttribution: Mixologic at Drupal Association commented 11 August 2020 at 12:32

Status:

Active

» Fixed

Upstream pull request was accepted, which fixes this issue:
https://github.com/composer/composer/pull/9116

Try with composer self-update --snapshot and I get

➜  drupal_8 git:(9.0.x) ✗ COMPOSER_ROOT_VERSION="9.0.4" composer update "drupal/core*"
> Drupal\Composer\Composer::ensureComposerVersion
Loading composer repositories with package information
Updating dependencies
Lock file operations: 0 installs, 3 updates, 0 removals
  - Downgrading drupal/core (9.0.x-dev => 9.0.4)
  - Downgrading drupal/core-project-message (9.0.x-dev => 9.0.4)
  - Downgrading drupal/core-vendor-hardening (9.0.x-dev => 9.0.4)
Writing lock file
Installing dependencies from lock file (including require-dev)
Package operations: 0 installs, 3 updates, 0 removals
  - Downgrading drupal/core-project-message (9.0.x-dev => 9.0.4): Source already present
  - Downgrading drupal/core-vendor-hardening (9.0.x-dev => 9.0.4): Source already present
  - Downgrading drupal/core (9.0.x-dev => 9.0.4): Source already present
Generating autoload files
> Drupal\Core\Composer\Composer::preAutoloadDump
Hardening vendor directory with .htaccess and web.config files.
41 packages you are using are looking for funding.
Use the `composer fund` command to find out more!
Cleaning vendor directory.
> Drupal\Composer\Composer::generateMetapackages
Updated metapackage file composer/Metapackage/CoreRecommended/composer.json.
Updated metapackage file composer/Metapackage/PinnedDevDependencies/composer.json.
If you make a patch, ensure that the files above are included.

Comment #6

11 August 2020 at 20:09

xjm credited greg.1.anderson.

Comment #7

xjm

she/her

English

CreditAttribution: xjm at Acquia commented 11 August 2020 at 20:09

When I run that I get:

[ayrton:maintainer | Tue 15:08:25] $ composer self-update snapshot
Updating to version snapshot (stable channel).

                                                                               
  [Composer\Downloader\TransportException]                                     
  The "https://getcomposer.org/download/snapshot/composer.phar.sig" file coul  
  d not be downloaded (HTTP/1.1 404 Not Found)                                 
                                                                               

self-update [-r|--rollback] [--clean-backups] [--no-progress] [--update-keys] [--stable] [--preview] [--snapshot] [--1] [--2] [--set-channel-only] [--] [<version>]

Any ideas? =/

Comment #8

xjm

she/her

English

CreditAttribution: xjm at Acquia commented 11 August 2020 at 20:14

Fixed by typing the correct thing:

ayrton:maintainer | Tue 15:08:25] $ composer self-update --snapshot

Resulted in:

[ayrton:maintainer | Tue 15:11:08] $ composer --version
Composer version 2.0-dev (2.0-dev+c0eb9834fef009316b0146763f9440ddd844169d) 2020-08-11 07:54:09

I was able to create a release using the snapshot! Hooray.