Think of this as sequel to #211182: Updates run in unpredictable order.
I've encountered this in D7, but the code doesn't look much different in D8, so I'm assuming it's just as problematic.

Current situation:
- Update order doesn't respect module weight
- We have hook_update_dependencies() as the only mechanism of controlling update order.

Installation profiles have a weight of 1000 in D7's system table.
But since that is ignored, the distribution updates end up running in the middle of other module updates.
(Seems to be alphabetical. Panopoly's "P" is late enough that they rarely see it, but Commerce Kickstart's "C" is very early...)
Now, most distribution updates enable or disable modules, triggering cache rebuilds,
which causes errors from any module that doesn't have an expected schema update applied yet.
For example, Rules is adding an "owner" column, but before that update runs, a Kickstart update enables a module,
triggers cache rebuilding, and Rules dies because it runs a query against the table without the "owner column".

This means that a distribution needs to declare update dependencies on any potentially problematic contrib updates (== all of them).
So I consider it a DX fail that Kickstart and Panopoly need to add hacks instead of being able to assume that their updates run in an "updated" environment.

I am guessing that the update hooks aren't ordered by module weight first for a reason, so maybe we can at least make the installation profile updates
run last?

Comments

bojanz’s picture

Issue summary: View changes
bojanz’s picture

Issue summary: View changes
alexpott’s picture

Since distributions are a very very special type of module that in my mind was a mistake to make a module. Since only one can be installed and they can only be installed during an installation. Therefore additional special casing seems the only way to go.

bojanz’s picture

Issue summary: View changes

Travis tests show that adding a dependency on a contrib update for the first distribution update will make that dependency valid even if some other distribution update runs first (so commerce_kickstart_update_7200 depends on rules_7201, but commerce_kickstart_update_7209 is the first one that needs to be run by update.php, then 7209 will still run after the rules one).

Updated the summary to reflect this.

Thanks for the input Alex, that's the road we'll take then (making sure that the distro updates run last).

dsnopek’s picture

Actually, I think this is more complicated than just making distribution update functions run last.

For example, we (Panopoly) have been having endless problems with a pathauto update that adds a new table. Since pathauto will attempt to query this table when loading an entity, if any other update functions attempt to load an entity before pathauto has had a chance to add it's table, it will error out.

We've been adding hook_update_dependencies() hacks when these types of problems come up. However, the order of module updates is super delicate, and even adding an entirely unrelated update function to an unrelated module will cause the updates to reorder in such that this issue comes back.

Here's an example build on Travis-CI:
https://travis-ci.org/panopoly/panopoly/jobs/23551978

This commit added an update function to panopoly_images which sets a variable. It doesn't do anything with pathauto or the module whose update is breaking things. But it mearly existing has changed the order of update function in an unpredictable way.

You could say that pathauto is to blame, and it should be further patched to work around this problem. However, this problem isn't unique! We've seen it with other modules that, for example, add a new cache table that didn't exist previously. If some other module's update function does something that could trigger reading/writing from that cache table, it'll break. But that other module may not have any way of knowing that the module with the new cache table is hooking into the operation they are performing!

But if constantly checking for the existance of new tables (probably applies to new columns too) is the best practice, I guess we should document that clearly somewhere and start making issues in contrib modules.

However, it be super awesome to fix in core!

To do that we'd need to:

  • Make the order of update functions more predictable. Adding an unrelated update function which has no dependencies shouldn't mess with the order of other update functions. I like the idea of including module weight, but I don't know if that'd be enough in practice.
  • Be able to force certain updates (like those that create tables) happen earlier in the update cycle. It'd be awesome to say that pathauto update has to happen before everything. Currently, I can only say that other updates depend on it. I'd love to say that ALL update functions depend on it. :-)

What do you think?

catch’s picture

dawehner’s picture

I actually think that distributions will probably not use hook_update_N() but rather hook_post_update_NAME()
which by construction has a really clear order of execution. Top from the file to bottom.

See #2538108: Add hook_post_update_NAME() for data value updates to reliably run after data format updates for that issue ...

dsnopek’s picture

This isn't really about the update hooks that distributions themselves provide, ie. in profile.install, but the order of all the install hooks provided by contrib. I'm really not sure how this will play out in Drupal 8 - I'm not even sure how to create a test case there, but this is a persistent problem in Drupal 7: we get support issues in Panopoly with almost every release, because installing even a single additional module (that we don't bundle in Panopoly) has the potential to unpredictably change the order of the update hooks.

DamienMcKenna’s picture

Why is this a separate issue and not just part of #211182: Updates run in unpredictable order?

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

catch’s picture

Status: Active » Closed (duplicate)
Issue tags: +Needs issue summary update

I'm marking this duplicate of #211182: Updates run in unpredictable order since there are no steps to reproduce here, and it's not clear what's different about distributions to regular modules in terms of update ordering.