The import of new node types currently only works only because 'field.*' files are imported before 'node.*' files.
Scenario:
0) Dev & Prod environments, exact same state and config
1) Create a node type "foo' on Dev.
- This adds node.type.foo.yml.
- node_add_body_field() (called by NodeType::postSave($update = FALSE)) and _comment_body_field_create() (called by comment's hook_node_type_insert()) create instances of respectively 'body' and 'comment_body':
field.instance.node.foo.body.yml
field.instance.comment.comment_node_foo.comment_body.yml
each with fresh UUIDs.
2) Import all of this to Prod:
What happens in HEAD is:
- thanks to alphabet ordering, the field.* files are imported first. The 'foo' node type doesn't exist yet, but FieldInstance doesn't check that the bundle actually exists (was done of purpose for similar cases in the upgrade path) --> the instances are created.
- node.type.foo.yml is imported. This triggers both NodeType::postSave() & hook_node_type_insert()
We're lucky, both node_add_body_field() & _comment_body_field_create() are wise enough to check and do nothing if the instances already exist (I'm not even sure why they do it). All is good.
If import happened the other way around:
- node.type.foo.yml is imported, node_add_body_field() & _comment_body_field_create() create *new* instances with their own UUIDs
- field.* files are imported - Import crashes because the instances already exist with different UUIDs than in the files (the UUIDs that were generated on Dev).
So, we're lucky now, but this feels really brittle. If another entity type wants to implement similar "autopopulate new bundles an inititial set of field instances", and is provided by a module that starts with [a-e], this explodes...
Proposed solution
Conclusion from the "CMI hard issues" discussion in DC Prague:
- Add config_synching flag (like cron_running) -- ConfigEntityBase::isSynching(), plus a setter/getter, not exported
- It’s the responsibility of runtime code (MyConfigEntity::postSave(), hook_entity_save()...) to check the flag and not trigger separate config changes if the flag is present
- Also add a global state flag
- Check the state flag and throw an exception if a configuration entity is saved without the local sync flag when the global state flag is there.
| Comment | File | Size | Author |
|---|---|---|---|
| #28 | 2069373-28-test-only.patch | 677 bytes | swentel |
| #28 | 2069373-28.patch | 7.67 KB | swentel |
| #28 | interdiff.txt | 2.16 KB | swentel |
| #22 | 2069373-22.patch | 7.64 KB | swentel |
| #22 | interdiff.txt | 946 bytes | swentel |
Comments
Comment #1
yched commentedActually, this is more general than just fields, so re-titling & re-categorizing.
Possible approaches for use cases of "creation/update of ConfigEntity A triggers creation/update of ConfigEntity B"
- Limit this to UI operations
i.e. only auto-create 'body' field on node types created through the UI
- Have a way to somehow identify that a ConfigEntity is being imported
i.e. do not auto-create 'body' field on import of a node type - whatever config changes that should happen are supposed to be part of the import set already.
Comment #2
yched commentedbit shorter
Comment #2.0
yched commentedGeneralize
Comment #3
Anonymous (not verified) commentedi think i prefer the latter - we could certainly stick something on the entity in importCreate().
we can bikeshed the names later, but i support the idea.
Comment #4
yched commentedI can think of a couple other cases in core right now where we cleanup entries in entity B when entity A gets deleted. Didn't investigate whether this specific case also has its own set of race conditions, but this smells too.
"No cascade modifications of config entities during import, the import set is the only authority" sounds like a good practice, now that we don't support partial imports. Import means "the final result is supposed to be identical to the import set".
Ideally the import process would enforce this (same as UpdateModuleHandler enforces no hook is fired during updates), but I guess that's not really doable ? Import triggers ConfigEntity CUD APIs (and rightfully so...), we cannot really control what happens in there.
So short of that, yes, a flag on a ConfigEntity indicating it's being created/updated/deleted as part of import, and the code reacting to CUD needs to check the flag before acting on other ConfigEntities...
Comment #5
mtiftUpdating tags
Comment #6
yched commentedConclusion from the "CMI hard issues" discussion in DC Prague:
- Add config_synching flag (like cron_running) -- ConfigEntityBase::isSynching(), plus a setter/getter, not exported
- It’s the responsibility of runtime code (MyConfigEntity::postSave(), hook_entity_save()...) to check the flag and not trigger separate config changes if the flag is present
- Also add a global state flag
- Check the state flag and throw an exception if a configuration entity is saved without the local sync flag when the global state flag is there.
Comment #6.0
yched commentedmore accurate
Comment #7
yched commentedWondering:
It might be worth promoting that "is syncing" flag to a new param in ConfigEntity::preSave()/postSave() / hook_entity_save(), to make it more obvious that this case needs some special consideration.
Comment #8
swentel commentedInitial start - should have the same failures as in #2109705: Config import of a node type without body field auto-creates new body field.
Comment #10
mtift#8: 2069373-8.patch queued for re-testing.
Comment #12
Anonymous (not verified) commentedimport hooks run during module install (obviously, right?), which means that setSyncing(TRUE) stops default field creation for all the things, which is why so many random tests seem to fail after this change.
there's a patch over at #2095489: Separate out module install config code from import code to fix that.
Comment #13
swentel commentedRight makes sense. The body field for article and page are not created on install now, so getting the other one in will make a huge difference already.
Comment #14
yched commentedCool! There are (probably a lot) more places in core where we'll need to check for the isSyncing flag (including "delete instances when field is deleted" and reversedly, in Field / FieldInstance classes - we'd need #2020895: Move save() / delete() logic in Field / FieldInstance to [pre|post]Save(), [pre|post]Delete() in first), but this looks like a good start.
What do you guys think of #7 ?
"Do not try to trigger further config changes when reacting to an import" is going to be an important rule that contrib will need to understand and apply correctly, I'm trying to think of ways to have the APIs make the question more obvious.
Comment #15
Anonymous (not verified) commented#7 seems like a good idea to me, but i've had very little to do with fields though, so i don't think my vote means much :-)
Comment #16
yched commented@beejeebus: it's nothing about fields specifically, it's about adding an $is_syncing param (defaults to FALSE) to EntityInterface::preSave()/postSave()/preDelete()/postDelete() and hook_entity_save()/_delete()... so that implementors are aware that "is syncing" is an important aspect they need to accomodate.
However, that additional flag would make sense only for config entities, while those methods / hooks are defined for all entity types, so maybe that's not a good idea after all.
As mentioned in the summary of the Prague discussion in the OP, it was agreed to raise an exception to explicitely prevent writing config changes while processing the import of a config entity. When we do that, we kind of achieve that effect of raising awareness, since, well, bad code will break on import instead of silently succeeding and having weird effects...
Comment #17
alexpottSo here are all the config entities that are created automatically during standard profile install and enabling all the modules.
Comment #18
swentel commented#8: 2069373-8.patch queued for re-testing.
Comment #20
swentel commentedThis should at least run al tests.
Comment #22
swentel commentedShould be green
Comment #23
yched commentedMinor nits :
s/the config/the config entity/ ?
s/is/is being/ ?
(also, I think we commonly say "config entity" in core comments ?)
Why do we need to remove this ?
Comment #24
yched commentedOther than that, I really think we should enforce that with an exception as mentioned in the OP (otherwise it's just too easy to overlook). But the moment we do that, it also means we need to fix all the places in core that will start raising that exception, so that's possibly best left to a separate followup ?
Also, this is at least major.
Comment #25
swentel commentedRe: removal of 'create_body': because we don't use that property then anymore ? see snippet above ? Or do we want to keep that, it seems redundant.
Comment #26
yched commentedSure, I mean why does the code replace the existing check instead of simply adding a new condition ?
if ($this->get('create_body') && !$this->isSyncing()) {?Comment #27
swentel commentedHmm right, I see what you're getting at. Just because no code in core is explicitly setting that property to FALSE, it doesn't mean that it can be used to disallow creating a body field for a node type, for whatever reason. So yeah, we can keep the property. I was not thinking about that use case.
This needs tests too for syncing a node type without a body field I presume, I'll do that one tomorrow evening.
Comment #28
swentel commentedReverted the create_body property. Added test as well.
Comment #29
alanburke commentedWhile trying out the config system, I did note that the body field kept getting added to newly imported content types, even if they didn't have the body field in their definition.
This patch sorts that out - thanks.
Comment #30
Anonymous (not verified) commentedlooks good to me, test to prove it works, RTBC.
Comment #31
yched commentedAgreed, I was about to RTBC myself :-)
Opened #2124535: Prevent secondary configuration creates and deletes from breaking the ConfigImporter for the "raise exception" followup.
Comment #32
webchickNice!
Committed and pushed to 8.x. Thanks!
Comment #32.0
webchickproposed approach
Comment #34
xjm