Problem/Motivation
In #2878556: Ensure that changes to untranslatable fields affect only one translation in pending revisions we introduced a new validation logic (coupled with complementary UI changes) that ensures that no changes to untranslatable fields can be made in a way that would cause multiple revision translations to be marked as affected in a single pending revision.
In #2878556-16: Ensure that changes to untranslatable fields affect only one translation in pending revisions it was brought up that translatable fields having field property synchronization enabled may run into the same issues, because if an untranslatable field property is changed in any revision translation, the same changes are propagated to all the other translations on presave. This causes all revision translations to be marked as affected, which is exactly what the parent issue aimed to prevent.
Proposed resolution
@effulgentsia's in #5:
For purposes of unblocking #2860097: Ensure that content translations can be moderated independently, what if all we did was still show the full widget (with both its translatable and untranslatable properties/elements), but we add a validation error if the translator actually tries to change an untranslatable element?
[…] it would prevent getting into the data integrity problem reported in #2860097-153: Ensure that content translations can be moderated independently.
@plach implemented this in #6.
Remaining tasks
Find a viable solutionA validatorWrite a patchSee #6- Reviews
User interface changes
A new validation error to prevent data integrity problems, only in those cases where there would otherwise have been data integrity problems.
API changes
None.
Data model changes
None.
| Comment | File | Size | Author |
|---|---|---|---|
| #48 | ct-field_sync_drafts-2940204-48.patch | 46.43 KB | plach |
Comments
Comment #2
plachI think that ideally we would like the UI to behave exactly as it does for untranslatable fields: in the default mode we would prevent changes to fields having field property synchronization enabled, while in the alternative mode we would hide (or disable?) only the form element related to the untranslatable property (the upload item in the case of an image field), while leaving the other ones (alt/title) editable, since those changes would affect only the current translation.
However I'm not sure how easy it is to implement such a solution in a generic way. I'm wondering whether there are acceptable alternative solutions we could implement.
Comment #3
plachComment #4
matsbla commentedMy humble opinion is that this should be the default mode. If not it will not be possible to edit alt/title attributes in a draft, you can only change them if you directly publish them. My guess is that many users will even consider that as a bug, and most users would like to be able to edit translations of alt/title attributes in drafts just like they edit any other translatable fields.
Comment #5
effulgentsia commentedThat's out of scope for this issue. The
"Hide non translatable fields on translation forms"checkbox on/admin/config/regional/content-languagewas already added to core in #2878556: Ensure that changes to untranslatable fields affect only one translation in pending revisions with a default state of off. But let's perhaps open a new issue to discuss the pros and cons of changing that to default to on whencontent_moderationis enabled?I think for this issue, we just need to make sure that the behavior and validation that's conditional on that checkbox also applies to non-translatable properties within translatable fields.
I agree that that would be the expected behavior for when that checkbox is checked.
That's worth exploring. It's a similar problem as what is solved by
WidgetBaseInterface::flagErrors(), but a little different. We may need to expand that API to cover other cases of mapping properties to widget sub-elements, such as this case of the translation form needing to hide/disable some of those subelements but not others.For purposes of unblocking #2860097: Ensure that content translations can be moderated independently, what if all we did was still show the full widget (with both its translatable and untranslatable properties/elements), but we add a validation error if the translator actually tries to change an untranslatable element? It's not the most ideal solution from a usability standpoint, but at least in the meantime, until we get a better UI in place, it would prevent getting into the data integrity problem reported in #2860097-153: Ensure that content translations can be moderated independently.
Comment #6
plachThis implements a validator that prevents the entity form being saved if a synchronized property is changed while attempting to save a pending revision.
Comment #7
wim leersSo #6 implements @effulgentsia's suggestion at the end of #5:
Updated the IS for that.
Comment #8
wim leersI would like to see a failing test-only patch. So I created that. Upon walking through that locally, I first thought that this:
meant that in one case, we expect a validation error, and in the other we don't. I was wrong. What this is testing, is that we get this validation error regardless of whether the (
untranslatable_fields_hide) setting is enabled for a particular bundle. In other words: testing with that setting on and off ensures that this validation is error is triggered regardless of configuration; the validation error is triggered when a particular set of changes in properties is attempted.At least, that is my understanding, with my limited background about how all this works or is supposed to work. Hopefully that helps the next reviewer.
Patch review:
I was going to ask why this should be internal, and why this should be a new interface, because https://www.drupal.org/core/d8-bc-policy#interfaces allows us to update the existing interface.
But I think I already found the answer: the validator is now able to check for this interface, and therefore this validator only has effect for this one implementation. Other implementations are allowed, but are warned that the implementation may change, since it's marked
@internal. A "proper" API may follow in the future.Nit: s/&& /&& /
Also: that is one very very very long
returnstatement. Can we split it up for legibility?Nits:
s/Non translatable/Non-translatable/
s/field elements/fields/ (because validators don't run only for entities changed via forms, they run for entities being changed from any place)
This effectively is returning early when it's not a pending revision, because we only want to protect the case of a pending revision in this validator.
I think a comment to that effect here would go a long way in making this more understandable/maintainable in the future.
Comment #10
effulgentsia commentedThe validator implements what this message says. But is that what's desired? This means that I can't use content_moderation to work on a draft of a node, and while working on that draft, change the image that I want in the image field. Even when I'm editing in the entity's original language, not a translation.
This is in contrast to the
EntityUntranslatableFieldsConstraintthat was committed in #2878556: Ensure that changes to untranslatable fields affect only one translation in pending revisions, which says:Seems to me that that "or the original language" is an important feature to retain, even for field elements, not just fields.
Comment #11
gábor hojtsyAgreed with @effulgentsia on wording. Also @timmillwood pointed out elsewhere that "Non translatable" should be "Non-translatable" I grepped core, and there are more hyphenated than non-hyphenated. Native English speaker feedback would be nice :) We'll need an issue to clean this up even after this one. (Minor).
Comment #12
plachHere's a new patch addressing the reviews above. Working on a comment with a few replies.
Once again the interdiff is basically useless :)
Comment #13
wim leersThings that weren't clear + nits in a first review pass over this patch:
s/affect are/are/
s/is/in/
Nit: "reconcile" vs "sync" — why not also say "sync" in the comment? Different terminology suggests different things/processes are at play here, introducing doubt when trying to understand this (at least for me).
Syncs the field values of all given columns.
Incomplete comment.
Shouldn't "columns" here be "properties"?
See #8.3, second nit.
It says "properties" here, but "column" above. See prior point.
"default revision translation" is hard to interpret. Is it "the default revision of a translation" or "the default translation of a revision"?
Either of those would be clearer than what's currently there.
"reconciled" here again, see prior remark.
I don't see anything about "active" translation above, only about the "default" translation. Are those things the same?
Comment #14
plachThanks for the updates and reviews!
@Wim Leers, #8:
I did not post a test only patch because assertions are relying on a validator that does not exist in HEAD, so failures are not telling a lot IMHO. Anyway, your analysis was correct, although the new patch completely changes that approach.
Patch review:
1: Correct, I want freedom to revamp this logic if we need to.
2: Fixed
3: In this case "elements" refers to "field properties" not "fields", it's what we are using in the UI elsewhere, so I think it makes sense to keep it. Basically prioritizing UX over DX ;)
4: Fixed and added an intro in the PHP docblock. Hope it's fine now ;)
@effulgentsia, #10:
The reason why I did not implement that in #6 was that (in the second mode) untranslatable fields are not causing non-default revision translations to be marked as affected, even when they change. OTOH this would not be possible with synchronized properties, because those are regular translatable fields for the RTA flag populate logic. However, after mulling a bit on that, I realized we just need to skip synchronization to mimic the same behavior for synchronized properties. This is what #10 implements.
Edit: the only relevant change is in bold.
Comment #15
effulgentsia commentedAdding credit to @matsbla for discovering the bug: thank you!
@matsbla: Can you confirm if this patch (in conjunction with #2860097: Ensure that content translations can be moderated independently) fixes everything that you discovered?
Comment #16
plachPlease note that the patch needs to be tested in combination with #2860097: Ensure that content translations can be moderated independently to get meaningful results from the UI.
Comment #17
plach@Wim Leers, #13:
1-5: Fixed
6/8: Yep, columns is an obsolete terminology that (unnecessarily) refers to the storage. We have been referring to field properties for a while when talking about those, but all the low level stuff still uses column. Updated the patch to use column instead of property for consistency.
9-10: Fixed
11: Active translation is what we usually call the translation object being passed around, it may or may not be the default translation.
Comment #18
wim leersThe patch looks good. But it now goes so deep into Entity/Field API land that it makes me unable to sign off on this. Hoping @effulgentsia or another Entity/Field API maintainer can RTBC this.
Comment #19
matsbla commentedReally great to see so rapid fixes on this!
I tested the patches and the validation error works as expected.
However I did find another issue. The problem now is when you've enabled that checkbox ("Hide non translatable fields on translation forms") which enable you to save drafts in original language where you can change a non-translatable image file (which is great and makes sense). Once you now publish that draft it seems to me like that image file is not being synchronized. So only the original language gets that image file, whereas all the other languages will keep the former image file. The synchronization seems to work only if you publish directly, but not if you publish a draft.
Also it seems to me that when you have a pending draft where you made changes to non-translatable image files it is not possible to add drafts for translations into new languages anymore; when you do that the form is loaded with the most recent draft of the original language (containing the new not published non-translatable image file, which you're not allowed to save as a draft in translations). I'm not sure if this is by design, but for me it seem a little bit strange that I'm going to start translate from the draft in review and not the published version of the original language.
Comment #20
effulgentsia commentedThanks for #19! For others looking at this, here's the steps to reproduce it:
The reason for this looks to be:
These lines are unchanged in the patch, but it looks like
$unchanged_itemsis based solely on the 1st language, so in step 8 it's different than what's in$values, and thereforesynchronizeItems()synchronizes that difference to the 2nd language. However, in step 11, it's the same (since step 10 and step 11 are done in two separate steps), sosynchronizeItems()doesn't synchronize the new image to the 2nd language.Comment #21
effulgentsia commentedI haven't tested the code path for this, but it strikes me as odd that the implementation is so different than in
synchronizeItems(). Seems like unlike the existing code insynchronizeItems(), this new code is assuming that prior to synchronization all languages have the same number and order of items? My hunch is that that's not a correct assumption, but if it is, why is it?Comment #22
effulgentsia commentedI don't think we should add a new interface. I think we need to add methods to
FieldTranslationSynchronizerInterfaceitself. Our BC policy allows that. I think it's preferable for modules/sites that implement a custom synchronizer (do any actually exist?) to have PHP give them a fatal error pointing to them to needing to add the new method(s) rather than having those sites silently not having this validator run.I don't think this belongs on this interface (or on
FieldTranslationSynchronizerInterface). Perhaps we should open an issue to add this to some interface withinDrupal\Core\Entity? In the meantime, could we either duplicate this as a protected method in the places that need it or add a @internal trait if we want to avoid that duplication?Something like this probably does belong on
FieldTranslationSynchronizerInterface. But can we make it a narrower API? Otherwise, it's just a dumping bag of whatever is in the config variable, which we shouldn't be leaking to callers that don't need to care about that internal structure. How aboutgetSynchronizedProperties()orgetPropertiesToSynchronize()? Passed a single field definition (not an array, because the caller can call it for each field that it's interested in separately). And where the implementation is responsible for dealing with "groups" as an internal implementation detail: the caller just gets an array of the property/column names that need synchronizing. I think for purposes of naming the method, "properties" is better than "columns", since the "column" terminology is a historical implementation artifact, not something we need to leak to the API. As a separate issue, perhaps we want to rename the$columnsparameter insynchronizeItems()to$properties, since I think in PHP, renaming a parameter name is neither an API change nor a BC break?Comment #23
effulgentsia commentedWhy not allow changes to default revisions regardless of
isDefaultTranslationAffectedOnly()likeEntityUntranslatableFieldsConstraintValidatordoes?Comment #24
plach@matsbla:
Thanks for manual testing this! The attached patch should address #19/#20.
@effulgentsia, #20:
Actually the reason was that we were enforcing synchronization only when saving a default revision for a non-default translation. In fact enforcing sync is required exactly because in such a scenario, no change would be detected, since we are switching the synchronization language.
@#21:
Field synchronization aims to make translatable fields behave exactly as untranslatable fields (except for textual properties, that can be changed per translation). For this reason it will make sure that every translation has the same number of items and in the same order. For this is reason, once items are synchronized, it's perfectly safe to make those assumptions.
@#22:
Fair enough, the goal was to be free to revamp it when working on a proper fix, but this patch has reached a level of polishment that no longer justifies that approach.
Good point. My original plan was to merge the
::getOriginalTranslation()logic in that method, but then it turned out not to be needed, so fine with making it protected. We have a series of issues aiming to clean-up / fix the original entity handling, so no point in doing more work here. See #2839195: Add a method to access the original property and #2833084: $entity->original doesn't adequately address intentions when saving a revision.Fixed.
Sounds like a good idea for a novice issue ;)
@#23:
We can't do that, it's the logic in
EntityUntranslatableFieldsConstraintValidatorthat's buggy, I didn't realize that earlier because field widgets are hidden in that case, but here's an example of what might happen if widgets weren't hidden:Depending on the selected mode, you can only perform untranslatable field/sync property changes to the default revision XOR the default translation. I updated the constraint messages to make that clearer. Gabor suggested something similar in #2878556: Ensure that changes to untranslatable fields affect only one translation in pending revisions, but I think we forgot to implement it. We probably need a follow-up to fix
EntityUntranslatableFieldsConstraintValidatoras well, although it's probably not super urgent.Comment #25
plachComment #26
plachSmall optimization.
Comment #27
wim leersInterdiff review:
Nit: can't we change this to:
i.e. use the third parameter,
$default, to specify the default we want?Can we add a comment here to document what "elements" means in this context? @plach explained this in #14.
Nice! 👌 Different execution branches, different messages, makes much more sense!
(patch review)
s/default revision translation/default translation/, because that's what
\Drupal\Core\Entity\TranslatableRevisionableInterface::isDefaultTranslationAffectedOnly()is about.@plach just explained to me in a call why this is A) a separate method (i.e. not part of
synchronizeItems()), B) why its code can be so much simpler (thansynchronizeItems()).The short version of A is: because changing
synchronizeItems()is relatively risky (despite lots of existing test coverage).The short version of B is: precisely because this runs after
synchronizeItems(), a lot of the preparation work in terms of ordering and ensuring a clean data structure is already done.This should be documented, probably on the docblock for
enforceSynchronization().Comment #28
wim leers@plach asked me to review, so I did my best … but I fear that my lack of deep knowledge about the pre-existing architecture is getting in the way.
#24
I think the comment here could be clearer, but maybe that's just me not having enough familiarity with this area.
protected function enforceSynchronizatino()'s docblock, at the very end of my previous comment.Hopefully @plach's comment 24 (which addressed @effulgentsia's feedback) and my feedback in this comment and my previous comment will result in one more reroll by @plach that will allow @effulgentsia to RTBC!
Comment #29
plachThis should address #27/#28.
After discussing this with Wim, I was able to find an alternative solution to get rid of the
enforceSynchronization()method altogether, so parts of his review did not apply anymore.Comment #30
plachAnd now with more $property sauce!
Comment #31
plachJust created the following child issues, as suggested above:
Comment #32
plachI performed a live review of this with @effulgentsia and he spotted a problem in how we were handling changes in multiple translation in a single revision.
Comment #33
plachOne more follow-up: #2941092: Add a $properties argument to FieldItemList::equals().
Comment #34
effulgentsia commented#33 is looking good to me. I just want to add/change a few code comments. I'll post that within the next few hours, but since it'll just be comments, it won't affect any behavior. @matsbla: if you're around today to test #33, it would be great to get your feedback on it!
Comment #35
matsbla commentedI made a quick check now and to me it looks like patch in #33 effectively solve this issue.
Just one thing that I think is a little bit strange. but maybe this is by design;
1. I have unchecked the box "Hide non translatable fields on translation forms"
2. I publish a node with image
3. start make a draft in a second language
4. update and publish a new image in the original language
5. publish the draft of the second language (with the old image)
By publishing that draft I now revert changes made in step 4. It means after a new image is published the same image must be uploaded to every pending revision / draft that was made before, if not publishing each of them will revert the new image.
In one way it makes sense as this mode allow all languages to modify those non-translatable properties. But in another way it would also make sense if new images that were published where synchronized directly to all pending revisions.
In the other mode, when you have that checkbox checked, the new image is persevered, but you only see it once you publish the draft. When you edit the draft or look at the preview, you see the old image. You see the new image first when you publish the draft. What if I were suppose to update the image text? It could have made sense to see the new image there before you publish the pending revision.
Anyway these are just some observations, and I think this patch solve the most important issues.
Great work!
Comment #36
wim leers@plach in #29: MUCH clearer!
@matsbla in #35: thanks for confirming that this solved the problem! I don't know how this is meant to work, so I can't add nuance to the new observation you posted. Thank you very much for your continued diligence! :) 👌
Comment #37
plachThis should address the first issue reported in #35, which was indeed a bug, thanks!
The preview issues also need to be solved but for this issue the focus is on fixing the data-integrity problems, so this will need to be addressed in a follow-up. The proper solution (also to the issue above) will likely be to introduce a new
hook_entity_revision_createhook exploiting which CT would mimic the merge logic we have for untranslatable fields. This would allow to display the form with all the correct values already, instead of "making things right" on submit.Comment #38
plach@matsbla:
Of course a confirmation that the issue is fixed and that the patch is finally preventing any kind of unintended revert would be great :)
Comment #39
matsbla commentedLooks like it is working fine now, nice!
Comment #40
effulgentsia commentedI would still like to improve/add some code comments per #34, which I didn't get to yesterday due to looking more into #35 (thanks again for that great find, @matsbla) with @plach. However, I woke up today not feeling very well, so am not yet sure if I'll be able to work on those docs/comments today. Meanwhile, beta is approaching, and getting this,
#2860097: Ensure that content translations can be moderated independently, and #2936511: ContentEntityType::getRevisionMetadataKeys() never considers the BC logic because of the new "revision_default" key in by then would be super awesome, whereas docs improvements can happen after beta if necessary. Therefore, since I'm happy with all of the code itself in #37, to keep things moving, RTBC!
This needs a change record.
Comment #41
plachThanks, here is the CR: https://www.drupal.org/node/2941474
Comment #44
gábor hojtsyLooks good to me too, committed to both 8.6 and 8.5. There was this little code style problems that I fixed on commit:
Publishing change record.
Comment #45
tacituseu commentedThis is failing on 8.5.x:
https://www.drupal.org/pift-ci-job/878041
https://www.drupal.org/pift-ci-job/878043
https://www.drupal.org/pift-ci-job/878038
Comment #48
plachReverted due to the failures above.
The test was failing on 8.5.x because of
TestFileCreationTrait, that relies on some test files provided by thesimpletestmodule. It seems that 8.6.x is able to automatically track this dependency while 8.5.x isn’t. The attached patch fixes the issue in my local environment, let's see whether testbots agree.Comment #49
plachComment #50
wim leers@tacituseu: thanks for your diligence!
@plach's explanation in #48 makes sense.
ContentTranslationFieldSyncRevisionTest()passed on 8.5.x, moving back to RTBC!Comment #53
gábor hojtsyThanks everyone, committed again. We should really make the extension discovery behavior harmonized between the two branches because this will cause problems. This is the second such issue since we opened 8.6.
Comment #55
sam152 commentedSome users are reporting that this validator is causing exceptions when changing the default language of an entity. Issue here: #2995655: Invalid translation language (und) specified. in Drupal\Core\Entity\ContentEntityBase when changing language of an entity.