Problem/Motivation
field_sql_storage_field_storage_write() does a DELETE ... INSERT.
The DELETE is restricted by field language:
// Delete languages present in the incoming $entity->$field_name.
// Delete all languages if $entity->$field_name is empty.
$languages = !empty($entity->$field_name) ? $field_languages : $all_languages;
This causes a problem in the following situation:
1. Create a new node
2. Add a translation in a new revision.
3. Revert back to the original revision.
The original revision will have values set for $entity->field_name but only in the language of the original revision.
This means that the DELETE query only deletes values from {field_data_FIELD_NAME} WHERE language = the original language.
The result is that the field values for the translations remain in the {field_data_FIELD_NAME} table, albeit within incorrect revision IDs (they point to the middle revision, not the new reverted revision).
Further to this, if you then load the node, because field_sql_storage just loads whatever's in the {field_data_*} table, it will happily load those field values back into the $node object - since they're there in the table ready to be loaded.
Major for two reasons:
1. if you had open translation permissions and someone added something to a translation which was the cause of the revert, the revert will fail. The only way to remove that content would be direct database queries or manually deleting then recreating the entire node, or saving a new revision with different translated values instead of reverting. Additionally if you do something like revert then edit, the edit operation will completely resurrect the translations including with the correct revision ID (I think).
2. The bug leaves the database in an inconsistent state - the field_data_* tables are supposed to store data for the current revision, not the current revision and any random multilingual data from previous revisions.
Proposed resolution
I'm uploading a patch but I don't think it's committable - this just removes the language check altogether. This removes support for saving an entity with only translations in some languages populated - since anything else will be deleted but not re-inserted.
This is similar-ish to #1126000: [meta] hook_field_*() called on fields not set in the $entity object (partial updates) in the sense that partial entity updates are broken.
Remaining tasks
Test whether the bug exists in 8.x, and fix the patch for 7.x.
| Comment | File | Size | Author |
|---|---|---|---|
| #31 | drupal-revision-revert-messes-up-field-translation-1992010-31_D7.patch | 1.6 KB | das-peter |
| #26 | 1992010_D7.patch | 1.62 KB | joelpittet |
| #15 | 1992010_D7.patch | 1.62 KB | catch |
| #9 | field_sql_storage_revisions-1992010-9.patch | 1.76 KB | berdir |
| #9 | field_sql_storage_revisions-1992010-9-interdiff.txt | 1.32 KB | berdir |
Comments
Comment #1
catchDrupal 7 patch.
Comment #2
dcam commentedhttp://drupal.org/node/1427826 contains instructions for updating the issue summary with the summary template.
Comment #3
gábor hojtsyThe entity API allows changes in one revision for any number of languages at once. When a revision is deleted, it should not just delete one language and leave others around, that would leave the entity in an inconsistent state. The revisions data should be deleted wholesale without consideration for language AFAIS.
Comment #4
catchThat's what the patch in #1 does, so I'm going to move this to CNR.
Additionally #1126000: [meta] hook_field_*() called on fields not set in the $entity object (partial updates) would disallow saving entities with missing fields. Disallowing saving with missing languages is similar enough.
Comment #6
gábor hojtsyfield_sql_storage_revisions.patch queued for re-testing.
Comment #8
gábor hojtsyI honestly don't see why would this fail installation :/
Comment #9
berdirBecause the patch still accesses some variables that don't exist anymore :)
The languages condition on the deletes can be dropped, $field_languages on the other hand still seems to be necessary.
Just re-rolling this to see if something breaks now. I think it makes sense to wait for #1497374: Switch from Field-based storage to Entity-based storage should be fairly easy to re-create, unlike the other issue.
Comment #11
plachIt seems failures are only caused by the behavior change, hence I'd say the direction is good :)
I agree we should postpone this on #1497374: Switch from Field-based storage to Entity-based storage.
Comment #12
catchThat was committed.
Comment #13
berdirYes. I think #1983554: Remove BC-mode from EntityNG already contains the changes here, can you confirm? If so, we can just use this to backport to 7.x once the other issue is resolved..
Comment #14
gábor hojtsyNeeds work rather than active.
Comment #15
catchUpdated 7.x patch.
Haven't tested whether the bug still exists in 8.x yet.
Comment #17
areke commentedComment #18
catchFrom the test failures it looks like we need to check for $entity->revision (although that's not standardized anywhere) and use the old logic if that's not set.
I added a temporary fix for this to drafty (although it's not got anything to do with draft revisions) at http://cgit.drupalcode.org/drafty/tree/modules/drafty_1992010?h=7.x-1.x - for that it just deletes any records in field_data_* that don't match the vid of the entity that was saved.
Comment #19
plach@catch:
I checked 8.0.x and we went exactly this way, so this issue should be fixed in D8.
IMO we should just update the test to remove any assumption on field language, the only problem is that the current solution implies a BC break. Could we use a variable and keep the current behavior on existing sites and enable the patch one only for new installations?
Comment #20
plachSpoke about this with @catch: we agreed a possible fix could be adding a
revision_flag_keythat would make the following code possible:Comment #21
catchI think this is essentially the same as the new critical and one should be a duplicate. Posting from phone so excuse the bump without proper cross references.
Comment #22
plachI'm not sure this is the same issue, see the patch over there.
Comment #23
plachActually I think this is fixed in D8 by #2453153: Node revisions cannot be reverted per translation (maybe combined with #2465907: Node revision UI reverts multiple languages when only one language should be reverted).
Comment #24
plachComment #26
joelpittetreposting patch for testbot.
Comment #28
das-peter commentedFollowing and added another related ticket: #2259279: field_sql_storage_field_storage_write does not properly update fields that have additional language values removed
Test-Fails seem odd. Try to re-run.
Comment #31
das-peter commentedUgh, I think this change definitely contradicts the last failing test in
FieldSqlStorageTestCase::testFieldAttachSaveMissingData():// Update: Field translation is missing but field is not empty. Translation // data should survive.To work around this we should be a bit more specific in what to delete. How about the attached approach:
Base Table: Delete everything not related to the current revision OR revision languages.
Revision Table: Basically keep as is.
Comment #32
damienmckennaAny further thoughts on @das-peter's question in #31?
Comment #33
damienmckennaThis has gained some importance due to Workbench Moderation's v7.x-3.0 release which uses Drafty which can run into this issue.
Comment #34
drup16 commentedHi,
Just wondering which patch should be applied? I am see #15 (with failed test) and then i see #31 is there.
Is it recommended to just do #31 or will both #15 and #31 need to be applied here?
Comment #35
yonailo+1 to make this RTBC for D7, this is quite important IMHO
Comment #36
ieguskiza commented+1 to RTBC and agree this is quite important. When dealing with environments with 24 languages and content moderation enabled, reverting to previous versions is quite common and this really affects the state of the database.
Tried the approach in #31 and works perfectly fine.
Comment #37
anybodyConfirming RTBC. Thank you!
Comment #38
stefan.r commentedComment #39
joseph.olstad31 works for us
Comment #40
othermachines commentedFWIW patch in #31 works well for me.
Comment #41
David_Rothstein commentedI think this looks right, but it could really use more reviewers to look at the actual code changes and review them - since this touches a pretty fundamental part of the codebase.
Also, shouldn't we have tests for this? Seems like there are tests in the above Drupal 8 issues for this kind of functionality that could be partially backported (for example in NodeRevisionsTest::testRevisionTranslationRevert()), or if those aren't exactly right then at least a simple little test could be added to FieldSqlStorageTestCase (which has lots of tests already that involve passing different entity/language combinations in and seeing what gets written to the field tables). Then it will be more clear what the bug is and whether or not it's fully fixed.
I'm also moving this back to Major for now (since that's what's suggested in the issue summary). It's not good to have the database incorrect like this, but it's also not clear how it breaks the site in any kind of critical way.
Comment #42
dsutter commentedRTBC+ patch #31
Comment #43
gdaw commentedConfirming RTBC for patch #31
Comment #44
catchThe site breakage is this, from the issue summary:
i.e. the revert doesn't actually remove the translations from the node when it's loaded, so they're still accessible on the site.
I'm not tied to this being critical, but it's not only the database being incorrect, it has runtime implications, so moving it back for a second look.
Comment #45
damienmckennaA quick status update:
Comment #46
Chris CharltonBump.
Comment #47
mustanggb commentedBack to NW for tests then.
Comment #48
joseph.olstadThis was already fixed in D8.
For D7 I'm not entirely convinced that the test would belong here. Probably belongs with the entity_translation module since D7 core doesn't have the entity_translation module and this really belongs to that. This is still a core issue, and the patch is needed by entity_translation when using the drafty module.
Here's what the tests might look like (make more sense in entity_translation tests?) :
This is the D8 version of this test, anyone want to take a stab at backporting this test?
I say, RTBC this patch, commit it in, and add the test to entity_translation where it belongs.
Comment #49
jollysolutionsI agree with #48 this should be RTBC and test in entity translation
Comment #50
joseph.olstadBumping to 7.61, this didn't make it into 7.60
Comment #51
joseph.olstadIs our test coverage sufficient already for this?
Comment #52
stephencamilo commentedComment #53
hestenetReset issue status.
Comment #54
avpadernoHello, stephencamilo, and welcome! Since Drupal 7 is still supported, setting these issues as won't fix is wrong, even if they are old issues.
Comment #55
izmeez commentedThis issue is almost 9 years old, it is marked as "critical" and has previously been set to a status of RTBC for the patch in comment #31 that passes automated tests. It was then reverted to "needs work" based on a question, "Is our test coverage sufficient already for this?"
I don't think the status should have been changed until the question was answered. And I think this issue should be on #3259739: [meta] Priorities for 2022-06-01 release of Drupal 7.
Comment #56
izmeez commentedChanging status based on my previous comment #55.
Comment #57
izmeez commentedAdded to #3259739: [meta] Priorities for 2022-06-01 release of Drupal 7 with question "Are tests sufficient?"
Comment #58
avpadernoOn comment #41, David_Rothstein (one of the Drupal 7 maintainers) added the Needs tests tag. Since there haven't been patches after that, tests are still necessary.
Comment #59
izmeez commentedComment #48 provides a possible test that might be back ported and suggests it should be be in entity_translation.
Comment #60
avpadernoComment #48 says that, since in Drupal 8 the tests are written for the Entity Translation module, in Drupal 7 the tests should be written for the same module, which is a contributed one.
That's wrong, IMO: Since the patches in this issue are changing Drupal 7 core code, the tests need to be added to Drupal core, to avoid a regression and get the same issue again. There aren't tests for this case, or they would have caused test failures in the past already.