Problem/Motivation

field_sql_storage_field_storage_write() does a DELETE ... INSERT.

The DELETE is restricted by field language:

      // Delete languages present in the incoming $entity->$field_name.
      // Delete all languages if $entity->$field_name is empty.
      $languages = !empty($entity->$field_name) ? $field_languages : $all_languages;

This causes a problem in the following situation:

1. Create a new node

2. Add a translation in a new revision.

3. Revert back to the original revision.

The original revision will have values set for $entity->field_name but only in the language of the original revision.

This means that the DELETE query only deletes values from {field_data_FIELD_NAME} WHERE language = the original language.

The result is that the field values for the translations remain in the {field_data_FIELD_NAME} table, albeit within incorrect revision IDs (they point to the middle revision, not the new reverted revision).

Further to this, if you then load the node, because field_sql_storage just loads whatever's in the {field_data_*} table, it will happily load those field values back into the $node object - since they're there in the table ready to be loaded.

Major for two reasons:

1. if you had open translation permissions and someone added something to a translation which was the cause of the revert, the revert will fail. The only way to remove that content would be direct database queries or manually deleting then recreating the entire node, or saving a new revision with different translated values instead of reverting. Additionally if you do something like revert then edit, the edit operation will completely resurrect the translations including with the correct revision ID (I think).

2. The bug leaves the database in an inconsistent state - the field_data_* tables are supposed to store data for the current revision, not the current revision and any random multilingual data from previous revisions.

Proposed resolution

I'm uploading a patch but I don't think it's committable - this just removes the language check altogether. This removes support for saving an entity with only translations in some languages populated - since anything else will be deleted but not re-inserted.

This is similar-ish to #1126000: [meta] hook_field_*() called on fields not set in the $entity object (partial updates) in the sense that partial entity updates are broken.

Remaining tasks

Test whether the bug exists in 8.x, and fix the patch for 7.x.

Comments

catch’s picture

Drupal 7 patch.

dcam’s picture

http://drupal.org/node/1427826 contains instructions for updating the issue summary with the summary template.

gábor hojtsy’s picture

The entity API allows changes in one revision for any number of languages at once. When a revision is deleted, it should not just delete one language and leave others around, that would leave the entity in an inconsistent state. The revisions data should be deleted wholesale without consideration for language AFAIS.

catch’s picture

Status: Needs work » Needs review

That's what the patch in #1 does, so I'm going to move this to CNR.

Additionally #1126000: [meta] hook_field_*() called on fields not set in the $entity object (partial updates) would disallow saving entities with missing fields. Disallowing saving with missing languages is similar enough.

Status: Needs review » Needs work
Issue tags: -Needs issue summary update, -translatable fields, -Needs backport to D7

The last submitted patch, field_sql_storage_revisions.patch, failed testing.

gábor hojtsy’s picture

Status: Needs work » Needs review

field_sql_storage_revisions.patch queued for re-testing.

Status: Needs review » Needs work
Issue tags: +Needs issue summary update, +translatable fields, +Needs backport to D7

The last submitted patch, field_sql_storage_revisions.patch, failed testing.

gábor hojtsy’s picture

I honestly don't see why would this fail installation :/

berdir’s picture

Status: Needs work » Needs review
StatusFileSize
new1.32 KB
new1.76 KB

Because the patch still accesses some variables that don't exist anymore :)

The languages condition on the deletes can be dropped, $field_languages on the other hand still seems to be necessary.

Just re-rolling this to see if something breaks now. I think it makes sense to wait for #1497374: Switch from Field-based storage to Entity-based storage should be fairly easy to re-create, unlike the other issue.

Status: Needs review » Needs work

The last submitted patch, field_sql_storage_revisions-1992010-9.patch, failed testing.

plach’s picture

Status: Needs work » Postponed

It seems failures are only caused by the behavior change, hence I'd say the direction is good :)

I agree we should postpone this on #1497374: Switch from Field-based storage to Entity-based storage.

catch’s picture

Status: Postponed » Active

That was committed.

berdir’s picture

Yes. I think #1983554: Remove BC-mode from EntityNG already contains the changes here, can you confirm? If so, we can just use this to backport to 7.x once the other issue is resolved..

gábor hojtsy’s picture

Issue summary: View changes
Status: Active » Needs work

Needs work rather than active.

catch’s picture

Version: 8.0.x-dev » 7.x-dev
Status: Needs work » Needs review
StatusFileSize
new1.62 KB

Updated 7.x patch.

Haven't tested whether the bug still exists in 8.x yet.

Status: Needs review » Needs work

The last submitted patch, 15: 1992010_D7.patch, failed testing.

areke’s picture

Issue summary: View changes
catch’s picture

From the test failures it looks like we need to check for $entity->revision (although that's not standardized anywhere) and use the old logic if that's not set.

I added a temporary fix for this to drafty (although it's not got anything to do with draft revisions) at http://cgit.drupalcode.org/drafty/tree/modules/drafty_1992010?h=7.x-1.x - for that it just deletes any records in field_data_* that don't match the vid of the entity that was saved.

plach’s picture

@catch:

I checked 8.0.x and we went exactly this way, so this issue should be fixed in D8.

From the test failures it looks like we need to check for $entity->revision (although that's not standardized anywhere) and use the old logic if that's not set.

IMO we should just update the test to remove any assumption on field language, the only problem is that the current solution implies a BC break. Could we use a variable and keep the current behavior on existing sites and enable the patch one only for new installations?

plach’s picture

Spoke about this with @catch: we agreed a possible fix could be adding a revision_flag_key that would make the following code possible:

if (!empty($entity->{$revision_flag_key})) {
  // Skip field language fiddling.
}
else {
  // Keep things as now.
}
catch’s picture

Version: 7.x-dev » 8.0.x-dev
Priority: Major » Critical

I think this is essentially the same as the new critical and one should be a duplicate. Posting from phone so excuse the bump without proper cross references.

plach’s picture

I'm not sure this is the same issue, see the patch over there.

plach’s picture

Status: Needs work » Needs review

joelpittet queued 15: 1992010_D7.patch for re-testing.

joelpittet’s picture

StatusFileSize
new1.62 KB

reposting patch for testbot.

Status: Needs review » Needs work

The last submitted patch, 26: 1992010_D7.patch, failed testing.

The last submitted patch, 1: 1992010_field_sql_storage_d7-do-not-test.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 26: 1992010_D7.patch, failed testing.

das-peter’s picture

Status: Needs work » Needs review
StatusFileSize
new1.6 KB

Ugh, I think this change definitely contradicts the last failing test in FieldSqlStorageTestCase::testFieldAttachSaveMissingData():

// Update: Field translation is missing but field is not empty. Translation
    // data should survive.

To work around this we should be a bit more specific in what to delete. How about the attached approach:
Base Table: Delete everything not related to the current revision OR revision languages.
Revision Table: Basically keep as is.

damienmckenna’s picture

Any further thoughts on @das-peter's question in #31?

damienmckenna’s picture

This has gained some importance due to Workbench Moderation's v7.x-3.0 release which uses Drafty which can run into this issue.

drup16’s picture

Hi,

Just wondering which patch should be applied? I am see #15 (with failed test) and then i see #31 is there.

Is it recommended to just do #31 or will both #15 and #31 need to be applied here?

yonailo’s picture

+1 to make this RTBC for D7, this is quite important IMHO

ieguskiza’s picture

Status: Needs review » Reviewed & tested by the community

+1 to RTBC and agree this is quite important. When dealing with environments with 24 languages and content moderation enabled, reverting to previous versions is quite common and this really affects the state of the database.
Tried the approach in #31 and works perfectly fine.

anybody’s picture

Confirming RTBC. Thank you!

stefan.r’s picture

joseph.olstad’s picture

31 works for us

othermachines’s picture

FWIW patch in #31 works well for me.

David_Rothstein’s picture

Priority: Critical » Major
Status: Reviewed & tested by the community » Needs review
Issue tags: -Drupal bugfix target +Needs subsystem maintainer review, +Needs tests, +Drupal 7.60 target

I think this looks right, but it could really use more reviewers to look at the actual code changes and review them - since this touches a pretty fundamental part of the codebase.

Also, shouldn't we have tests for this? Seems like there are tests in the above Drupal 8 issues for this kind of functionality that could be partially backported (for example in NodeRevisionsTest::testRevisionTranslationRevert()), or if those aren't exactly right then at least a simple little test could be added to FieldSqlStorageTestCase (which has lots of tests already that involve passing different entity/language combinations in and seeing what gets written to the field tables). Then it will be more clear what the bug is and whether or not it's fully fixed.

I'm also moving this back to Major for now (since that's what's suggested in the issue summary). It's not good to have the database incorrect like this, but it's also not clear how it breaks the site in any kind of critical way.

dsutter’s picture

RTBC+ patch #31

gdaw’s picture

Confirming RTBC for patch #31

catch’s picture

Priority: Major » Critical

The site breakage is this, from the issue summary:

Further to this, if you then load the node, because field_sql_storage just loads whatever's in the {field_data_*} table, it will happily load those field values back into the $node object - since they're there in the table ready to be loaded.

i.e. the revert doesn't actually remove the translations from the node when it's loaded, so they're still accessible on the site.

I'm not tied to this being critical, but it's not only the database being incorrect, it has runtime implications, so moving it back for a second look.

damienmckenna’s picture

A quick status update:

  • Several people have RTBC'd it.
  • It needs tests.
  • It needs an update to the issue summary to highlight the potential for data corruption.
  • The approach needs signoff from a subsystem maintainer; who'd be the best person/people for this?
Chris Charlton’s picture

Bump.

mustanggb’s picture

Status: Needs review » Needs work

Back to NW for tests then.

joseph.olstad’s picture

This was already fixed in D8.
For D7 I'm not entirely convinced that the test would belong here. Probably belongs with the entity_translation module since D7 core doesn't have the entity_translation module and this really belongs to that. This is still a core issue, and the patch is needed by entity_translation when using the drafty module.

Here's what the tests might look like (make more sense in entity_translation tests?) :

This is the D8 version of this test, anyone want to take a stab at backporting this test?

+++ b/core/modules/system/src/Tests/Entity/EntityRevisionTranslationTest.php
@@ -57,4 +57,39 @@ public function testNewRevisionAfterTranslation() {
     $this->assertTrue($this->reloadEntity($entity)->getRevisionId() > $old_rev_id, 'The entity from the storage has a newer revision id.');
   }

  /**
   * Tests if the translation object has the right revision id after new revision.
   */
  public function testRevertRevisionAfterTranslation() {
    $user = $this->createUser();
    $storage = $this->entityManager->getStorage('entity_test_mulrev');

    // Create a test entity.
    $entity = EntityTestMulRev::create([
      'name' => $this->randomString(),
      'user_id' => $user->id(),
      'language' => 'en',
    ]);
    $entity->save();
    $old_rev_id = $entity->getRevisionId();

    $translation = $entity->addTranslation('de');
    $translation->setNewRevision();
    $translation->save();

    $entity = $this->reloadEntity($entity);

    $this->assertTrue($entity->hasTranslation('de'));

    $entity = $storage->loadRevision($old_rev_id);

    $entity->setNewRevision();
    $entity->isDefaultRevision(TRUE);
    $entity->save();

    $entity = $this->reloadEntity($entity);

    $this->assertFalse($entity->hasTranslation('de'));
  }

 }

I say, RTBC this patch, commit it in, and add the test to entity_translation where it belongs.

jollysolutions’s picture

Status: Needs work » Reviewed & tested by the community

I agree with #48 this should be RTBC and test in entity translation

joseph.olstad’s picture

Issue tags: -Drupal 7.60 target +Drupal 7.70 target

Bumping to 7.61, this didn't make it into 7.60

joseph.olstad’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: -Drupal 7.70 target +Drupal 7.74 target

Is our test coverage sufficient already for this?

stephencamilo’s picture

Status: Needs work » Closed (won't fix)
hestenet’s picture

Status: Closed (won't fix) » Needs work

Reset issue status.

avpaderno’s picture

Hello, stephencamilo, and welcome! Since Drupal 7 is still supported, setting these issues as won't fix is wrong, even if they are old issues.

izmeez’s picture

This issue is almost 9 years old, it is marked as "critical" and has previously been set to a status of RTBC for the patch in comment #31 that passes automated tests. It was then reverted to "needs work" based on a question, "Is our test coverage sufficient already for this?"

I don't think the status should have been changed until the question was answered. And I think this issue should be on #3259739: [meta] Priorities for 2022-06-01 release of Drupal 7.

izmeez’s picture

Status: Needs work » Reviewed & tested by the community

Changing status based on my previous comment #55.

izmeez’s picture

Added to #3259739: [meta] Priorities for 2022-06-01 release of Drupal 7 with question "Are tests sufficient?"

avpaderno’s picture

Issue tags: -Drupal 7.74 target

On comment #41, David_Rothstein (one of the Drupal 7 maintainers) added the Needs tests tag. Since there haven't been patches after that, tests are still necessary.

izmeez’s picture

Comment #48 provides a possible test that might be back ported and suggests it should be be in entity_translation.

avpaderno’s picture

Comment #48 says that, since in Drupal 8 the tests are written for the Entity Translation module, in Drupal 7 the tests should be written for the same module, which is a contributed one.
That's wrong, IMO: Since the patches in this issue are changing Drupal 7 core code, the tests need to be added to Drupal core, to avoid a regression and get the same issue again. There aren't tests for this case, or they would have caused test failures in the past already.

Status: Reviewed & tested by the community » Closed (outdated)

Automatically closed because Drupal 7 security and bugfix support has ended as of 5 January 2025. If the issue verifiably applies to later versions, please reopen with details and update the version.