Problem/Motivation

This is easily reproducible on simplytest.me:

1. Install Drupal w/ standard profile
2. Add an article node with garbage info
3. Create a view with base table of Content Revisions, displaying fields.
4. Add field "Tags" under category "Content (historical data)"
5. In settings for that field, rewrite the field using "Override the output of this field with custom text" (actually doesn't matter what you pick, you just need something that exposes the replacement tokens)
6. Observe that the replacement token for that field contains a hyphen: "{{ field_tags-revision_id__target_id }}"

Before

Suggested token names contain a hypen
Field is rendered as 0

The problem is that you cannot have hyphens in twig variables like that. I believe it should be two underscores instead based on what I've seen in some other issues.

Proposed resolution

- Use the field alias instead, but don't break things that use field name in the view.
- Views post-update that is run as batch to be scalable for sites with many views.
- Write a test to assert the problem.
- Write a test to assert the post-update.

After

Suggested token names contain no hyphen
Field is rendered as normal tag

Remaining tasks

- Re-roll patch for 8.8.x branch. :|. Done in #52, tested in #58.
- Review current patch, tested in #58
- Manually test the patch for views post-update to confirm that the view was updated automatically (See #50 for ideal testing plan steps), done in #58.
- Don't override possible '-revision_id' string added by the administrators on the override strings. Use Regexp to actually replace only instances of '-revision_id' inside {{ }}. Done in #65

Completed

- Rewrote views post-update to use Batch API and added a test for views post-update.
- Patch #25 with views post-update, fix and test to assert the issue.
- Manual testing (DONE): Add Before/After screenshots.

Original report by bkosborne

This causes a couple issues:

1. There's an assertion in PluginBase::viewsTokenReplace() that checks if tokens are valid twig variables. So if the token replacement is performed for whatever reason, this assertion will fail and cause a 500 error.
2. I believe this would also prevent twig from performing the replacement correctly and/or using the proper field template for the field, but I can't be sure because I didn't test that behavior. The replacement token replaces the string 0 or 1, but not the actual value for the field.

I did a little big of digging and see that the token name for a given field is literally the "id" of the field, see FieldPluginBase::getFieldTokenPlaceholder(). Not sure on the solution here since I'm completely unfamiliar with this code.

CommentFileSizeAuthor
#177 2831233-177.patch9.98 KBliquidcms
#168 interdiff-2831233-159-168.txt2.38 KBacbramley
#168 2831233-168.patch19.69 KBacbramley
#166 2831233-166.patch19.54 KBameymudras
#160 2831233-159.patch19.63 KBlendude
#160 2831233-TEST_ONLY-160.patch3.93 KBlendude
#159 interdiff-2831233-157-159.txt3.02 KBacbramley
#159 2831233-159.patch19.63 KBacbramley
#157 2831233-157.patch19.51 KB_utsavsharma
#157 interdiff_156-157.txt759 bytes_utsavsharma
#156 interdiff-2831233-146-156.txt14.86 KBmradcliffe
#156 2831233-156.patch19.53 KBmradcliffe
#154 2022-12-12-field-tokens-for-historical-data-still-broken-in-10.1.x.png99.87 KBmradcliffe
#152 2831233-152.patch5.59 KBnitin shrivastava
#151 2831233-151.patch5.67 KBnitin shrivastava
#146 interdiff-2831233-144-146.txt615 bytesmohit_aghera
#146 2831233-146.patch5.49 KBmohit_aghera
#144 2831233-144.patch5.49 KBefpapado
#141 Screenshot 2021-06-02 at 11.26.16 AM.png86.92 KBnishantghetiya
#141 Screenshot 2021-06-02 at 11.23.44 AM.png60.24 KBnishantghetiya
#139 interdiff_137-139.txt1.18 KBvsujeetkumar
#139 2831233-139.patch17.66 KBvsujeetkumar
#137 2831233-137.patch17.65 KBvsujeetkumar
#132 2831233-130--tests-only.patch13.67 KBjedihe
#130 interdiff_121-130.txt553 bytesvsujeetkumar
#130 2831233-130.patch17.65 KBvsujeetkumar
#121 2831233-117.patch17.64 KBselwynpolit
#121 testonly-2831233.patch1.14 KBselwynpolit
#117 interdiff-2831233-112-117.txt734 bytesselwynpolit
#117 2831233-117.patch17.64 KBselwynpolit
#113 interdif_106_112.txt837 bytescpierce42
#113 2831233-112.patch17.64 KBcpierce42
#106 interdiff_103-106.txt573 bytescpierce42
#106 2831233-106.patch17.64 KBcpierce42
#103 2831233-103.patch17.76 KBjibran
#99 interdiff_95-99.txt11.54 KBKumar Kundan
#99 2831233-99.patch29.46 KBKumar Kundan
#95 interdiff_92-95.txt4.74 KBadityasingh
#95 2831233-95.patch19.62 KBadityasingh
#92 2831233-92.patch18 KBanushrikumari
#89 2831233-89.patch8.05 KBkapilv
#83 2831233-83.patch17.69 KBjibran
#74 interdiff_72-74.txt417 bytesphilltran
#74 drupal-2831233-views-fix-field-token-revision-8.8-74.patch17.79 KBphilltran
#72 interdiff_69-72.txt407 bytesphilltran
#72 drupal-2831233-views-fix-field-token-revision-8.8-72.patch17.8 KBphilltran
#69 interdiff_65-69.txt1.21 KBphilltran
#69 drupal-2831233-views-fix-field-token-revision-8.8-69.patch17.78 KBphilltran
#65 interdiff_63-64.txt626 bytesphilltran
#65 drupal-2831233-views-fix-field-token-revision-8.8-64.patch18.34 KBphilltran
#63 interdiff_60-63.txt6.78 KBphilltran
#63 drupal-2831233-views-fix-field-token-revision-8.8-63.patch18.33 KBphilltran
#60 interdiff_52-60.txt3.7 KBphilltran
#60 drupal-2831233-views-fix-field-token-revision-8.8-60.patch15.26 KBphilltran
#52 interdiff_45-52.txt2.09 KBmike.roman
#52 drupal-2831233-views-fix-field-token-revision-8.8-52.patch14.89 KBmike.roman
#45 drupal-2831233-views-fix-field-token-revision-8.7-45.patch14.83 KBmradcliffe
#44 field_token_revision-2831233-44.patch7.77 KBkalyansamanta
#41 drupal-2831233-views-fix-field-token-revision-8.7-41.patch14.83 KBmradcliffe
#41 drupal-2831233-views-fix-field-token-revision-8.7-testonly-41.patch10.92 KBmradcliffe
#41 interdiff-2831233-25-41.txt9.92 KBmradcliffe
#35 tokens-after-2.png16.18 KBinterx
#35 tokens-after-1.png52.09 KBinterx
#35 tokens-before-2.png12.46 KBinterx
#35 tokens-before-1.png52.85 KBinterx
#27 drupal-2831233-views-fix-field-token-revision-8.7-testonly-26.patch3.92 KBmradcliffe
#25 drupal-2831233-views-fix-field-token-revision-8.7-25.patch7.56 KBmradcliffe
#25 drupal-2831233-views-fix-field-token-revision-8.7-testonly-25.patch3.91 KBmradcliffe
#21 drupal-2831233-views-fix-field-token-revision-8.7-21.patch6.67 KBmradcliffe
#21 2831233-interdiff-18-21.txt3.05 KBmradcliffe
#18 interdiff-15-17.txt572 bytess.messaris
#18 drupal-2831233-views-fix-field-token-revision-8.6-17.patch3.64 KBs.messaris
#18 drupal-2831233-views-fix-field-token-revision-8.5-17.patch3.59 KBs.messaris
#18 drupal-2831233-views-fix-field-token-revision-8.4-17.patch3.63 KBs.messaris
#15 drupal-2831233-views-fix-field-token-revision-8.6-15.patch3.12 KBmradcliffe
#15 drupal-2831233-views-fix-field-token-revision-8.5-15.patch3.12 KBmradcliffe
#15 drupal-2831233-views-fix-field-token-revision-8.4-15.patch3.14 KBmradcliffe
#11 2831233-11.patch3 KBMunavijayalakshmi
#8 2831233-8.patch2.99 KBAnonymous (not verified)
#4 2831233-4.patch558 bytesAnonymous (not verified)
Screen Shot 2016-11-28 at 1.27.27 PM.png26.89 KBbkosborne

Comments

bkosborne created an issue. See original summary.

bkosborne’s picture

Title: Field tokens for "historical data" fields (from revisions) contain a hyphen, breaking twig templates and throwing an assertion error » Field tokens for "historical data" fields (revisions) contain a hyphen, breaking twig templates and throwing an assertion error
Issue summary: View changes
lendude’s picture

Seems like one that is left over from #2546210: Views subtokens are broken since the introduction of inline templates for replacements

The offending bit of code seems to be in views.view.inc views_field_default_views_data()

    else {
      $group = t('@group (historical data)', array('@group' => $group_name));
      $field_alias = $field_name . '-revision_id';
    }
Anonymous’s picture

Status: Active » Needs review
StatusFileSize
new558 bytes

Replacement '-' to '__' solves the problem, but it leads to a broken exist field, of course. Also, I do not know how to do the tests. Add a field to any existing view configuration with base_table: node_field_revision or create a new? Or need a total test on prohibition '-' in tokens of twig?

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

tacituseu’s picture

Status: Needs review » Needs work

Confirm the issue, #4 looks good and works, needs an update for broken fields.

tacituseu’s picture

Anonymous’s picture

Status: Needs work » Needs review
StatusFileSize
new2.99 KB

Okay, my lame step to update for broken fields :)

prash_98’s picture

The patch applies cleanly and I guess the require changes happen. So the status of the issue could be changed .

Munavijayalakshmi’s picture

Status: Needs review » Needs work
+++ b/core/modules/views/views.post_update.php
@@ -260,3 +260,66 @@ function views_post_update_revision_metadata_fields() {
+  /** @var \Drupal\views\Entity\View $view */

The Multiline comments should not be in single line as per coding standards.

Munavijayalakshmi’s picture

Status: Needs work » Needs review
StatusFileSize
new3 KB

Applied the patch for the coding standard error , Please review.

prash_98’s picture

The patch applies well.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

mradcliffe’s picture

Re-rolled a couple of patches. Too many changes for `interdiff` command to work correctly.

I was not sure whether to include the post_update hook in later versions.

mradcliffe’s picture

Issue summary: View changes
Issue tags: +Needs tests

Adjusted issue summary to confirm that the replacement tokens do not display correctly. I get strings "0" or "1" for any field value when using replacement tokens, but not when the field is displayed normally.

Also we could add a test that asserts that replacement token output is correct.

s.messaris’s picture

Hi, after applying this patch I was able to add a revision field to used fields, but I am having a problem adding it in a combined filter. I get the error SQLSTATE[42S22]: Column not found: 1054 Unknown column 'profile_revision_commerce_order__profile_revision__field_phone.field_phone__revision_id_value' in 'where clause':

Any ideas?

s.messaris’s picture

Found the cause of the problem I mentioned in #17, $field_alias was used to determine the 'real field' instead of $field_name . In case of non-revision fields we set $field_alias = $field_name; so it works, but for revision fields it breaks, since we set $field_alias = $field_name . '__revision_id'; .

I rerolled the patches in 15 and made the correction to use the $field_name variable to determine the 'real field' value.

rduplock’s picture

For future Googlers, my content moderation view was getting this error:
AssertionError: Tokens need to be valid Twig variables. in assert() (line 371 of core/modules/views/src/Plugin/views/PluginBase.php).
This patch (#18) fixed the issue.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

mradcliffe’s picture

I re-rolled this for 8.7.x and tried to write a test.

I wasn't sure whether FieldFieldTest or FieldKernelTest would be appropriate. The former has things setup for revisions and asserts values through the styles and the latter has things setup for tokens and asserts values through a renderer. I added it in FieldFieldTest, but regardless of the method used I wasn't successful in asserting that the token was replaced (getField() assertion vs Renderer assertion). The replacement IS happening as an empty string.

I tried to model the view config as closely as I could to the issue summary, but for the field_test field, and I think that there is something wrong there as the test_field_field_revision_test view is missing a lot of properties than generating a content view following the issue summary.

I really want to get this fixed because every security issue to 8.5.x means resetting and re-running views post updates, which is an annoying manual step.

Status: Needs review » Needs work

The last submitted patch, 21: drupal-2831233-views-fix-field-token-revision-8.7-21.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

mradcliffe’s picture

Status: Needs work » Needs review
-      $group = t('@group (historical data)', ['@group' => $group_name]);
-      $field_alias = $field_name . '-revision_id';
+      $group = t('@group (historical data)', array('@group' => $group_name));
+      $field_alias = $field_name . '__revision_id';

This came in the patch in #18 and also in #21. The change to array() should be reverted back to using short array syntax per coding standards.

mradcliffe’s picture

Status: Needs review » Needs work

Flip back to Needs Work because refresh.

mradcliffe’s picture

Went all the way to install the test modules into a test site, add test_field, and re-export the view. Woo!

I had trouble doing an interdiff between 21 and 25. Please review the differences as necessary.

mradcliffe’s picture

Maybe this patch for testonly.

mradcliffe’s picture

Issue summary: View changes
Issue tags: +MWDS2018

Test run looks good for both 26 (test-only) and patch 25.

Added the issue summary template to the issue.

Status: Needs review » Needs work
mradcliffe’s picture

Issue summary: View changes
Status: Needs work » Needs review

Flip back to Needs Review because #25 is working and the patch in #27 is test-only.

mradcliffe’s picture

Issue summary: View changes
Issue tags: +Novice, +Needs manual testing

Adding Novice tag and Needs manual testing tag so maybe getting Before/After screenshots would be nice.

mradcliffe’s picture

Issue tags: +DrupalEurope

Adding tag.

interx’s picture

I'm working on this at DrupalEurope

filsterjisah’s picture

I'm joining interX for the testing.

interx’s picture

StatusFileSize
new52.85 KB
new12.46 KB
new52.09 KB
new16.18 KB

The test seems to work fine.

Before:

  • Suggested token names contain a hypen:
    before 1
  • Field is rendered as 0:
    before 2

After:

  • Suggested token names contain a hypen:
    after 1
  • Field is rendered as normal tag:
    after 2

The update hook works too. The renaming of the token causes a Broken/missing handler, after the update hook, the field is working again.

filsterjisah’s picture

Status: Needs review » Reviewed & tested by the community
mradcliffe’s picture

Issue summary: View changes
Issue tags: -Novice, -Needs manual testing

Thank you for collaborating on getting some screenshots, @interX, @filsterjisah. Usually we embed the screenshots in the issue summary per Contributor task: Manually test a patch for a Drupal issue. I think it makes sense under the Proposed Resolution beneath the list. I went ahead and updated the issue summary.

Removed the Needs tests and Novice tag as there isn't anything left to do there.

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs upgrade path tests

@mradcliffe the latest patch on the issue needs to be the rtbc one for the rtbc-tester to work and to also make things less confusing.

We need upgrade path tests also we need to \Drupal\Core\Config\Entity\ConfigEntityUpdater and to make sure we batch the updates correctly - some sites have lots of views.

alexpott’s picture

Adding a duplicate issue.

mradcliffe’s picture

Issue summary: View changes

Thank you for the feedback, @alexpott.

I've updated the issue summary.

mradcliffe’s picture

I updated the post update to use ConfigEntityUpdater. Neat! That's a really useful utility class. Thanks for pointing it out, @alexpott. :-)

The patch changes:
1. Uses ConfigEntityUpdater
2. Removes the views post update message because ConfigEntityUpdater only returns the number of total entities loaded so the original message that was come up with doesn't fit.
3. Removes the End of addtogroup 8.3.x as there's no start doc block for this. It's probably coming from one of the reroll patches.
4. Adds test view for updating to test updating text.
5. Expands the views post update to fix any tokens in an altered path, which is likely to be used as well.

If the tests come back good, then I think the issue needs Manual testing and an issue summary update to reflect the changes to the views post-update.

mradcliffe’s picture

Adding the tags I mentioned in #41.

mradcliffe’s picture

Status: Needs review » Needs work
StatusFileSize
new14.83 KB

Thank you for posting an additional patch, @kalyansamanta. It really helps reviewers to explain why you made changes from the previous patch in #41 and why it needed changing? Part of doing this would be to provide an interdiff with your patch.

For instance, the test and test fixtures around the views post-update were removed. This was specifically requested by @alexpott.

I'm unhiding the previous patches and screenshots because those are still relevant to patch in #41, which needs to be uploaded.

The next step in this issue, @kalyansamanta, is to update the issue summary based on comment #41 and this comment, then provide confirmation of the patch in #41 works in a scalable way as suggested by @alexpott (See Manually test a patch for a Drupal issue).

I'm re-uploading the full patch in #41 because it reverts those changes. I'm leaving this at "Needs Work" for the issue summary and manual testing changes that I requested in #41.

mradcliffe’s picture

Issue summary: View changes
Status: Needs work » Needs review
Issue tags: -Needs issue summary update

I updated the issue summary's remaining tasks to include Manually Testing instructions and Patch Review, and changed the Status back to Needs review.

I hope that someone else can review my patch so that we can finally get this to RTBC again. If anyone has any questions about how to move this forward please feel free to contact me on Slack, IRC or DrupalChat.

AnaSwin’s picture

My content moderation view was getting this error too :
AssertionError: Tokens need to be valid Twig variables. in assert() (line 371 of core/modules/views/src/Plugin/views/PluginBase.php).
And this patch (#18) fixed the issue. My project work with Drupal 8.6.10 with php 7.2.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

mradcliffe’s picture

Issue summary: View changes
Status: Needs review » Needs work
Issue tags: +Needs reroll
+/**
+ * @addtogroup updates-8.7.x
+ * @{
+ */
+/**
+ * @} End of "addtogroup updates-8.7.x".
+ */

The addtogroup updates need to be changed to 8.8.x now.

mradcliffe’s picture

A manual testing plan for the views post-update could be to

1. Install drupal without the patch on the latest drupal dev 8.8.x.
2. Create some views including a view described in the issue summary, but also other views that use fields without revisions.
3. Take some screenshots of the view outputs.
3. Apply a re-rolled patch.
4. Run database updates (post-update).
5. Confirm the views with screenshots.

Someone else can add this to the issue summary as well. I added the Needs issue summary update tag.

mike.roman’s picture

Working on this ticket at Chicago Midcamp sprint.

mike.roman’s picture

Status: Needs work » Needs review
StatusFileSize
new14.89 KB
new2.09 KB

Attached is an updated patch for 8.8.x and the interdiff from the previous patch in comment #45. All that needed to be changed were the 8.7.x comments to 8.8.x... the interdiff shows more lines that were changed, but that seems to be just a side-effect of line number changes or something.

mtift’s picture

Issue tags: -Needs reroll
mradcliffe’s picture

Issue tags: +Seattle2019

Added tag for seattle.

tatarbj’s picture

Issue tags: +DrupalCampBelarus2019
aaronmchale’s picture

Issue tags: +DCScot19
gambry’s picture

Working on this at DCScot19

gambry’s picture

Issue summary: View changes
Status: Needs review » Needs work
Issue tags: -Needs manual testing, -Needs issue summary update

Manually tested work done on #52 and does what expected, however we are blindly updating all instances of '-revision_id', despite them being inside {{ }} or not. There could be - remote - possibilities where administrators are actually using '-revision_id' as text in overrides, i.e. internal-revision_id: {{ field_internal_id }}. We'll override this, while not needed.

Shall we process overrides with a Regexp rather than str_replace()? So we can validate the occurrence is inside {{ }}.

Also is there any way we can pre-filter the views selecting only the ones querying Content revisions?
Just to improve a bit the performances.

johne’s picture

#41 works for me. In my case I'm only using a {{ nid }} token, but this is a view that uses revision-id.

philltran’s picture

Status: Needs work » Needs review
StatusFileSize
new15.26 KB
new3.7 KB

I switched to using a regex to only change revision_id inside of `{{ }}`.

I also added `$view->save();` to save the updated view.

mradcliffe’s picture

Issue tags: +mwds2019

Added the mwds2019 tag.

Thank you, @philltran. I haven't reviewed the entire patch in #60, but I think it would be good to adjust the test view's alter text to have the token name in and out of the token, and then update the test so that the assertion shows that post-update only updated the token and not the text outside of the token.

gambry’s picture

Thanks @philltran! Amazing work.

Just few comments:

  1. +++ b/core/modules/views/views.post_update.php
    @@ -436,41 +436,45 @@
    +  $old_part = '/({{) ([a-z0-9_]+)(-revision_id)([a-z0-9_]*) (}})/';
    

    If I'm correct code within {{}} can be any valid twig code, so we should expect more than just [a-z0-9_]. Besides I think the space is not required,so this regexp won't match {{entity-revision_id}}. We can use something like /{{([^}]+)(-revision_id)/ as pattern and {{$1$2 as replacement ?

  2. +++ b/core/modules/views/views.post_update.php
    @@ -436,41 +436,45 @@
    +        foreach ($display['display_options']['fields'] as $field_name => &$field) {
    

    Why does this need to be a reference?

  3. +++ b/core/modules/views/views.post_update.php
    @@ -436,41 +436,45 @@
    +            if (preg_match($old_part, $alter_text) == 1) {
    

    This should be a strict comparison. Not needed in this case, but it's good habit.

  4. +++ b/core/modules/views/views.post_update.php
    @@ -436,41 +436,45 @@
    +            if (preg_match($old_part, $alter_path) == 1) {
    

    As above, this should be strict comparison.

  5. +++ b/core/modules/views/views.post_update.php
    @@ -483,8 +487,13 @@
    +    return $message;
    

    We were returning $is_update if order for the Config Updater to save the entity if value is TRUE. Now we are returning a string hoping this is printed, although I don't think that happens?
    If I'm correct this needs to go back to previous implementation and if we need a message the logic should me moved outside the callback and inside the post_update hook.

philltran’s picture

@gambry

Thanks for the review.

1. I thought a valid match would need to be machine-name. It doesn't hurt to broaden the pattern.

2. I switched to a reference as the view was not saving. This was related to point 5. I changed it back as view now saves when point 5 corrected.

3 and 4. I will made the switch to strict comparison.

5. I was not aware that this return triggered the config save. I switched it back to $is_update.

Thanks again for the detailed review.

Here is a new patch. I also added the test that @mradcliffe suggested.

Status: Needs review » Needs work

The last submitted patch, 63: drupal-2831233-views-fix-field-token-revision-8.8-63.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

philltran’s picture

Edited regex substitution. It should pass test now.

mradcliffe’s picture

Status: Needs work » Needs review

Bumping the status to Needs review since there's a new ptach to test.

philltran’s picture

gambry’s picture

+++ b/core/modules/views/views.post_update.php
@@ -233,7 +233,7 @@ function views_post_update_entity_link_url() {
+        foreach ($display['display_options']['fields'] as $field_name => $field) {

I think this is a leak of "Find and Replace"? :D

The rest looks good to me.
I was going to suggest to add additional tests, but remembered this actually does not work at all at the moment, so unlikely (IMO impossible) any site-builder is using this replacement placeholder.

A part from change above, this is +1 RTBC for me.

Thanks @philltran!!!

philltran’s picture

@gambry Oh wow. Thanks for catching that. It was a find and replace error. I changed the correct code this time.

Attached is a new patch

gambry’s picture

Thanks @philltran!

Sorry, I've got one more which is actually not a change but more a doubt:

+++ b/core/modules/views/views.post_update.php
@@ -426,3 +426,73 @@ function views_post_update_limit_operator_defaults(&$sandbox = NULL) {
+/**
+ * Fix '-revision_id' of field alias.
+ */

I'm not sure this description is correct?

philltran’s picture

Status: Needs review » Needs work

@gambry, No worries, we want it correct. Also, I didn't write that bit LOL.

It's updating the replacement token references, right?

philltran’s picture

Status: Needs work » Needs review
StatusFileSize
new17.8 KB
new407 bytes

Here is a fresh patch. Please review.

Thanks!

gambry’s picture

Not sure what token references would mean? Maybe Fix '-revision_id' replacement token syntax.?

philltran’s picture

That sounds better.

mradcliffe’s picture

I manually tested the patch in #74 on a site that's affected by this to confirm that it worked after I updated to 8.8 (from 8.7).

Thank you for following up and working on this, @philltran and @gambry.

philltran’s picture

@mradcliffe Thanks for testing and the help at mwds2019.
You are welcome.

gambry’s picture

Status: Needs review » Reviewed & tested by the community

So this now can be RTBC!

I've been thinking if this needs a Change Record, and I don't think so as - although we are changing something (a replacement token) - that has never work so it's 99.99% unlikely anyone is using the old - not working - version.

Thanks everyone!

philltran’s picture

Thanks!

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

mradcliffe’s picture

Issue tags: -Novice

I'm going to remove the Novice tag on this until it needs a re-roll. I added a test run for 8.9.x PHP 7.3, MySQL 5.6 on #74.

mradcliffe’s picture

I'm going to add this tag for @Lendude

lendude’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: -Needs subsystem maintainer review +Needs tests

Discussed this with @mradcliffe and what we still miss here is a test-only patch that we expect to pass on HEAD. The test only patch we have only tests the update (which I wouldn't expect the pass without the update) and the kernel test that tests the new formatting (which I don't expect to pass).

We we should have a test that follows the steps to reproduce and shows that we have a problem in HEAD.

The rest looks great, awesome work!

jibran’s picture

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

bygeoffthompson’s picture

Thank you for this issue report and patch. I have applied #83 against Drupal 8.9.3 and see everything working as expected with my Twig Tokens, "Rewrite Results", and Moderated Content view. A couple notes:

  • Install patch
  • Run database update
  • Re-save view with previously reported error: "AssertionError: Tokens need to be valid Twig variables."
  • Flush application caches

This has updated my replacement tokens from using hyphens to acceptable underscores.

mkolar’s picture

Status: Needs review » Reviewed & tested by the community

This looks good to me, using on project patch #83 since February 2020 without any problem.

mradcliffe’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs reroll

Hi, @mkolar. We shouldn't mark this as Reviewed & tested by the community, unfortunately, because we haven't met what needs to be done to make this reviewable.

We still need to address @Lendude's comment of creating a standalone test in views.

It looks like the patch cannot be applied to 8.9.x (and probably 9.1.x), and so we need to re-roll the patch again.

mkolar’s picture

Im sorry, my bad. sorry for spam.

kapilv’s picture

StatusFileSize
new8.05 KB

Hear a reroll patch.

kapilv’s picture

Status: Needs work » Needs review
philltran’s picture

@kapilkumar0324 Thanks for the re-roll.

Just noting that re still need to address @Lendude's comment of creating a standalone test in views.

anushrikumari’s picture

StatusFileSize
new18 KB

re-rolling for 9.1.x as patch #89 failed to apply.
#82 is still pending.

Status: Needs review » Needs work

The last submitted patch, 92: 2831233-92.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

adityasingh’s picture

Status: Needs work » Needs review
Issue tags: -Needs reroll
StatusFileSize
new19.62 KB
new4.74 KB

Worked on test cases.

gambry’s picture

@adityasingh can you explain what work you have done, against what test cases?
I can see you have unblocked the tests from #92 system/tests/fixtures/update/drupal-8.4.0.bare.standard.php.gz): failed to open stream: operation failed and fixed some conding standard issues (I'm not sure we should do, as not part of this work?). But for instance I'm not sure why we need the changes against SqlContentEntityStorageSchema ?

Thanks a lot for your help!!

Status: Needs review » Needs work

The last submitted patch, 95: 2831233-95.patch, failed testing. View results

adityasingh’s picture

Hi @gambry
Coding standard fix I think were scope creep, and can be dropped.
The updates/work on the patch you had got right as can be seen in interdiff... as far the schema file is concerned I was pointed to that file while testing on my local and hence went ahead updating that file.
Point is taken... will try to brief about the patch work in future.

Thanks..

Kumar Kundan’s picture

Status: Needs work » Needs review
StatusFileSize
new29.46 KB
new11.54 KB

Addressed failed test case of #95.

Status: Needs review » Needs work

The last submitted patch, 99: 2831233-99.patch, failed testing. View results

Kumar Kundan’s picture

Status: Needs work » Needs review
mradcliffe’s picture

Status: Needs review » Needs work
  1. --- /dev/null
    +++ b/core/core/modules/views/tests/fixtures/update/fix-revision-id-update.php
    
    --- /dev/null
    +++ b/core/core/modules/views/tests/fixtures/update/views.view.test_fix_revision_id_update.yml
    
    --- /dev/null
    +++ b/core/core/modules/views/tests/src/Functional/Update/ViewsFixRevisionIdUpdateTest.php
    

    These files shouldn't be here. It looks like the patch was applied in the wrong directory?

  2. +++ b/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorageSchema.php
    @@ -1488,11 +1488,15 @@ protected function processRevisionDataTable(ContentEntityTypeInterface $entity_t
    -    if ($schema['fields'][$key]['type'] == 'int') {
    ...
    -    unset($schema['fields'][$key]['default']);
    
    +++ b/core/modules/views/views.views.inc
    @@ -21,15 +21,15 @@
    -  // #global is a special flag which allows a table to appear all the time.
    ...
       $data['views']['random'] = [
    
    @@ -115,7 +115,7 @@ function views_views_data() {
    -   'title' => t('Combine fields filter'),
    

    Can we revert the changes made in #95 so that the patch is only about the bug fix?

    It might be easier to revert back to the patch in #89 and try to re-roll that one before proceeding.

    At the moment it is hard to see what changed between #95 because of the extra files.

jibran’s picture

Status: Needs work » Needs review
StatusFileSize
new17.76 KB

Here is a reroll from #83.

mradcliffe’s picture

Status: Needs review » Needs work
Issue tags: +NorthAmerica2021

It looks like we need to fix the code standards in patch in comment #103.

Then after we get that, we can start working on the tests required from comment #82.

cpierce42’s picture

Status: Needs work » Needs review

Going to attempt to fix code standards in patch #103

cpierce42’s picture

StatusFileSize
new17.64 KB
new573 bytes

Applying coding standard fix to patch to change array() to []
Tomorrow I'd like to work on writing a test with Selwyn Polit.

selwynpolit’s picture

I assisted Caleb with this issue and plan to try to write the functional test tomorrow.

mradcliffe’s picture

@antojose and I mentored @cpierce42 and @selwynpolit at DrupalCon NorthAmerica2021.

Thanks for submitting a patch, Caleb!

Status: Needs review » Needs work

The last submitted patch, 106: 2831233-106.patch, failed testing. View results

mradcliffe’s picture

+++ b/core/modules/views/tests/src/Functional/Update/ViewsFixRevisionIdUpdateTest.php
@@ -0,0 +1,78 @@
+      __DIR__ . '/../../../../../system/tests/fixtures/update/drupal-8.4.0.bare.standard.php.gz',

This got updated recently. If we look at the directory there should be a drupal-8.8.0.bare.standard.php.gz file to use instead.

cpierce42’s picture

@mradcliffe, today going to try to fix the patch to point to 8.8.0.bare.standard.php.gz instead.

selwynpolit’s picture

I'm going to explore writing the functional test to test the patch.

cpierce42’s picture

Status: Needs work » Needs review
StatusFileSize
new17.64 KB
new837 bytes

Applying patch for changing 8.4.0.bare.standard.php.gz to 8.8.0.bare.standard.php.gz

Status: Needs review » Needs work

The last submitted patch, 113: 2831233-112.patch, failed testing. View results

mradcliffe’s picture

It looks like ViewsFixRevisionIdUpdateTest needs some changes.

selwynpolit’s picture

Today I'm looking at fixing the error in the test at core/modules/views/tests/src/Functional/Update/ViewsFixRevisionIdUpdateTest.php

selwynpolit’s picture

StatusFileSize
new17.64 KB
new734 bytes

Here is the updated patch to core/modules/views/tests/src/Functional/Update/ViewsFixRevisionIdUpdateTest.php which runs successfully on my local system. Also included is the interdiff.

Previous filename for the patch was 2831233-112 even though the attachment was in comment #113

kristen pol’s picture

Working during mentored DrupalCon NA 2021 event with bugsmash team and first time contributors. Some people are working on tests and I'm going to help people on issue triage to make sure the issue looks okay otherwise.

nicoloye’s picture

I worked with Selwyn to help with the patch and test writing during DrupalCon NA 2021.

jedihe’s picture

Working with the Bugsmash team on this one, during the DrupalCon NA 2021 contribution event.

selwynpolit’s picture

StatusFileSize
new1.14 KB
new17.64 KB

Uploading testonly patch which should fail. This should address the test requirement issue in #82. If this test fails, then all testing requirements should be satisfied.

I reuploaded the patch from 117 to stop the bott from changing the status when my testonly patch fails.

See interdiff in #117

antojose’s picture

Working with Selwyn on the patch, during the DrupalCon NA 2021 contribution event.

selwynpolit’s picture

Status: Needs work » Needs review

The whole party, represented by 5 different continents agrees in principal to set the status to needs review to inspire the tests to run, rather than manually clicking the button.

larowlan’s picture

Status: Needs review » Needs work
Issue tags: +Bug Smash Initiative

@nicoloye @Kristen Pol @selwynpolit @antojose @jedihe @mradcliffe at DrupalCon on this issue, assigning issue credit for those folks.

Tagging for Bug Smash

jedihe’s picture

Status: Needs work » Needs review

Setting back to Needs Review.

jedihe’s picture

I just did a quick review for:

- Interdiff #106, Ok
- Interdiff #113, Ok
- Interdiff #117, Ok, the fixture update was decided in the Bug Smash video call.

The adjustment asked for in #115 was ruled out by @larowlan, in the Bug Smash video call.

Status: Needs review » Needs work

The last submitted patch, 121: 2831233-117.patch, failed testing. View results

The last submitted patch, 121: testonly-2831233.patch, failed testing. View results

jedihe’s picture

Status: Needs review » Needs work

Patch in #121 failed testing, back to Needs work.

vsujeetkumar’s picture

Status: Needs work » Needs review
StatusFileSize
new17.65 KB
new553 bytes

Fixing fail test mentioned in #129.

jedihe’s picture

Issue summary: View changes
jedihe’s picture

StatusFileSize
new13.67 KB

Since the patch in #130 touched on the tests, the fail result from #121 must be re-validated. I'm now attaching the tests-only patch for #130 to perform that re-validation.

Status: Needs review » Needs work

The last submitted patch, 132: 2831233-130--tests-only.patch, failed testing. View results

jedihe’s picture

Great! both new tests fail; Drupal\Tests\views\Kernel\Handler\FieldFieldTest failing should satisfy #82.

Patch #130 is ready for review.

mohit_aghera’s picture

Issue tags: -Needs tests

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

vsujeetkumar’s picture

Status: Needs work » Needs review
StatusFileSize
new17.65 KB

Re-roll patch given for 9.3.x, As per comment #134, It is ready for review.

Status: Needs review » Needs work

The last submitted patch, 137: 2831233-137.patch, failed testing. View results

vsujeetkumar’s picture

Status: Needs work » Needs review
StatusFileSize
new17.66 KB
new1.18 KB

Fixing the fail tests. Moved to the needs review.

Status: Needs review » Needs work

The last submitted patch, 139: 2831233-139.patch, failed testing. View results

nishantghetiya’s picture

Status: Needs work » Reviewed & tested by the community
StatusFileSize
new60.24 KB
new86.92 KB

Patch #139 applied successfully. Adding screenshots of before and after patches.

Testing steps:
1. Install Drupal w/ standard profile
2. Add an article node with garbage info
3. Create a view with a base table of Content Revisions, displaying fields.
4. Add field "Tags" under category "Content (historical data)"
5. In settings for that field, rewrite the field using "Override the output of this field with the custom text" (actually doesn't matter what you pick, you just need something that exposes the replacement tokens)
6. Observe that the replacement token for that field contains a hyphen: "{{ field_tags-revision_id__target_id }}"

kim.pepper’s picture

Issue tags: +#pnx-sprint
catch’s picture

Status: Reviewed & tested by the community » Needs work
+++ b/core/modules/views/views.post_update.php
@@ -68,3 +68,64 @@ function views_post_update_rename_default_display_setting() {
+ */
+function views_post_update_fix_revision_id_part(&$sandbox = NULL) {
+  $message = '';
+  // Regex to search only for token with machine name '-revision_id'
+  $old_part = '/{{([^}]+)(-revision_id)/';
+  $new_part = '{{$1__revision_id';
+  $old_field = '-revision_id';
+  $new_field = '__revision_id';
+  \Drupal::classResolver(ConfigEntityUpdater::class)->update($sandbox, 'view', function ($view) use ($message, $old_part, $new_part, $old_field, $new_field) {
+    /** @var \Drupal\views\ViewEntityInterface $view */
+    $is_update = FALSE;
+    $displays = $view->get('display');
+    foreach ($displays as $display_name => &$display) {

The actual update logic should be happening in a hook_entity_presave(), so that the post update only needs to load and save the config entities - will ensure the same change is made to default config in modules when it's installed. See ViewsConfigUpdater for other examples.

efpapado’s picture

StatusFileSize
new5.49 KB

Uploading a patch that is based on #139 and on catch's #143 suggestion to move the actual update in hook_entity_presave()
The uploaded patch contains the code that performs the update, as well as the fix in views.view.inc, without any of the #139's tests.

mohit_aghera’s picture

Assigned: Unassigned » mohit_aghera
mohit_aghera’s picture

Assigned: mohit_aghera » Unassigned
Status: Needs work » Needs review
StatusFileSize
new5.49 KB
new615 bytes

Fixing the code quality check failures.
"sh ./core/scripts/dev/commit-code-check.sh" is passing for all the files now.

bkosborne’s picture

Status: Needs review » Needs work
Issue tags: +Needs tests

At one point there were tests for this, but they are gone now.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

nitin shrivastava’s picture

StatusFileSize
new5.67 KB

reroll for 9.4.x

nitin shrivastava’s picture

StatusFileSize
new5.59 KB

try to fix #151 CCF error

jurgenhaas’s picture

The patches #146 to #152 do not apply for Drupal 10 any longer. Replaying the instructions from the OP shows that it is not even required any more, the bug seems to be fixed. The previously broken Twit variable name is now called {{ field_tags__revision_id__target_id }} == Raw target_id which is perfectly OK.

mradcliffe’s picture

Thank you for testing, @jurgenhass. I could no confirm this is fixed in 10.1.x-dev. I followed the steps to reproduce just now using 10.1.x-dev on DrupalPod, and I was still able to reproduce the issue.

The token was still incorrect as {{ field_tags-revision_id }} == Tags and {{ field_tags-revision_id__target_id }} == Raw target_id.

Views configure field modal with override text and replacement patterns displaying incorrect token

nicolas bouteille’s picture

With patch #152 applied on 9.4.8, I get an error when I try to save my view.
ParseError: syntax error, unexpected ')' in Composer\Autoload\includeFile() (line 587 of core/modules/views/src/ViewsConfigUpdater.php).
After looking at the code, it seems the foreach line 587 is indeed incomplete:
foreach ($displays) {
Also, the new tokens don't seem to be available...
Patch #151 does bring the new tokens which works great thanks and does not prevent saving the view.

mradcliffe’s picture

Status: Needs work » Needs review
StatusFileSize
new19.53 KB
new14.86 KB

I redid the re-roll of the patch from #146 and then re-added the tests in #139.

_utsavsharma’s picture

StatusFileSize
new759 bytes
new19.51 KB

Fixed CCF for #156.

Status: Needs review » Needs work

The last submitted patch, 157: 2831233-157.patch, failed testing. View results

acbramley’s picture

Status: Needs work » Needs review
Issue tags: -Needs tests
StatusFileSize
new19.63 KB
new3.02 KB

Tests are back as of a few patches ago.

  1. Fixed the initial test fail by adding a UUID to the fixture yaml.
  2. Fixed the actual failure by referencing $display by reference.
  3. Fixed the doc block for processRevisionFieldHyphenFix
  4. Added various return types, etc
lendude’s picture

StatusFileSize
new3.93 KB
new19.63 KB

So to get back to my point in #82, we still miss a proper test-only patch that proves we are fixing a bug here.

So here is my take on that, changed the test view to use the old replacement token en matched the test to use that token. As expected it fails, but now we have proof.

Also uploading the patch from #159 again to make the testbot happy and it won't get stuck on the test-only patch

The last submitted patch, 160: 2831233-TEST_ONLY-160.patch, failed testing. View results

smustgrave’s picture

Status: Needs review » Needs work
Issue tags: +Needs Review Queue Initiative, +Needs change record

Confirmed the issue following the steps in the IS.
Applied patch #160 which solves the issue
Also patch TEST_ONLY-160 shows the issue.

Think this will need a change record to announce the new public function processRevisionFieldHyphenFix

acbramley’s picture

Status: Needs work » Needs review
Issue tags: -Needs change record

Discussed in slack and agreed we do not need a CR.

smustgrave’s picture

Status: Needs review » Reviewed & tested by the community

Don't mind marking it then.

larowlan’s picture

Status: Reviewed & tested by the community » Needs work

This looks really close, but unfortunately doesn't apply anymore.

Also, I think there's a couple of lines of dead code in the updater now too (as below).

  1. +++ b/core/modules/views/src/ViewsConfigUpdater.php
    @@ -264,4 +267,86 @@ protected function processOembedEagerLoadFieldHandler(array &$handler, string $h
    +            unset($display['display_options']['fields'][$field_name]);
    

    nit: this line isn't needed anymore right? we're replacing the whole array so the old key should be gone

  2. +++ b/core/modules/views/src/ViewsConfigUpdater.php
    @@ -264,4 +267,86 @@ protected function processOembedEagerLoadFieldHandler(array &$handler, string $h
    +            $display['display_options']['fields'][$field_name_update] = $field;
    

    same here I think

ameymudras’s picture

Status: Needs work » Needs review
StatusFileSize
new19.54 KB

I have re rolled the above patch #160 and fixed the issues mentioned above. Couldn't generate a interdiff because the above patch doesn't apply

Status: Needs review » Needs work

The last submitted patch, 166: 2831233-166.patch, failed testing. View results

acbramley’s picture

Status: Needs work » Needs review
StatusFileSize
new19.69 KB
new2.38 KB

Reroll from #159, #166 was rolled from 160 which is not correct.

Also removed #165.1, the other line is needed though. Added test coverage to prove we're removing the old field.

smustgrave’s picture

Status: Needs review » Reviewed & tested by the community

Don't mind remarking then.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 168: 2831233-168.patch, failed testing. View results

smustgrave’s picture

Status: Needs work » Reviewed & tested by the community

Another random failure

larowlan’s picture

Updating issue credits

  • larowlan committed 246d721a on 10.1.x
    Issue #2831233 by mradcliffe, philltran, vsujeetkumar, selwynpolit, s....
larowlan’s picture

Status: Reviewed & tested by the community » Fixed

Committed to 10.1.x

Not backporting because of update path

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

liquidcms’s picture

I know this is closed and I am late to the game here; but do I have this right?

In D10 historical fields in views were changed from __revision_id to -revision_id. There was no updb script for this, so when i migrated to D10 a bunch of views now have broken/missing handlers. And then, once i go through and fix all of these by re-adding all these fields, all the rewrites are broken because twig can't handle the new field values in its tokens?

And the fix here, is to preprocess the twig token values and put them back to what they used to be (D9)? Silly question, but wouldnt it have been easier (and not broken a ton of sites) to just change the field ids back to what they were?

That being said, anyone know if any of these patches apply to D10.0? Haven't found one yet. :(

liquidcms’s picture

StatusFileSize
new9.98 KB

No, it does seem like this patch is putting the views field ids back as they were in D9 __revision_id. Yay!

Couldn't find a patch here that worked with 10.0.11 so attaching one here. It is based off the patch from #168.