Problem/Motivation

Views currently has a concept of handlers, which controls how a certain part of a sql table is accessed in a view. So each sql table field
has a handler which takes care about the behaviour. In the actual view config currently views just stores the $table.$field and then work out the used handler (which is a Durpal 8 plugin) out of the hook_views_data. Sadly the proposed metadata config needs to have a plugin_id to be able to load it's submetadata. Another issue on which we need this is a reworked version of broken handlers. With the plugin_id as part of the config we can tell the user which plugin is missing.

Proposed resolution

Save the plugin_id as well on the config and alter all existing views data to contain this information.
The name export would look similar to that:

display:
  default:
    fields:
      title:
        id: title
        table: node
        field: title
        plugin_id: node_title
        module: node (this would be added in the other issue)

Comments

dawehner’s picture

Status: Active » Needs review
StatusFileSize
new860 bytes

This patch is required to have a proper translation support for views.

Started to write a patch to convert all existing views config files

use Symfony\Component\Yaml\Parser;
use Symfony\Component\Yaml\Dumper;

$files = file_scan_directory(drupal_get_path('module', 'views'), '@\.yml$@');

$parser = new Parser();
$dumper = new Dumper();
$dumper->setIndentation(2);
foreach ($files as $file) {
  $uri = $file->uri;
  $content = file_get_contents($uri);
  $data = $parser->parse($content);

  if (isset($data['display'])) {
    foreach ($data['display'] as $display_id => &$display) {
      if (isset($display['display_options']['fields'])) {
        foreach ($display['display_options']['fields'] as &$field) {
          $data = views_fetch_data($field['table']);
          if (isset($data[$field['field']]['field']['id'])) {
            $plugin_id = $data[$field['field']]['field']['id'];
            $field['plugin_id'] = $plugin_id;
          }
          else {
            dsm($uri);
          }
       }
      }
     }


  $output = $dumper->dump($data, PHP_INT_MAX);
  file_put_contents($uri, $output);
  }

}
dawehner’s picture

StatusFileSize
new30.95 KB
new6.15 KB

Currently the script needs to support other handler types then fields.

This patch runned the previous script and changed all of the test patches by hand, because the views data for the tests is dynamic so you can't call views_fetch_data on that.

Status: Needs review » Needs work

The last submitted patch, views-1810480-2.patch, failed testing.

xjm’s picture

Project: Views (for Drupal 7) » Drupal core
Version: 8.x-3.x-dev » 8.x-dev
Component: Code » views.module
dawehner’s picture

StatusFileSize
new1.38 KB
new43.06 KB

Did some work on that, now it fixes already a lot of the yml files.

Sadly the script doesn't work in all situations yet, so you still have to change a lot of places manually.

dawehner’s picture

Status: Needs work » Needs review
StatusFileSize
new60.79 KB
new4.41 KB
new5.01 KB

For sanity here is a patch which just shows the php changes.

To be able to compare arrays really easy i added a helper assertion for that, which also gives you the diff
if something is not equal. Maybe this could be moved into the main assertions?

dawehner’s picture

PS: This interdiff is coming from another issue.

yesct’s picture

Issue tags: +D8MI

adding tag for d8mi since in #1 @dawehner says

This patch is required to have a proper translation support for views.
dawehner’s picture

StatusFileSize
new63.26 KB

Updated the new views config files as well, so basically a rerole.

Status: Needs review » Needs work
Issue tags: -D8MI, -VDC

The last submitted patch, drupal-1810480-9.patch, failed testing.

yesct’s picture

Status: Needs work » Needs review
Issue tags: +D8MI, +VDC

#9: drupal-1810480-9.patch queued for re-testing.

ACF’s picture

StatusFileSize
new64.15 KB

Trying a re-roll.

dawehner’s picture

StatusFileSize
new63.48 KB

Another reroll against some other recent changes.

xjm’s picture

But #1648930: Introduce configuration schema and use for translation :( Totally thinking of a different issue. Proceed.

xjm’s picture

Issue summary: View changes

update

damiankloip’s picture

Yeah, I was going to say... :)

This patch looks good to me, there are a couple of places that are wrapping '' around strings. Do we need to do this? generally in yaml you don't need to for single word blocks. Do we have a standard? or are those the only ones left unchanged?

dawehner’s picture

StatusFileSize
new63.47 KB
new1.75 KB

Yeah the plugin is called 'null' so there is no way around that. One odd thing was nid: but i guess the casting took care about that.

damiankloip’s picture

Status: Needs review » Reviewed & tested by the community

Fair enough, I like this in that case.

xjm’s picture

+++ b/core/modules/views/lib/Drupal/views/Tests/ViewTestBase.phpundefined
@@ -60,6 +60,35 @@ protected function enableViewsTestModule() {
+      $this->assert(TRUE, $message ? $message : t('Value @first is equal to value @second.', array('@first' => var_export($first, TRUE), '@second' => var_export($second, TRUE))), $group);
...
+      $this->assert(FALSE, $message ? $message : t('The first value differs from the second by @diff.', array('@diff' => var_export($diff, TRUE))));

Should be format_string() rather than t().

diff --git a/core/modules/views/lib/Drupal/views/ViewExecutable.php b/core/modules/views/lib/Drupal/views/ViewExecutable.php
@@ -2061,15 +2061,18 @@ public function addItem($display_id, $type, $table, $field, $options = array(),
     // If the desired type is not found, use the original value directly.
     $handler_type = !empty($types[$type]['type']) ? $types[$type]['type'] : $type;
 
-    // @todo This variable is never used.
-    $handler = views_get_handler($table, $field, $handler_type);
-
     $fields[$id] = array(
       'id' => $id,
       'table' => $table,
       'field' => $field,
     ) + $options;
 
+    // Load the plugin ID if available.
+    $data = views_fetch_data($table);
+    if (isset($data[$field][$handler_type]['id'])) {
+      $fields[$id]['plugin_id'] = $data[$field][$handler_type]['id'];
+    }

For the sake of reviewers, here's the actual change.

diff --git a/core/modules/views/tests/views_test_config/test_views/views.view.test_views_move_to_field.yml b/core/modules/views/tests/views_test_config/test_views/views.view.test_views_move_to_field.yml
deleted file mode 100644
index 22642b5..0000000

diff --git a/core/modules/views/tests/views_test_config/test_views/views.view.test_views_move_to_handler.yml b/core/modules/views/tests/views_test_config/test_views/views.view.test_views_move_to_handler.yml
deleted file mode 100644
index d66c95d..0000000

diff --git a/core/modules/views/tests/views_test_config/test_views/views.view.test_views_move_to_table.yml b/core/modules/views/tests/views_test_config/test_views/views.view.test_views_move_to_table.yml
deleted file mode 100644
index 956a7f0..0000000

I think removing these is out of scope? Can we file a separate issue to kill them if appropriate?

dawehner’s picture

StatusFileSize
new61.72 KB

Yeah regarding the actual php nothing changed since #6.

I think removing these is out of scope? Can we file a separate issue to kill them if appropriate?

OK.

Fixed the assertion message.

dawehner’s picture

catch’s picture

+++ b/core/modules/views/lib/Drupal/views/Tests/ViewTestBase.phpundefined
@@ -60,6 +60,35 @@ protected function enableViewsTestModule() {
   /**
+   * Asserts that two arrays are the same, but not checking the order of it.
+   *
+   * @param array $first
+   *   The first value to check.
+   * @param array $second
+   *   The second value to check.
+   * @param string $message
+   *   (optional) A message to display with the assertion. Do not translate
+   *   messages: use format_string() to embed variables in the message text, not
+   *   t(). If left blank, a default message will be displayed.
+   * @param string $group
+   *   (optional) The group this message is in, which is displayed in a column
+   *   in test output. Use 'Debug' to indicate this is debugging output. Do not
+   *   translate this string. Defaults to 'Other'; most tests do not override
+   *   this default.
+   * @return bool
+   *   TRUE on pass, FALSE on fail.
+   */
+  protected function assertArrayEqual(array $first, array $second, $message = '', $group = 'Other') {
+    $diff = array_diff($first, $second);
+    if (empty($diff)) {
+      $this->assert(TRUE, $message ? $message : format_string('Value @first is equal to value @second.', array('@first' => var_export($first, TRUE), '@second' => var_export($second, TRUE))), $group);
+    }
+    else {
+      $this->assert(FALSE, $message ? $message : format_string('The first value differs from the second by @diff.', array('@diff' => var_export($diff, TRUE))));
+    }

This might be useful for other things, could put directly in Simpletest maybe?

+++ b/core/modules/views/lib/Drupal/views/ViewExecutable.phpundefined
@@ -2061,15 +2061,18 @@ public function addItem($display_id, $type, $table, $field, $options = array(),
+    // Load the plugin ID if available.
+    $data = views_fetch_data($table);
+    if (isset($data[$field][$handler_type]['id'])) {
+      $fields[$id]['plugin_id'] = $data[$field][$handler_type]['id'];
+    }

How was the plugin ID found before? Doesn't look this this patch removes that code?

dawehner’s picture

What about moving this function to the top in a follow up?

How was the plugin ID found before? Doesn't look this this patch removes that code?

So before the patch the data for handlers looks like that:

  table: node
  field: title

then views uses that information && $data = hook_views_data() and get the id out of $data[$table][$field]['field']['id']. This is currently not yet affected by this patch, to make it as simple as possible.

Sadly the proposed metadata config needs to have a plugin_id to be able to load it's submetadata. Another issue on which we need this is a reworked version of broken handlers. With the plugin_id as part of the config we can tell the user which plugin is missing.

catch’s picture

hmm OK, is there a follow-up for the hook_views_data() bit?

dawehner’s picture

To be honest i don't think this will be useful, because we need the metadata attached in hook_views_data anyway to initialize a handler.

catch’s picture

OK one more question then.

Something like
+ plugin_id: node_created_month

If a module changes hook_views_data() and their views plugin to a different name, then currently can an existing view survive this since that's inferred rather than explicity? With the patch wouldn't every view need to be updated to point to the new plugin name?

dawehner’s picture

Yeah you are totally right about that, but this is a general problem who ones a certain piece of configuration. For normal plugins like a style plugin this is already the case,
so i guess we will have to come up with an easy way to either load and save all views or manually update them?

xjm’s picture

This might be useful for other things, could put directly in Simpletest maybe?

The Views tests also have a method like that for objects IIRC.

dawehner’s picture

Opened a follow up for the assertion message: #1870786: Add an assertArrayEqual method to compare two arrays

xjm’s picture

Regarding changing a plugin ID: IMO, if a module changes its data tables or plugin IDs or whatever, it's that module's responsibility to provide an upgrade path with that. I think it'd be a matter of loading all saved views, checking them for the old plugin, and then updating them and re-saving. Pretty simple. However, we could either provide API for it or document how to do it. Loosely related issues:

I filed a specific followup to discuss this further: #1874394: Provide an API for upgrading configuration

webchick’s picture

Status: Reviewed & tested by the community » Fixed

Ok sounds like the follow-ups in #29 should address catch's concerns. If I'm wrong, please feel free to roll this back.

Committed and pushed to 8.x. Thanks!

webchick’s picture

Status: Fixed » Needs work

Nevermind. :P This broke HEAD because it references the non-existent views_fetch_data() function.

Reverted.

dawehner’s picture

Status: Needs work » Needs review
StatusFileSize
new62.34 KB

I'm sorry for breaking core, so we should really retest our RTBC issues more often.

dawehner’s picture

Anyone up for a RTBC again?

damiankloip’s picture

Status: Needs review » Reviewed & tested by the community

Yep, happy with this now, as it uses our shiny new views_data service and not views_fetch_data.

catch’s picture

Issue tags: -D8MI, -VDC

#32: drupal-1810480-32.patch queued for re-testing.

Status: Reviewed & tested by the community » Needs work
Issue tags: +D8MI, +VDC

The last submitted patch, drupal-1810480-32.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
StatusFileSize
new68.29 KB
new68.29 KB

Let's update all the changes yml files in the meantime.

This patch is a damn nighmare.

Status: Needs review » Needs work

The last submitted patch, drupal-1810480-37.patch, failed testing.

dawehner’s picture

StatusFileSize
new75.5 KB

Please review that, as it's a pain to rerole.

dawehner’s picture

Status: Needs work » Needs review

.

damiankloip’s picture

+++ b/core/modules/comment/tests/modules/comment_test_views/test_views/views.view.test_comment_rss.ymlundefined
@@ -61,4 +62,3 @@ human_name: test_comment_rss
-uuid: 9b1b1e58-d41b-468a-9d04-4e6bde742c29

This might not be allowed, but I don't mind. Having an issue just for that seems overkill.

+++ b/core/modules/node/tests/modules/node_test_views/test_views/views.view.test_node_revision_vid.ymlundefined
@@ -9,26 +9,31 @@ display:
-          id: nid:

Same, here, but still ok :)

+++ b/core/modules/system/lib/Drupal/system/Plugin/views/row/EntityRow.phpundefined
@@ -117,7 +117,7 @@ public function pre_render($result) {
+      // Get all entities which will be used to render in rows.c

I think this one IS a mistake?

+++ b/core/modules/views/lib/Drupal/views/Tests/ViewStorageTest.phpundefined
@@ -332,12 +339,13 @@ protected function displayMethodTests() {
+    $this->assertArrayEqual($item2, $expected_item);

Awesome

+++ b/core/modules/views/lib/Drupal/views/ViewExecutable.phpundefined
@@ -1989,9 +1989,6 @@ public function addItem($display_id, $type, $table, $field, $options = array(),
-    // @todo This variable is never used.

We might not want to bother removing this if we want #1825896: Add module owner to plugin data on handlers in?

Otherwise, let's please try and get this in asap!

Status: Needs review » Needs work

The last submitted patch, drupal-1810480-39.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
StatusFileSize
new74.84 KB
new675 bytes

Yeah this @todo is a long outstanding one.

Regarding the ':': It seems to be that the yml parser might ignore that?

sun’s picture

+++ b/core/modules/node/tests/modules/node_test_views/test_views/views.view.test_node_revision_vid.yml
@@ -9,26 +9,31 @@ display:
       fields:
         vid:
           id: vid
           table: node_revision
           field: vid
+          plugin_id: standard
         nid_1:
           id: nid_1
           table: node_revision
           field: nid
+          plugin_id: standard

+++ b/core/modules/node/tests/modules/node_test_views/test_views/views.view.test_status_extra.yml
@@ -111,12 +112,14 @@ display:
       sorts:
         nid:
           id: nid
           table: node
           field: nid
           order: ASC
+          plugin_id: standard

+++ b/core/modules/taxonomy/tests/modules/taxonomy_test_views/test_views/views.view.test_groupwise_term.yml
@@ -51,6 +54,7 @@ display:
           id: tid
           order: DESC
           table: taxonomy_term_data
+          plugin_id: standard

Is "standard" really a plugin ID...? (?)

+++ b/core/modules/user/tests/modules/user_test_views/test_views/views.view.test_view_argument_validate_user.yml
@@ -17,6 +17,7 @@ display:
           table: views
           validate:
             type: user
+          plugin_id: 'null'

null?

+++ b/core/modules/views/lib/Drupal/views/Tests/ViewTestBase.php
@@ -60,6 +60,35 @@ protected function enableViewsTestModule() {
   /**
+   * Asserts that two arrays are the same, but not checking the order of it.
...
+  protected function assertArrayEqual(array $first, array $second, $message = '', $group = 'Other') {

I guess you meant "ignoring the array keys"?

assertEqual() ignores the order already.

Status: Needs review » Needs work

The last submitted patch, drupal-1810480-43.patch, failed testing.

damiankloip’s picture

The comment is fair enough I think, but I don't think we should be discussing actual plugin IDs in this issue. And yes, we have one called standard.. I know :)

dawehner’s picture

Status: Needs work » Needs review
StatusFileSize
new4.63 KB
new71.67 KB

Is "standard" really a plugin ID...? (?)

Yes our default plugins are called standard, especially because you can't have a Default.php (sure I know that $file_name != $plugin_id).

null?

Yes, we have a argument plugin, which does nothing with the query, aka. ignores the argument value.

Interesting comment regarding assertArrayEqual(), let's remove that from the patch as it's not needed.

tim.plunkett’s picture

#47: drupal-1810480-47.patch queued for re-testing.

tim.plunkett’s picture

Status: Needs review » Reviewed & tested by the community

RTBC if it still is green.

webchick’s picture

Status: Reviewed & tested by the community » Fixed

Committed and pushed to 8.x. Thanks!

Automatically closed -- issue fixed for 2 weeks with no activity.

Anonymous’s picture

Issue summary: View changes

update view export