Problem/Motivation

As per #2183983: Find hidden configuration schema issues, several views and views_ui tests fail strict config schema checking. We fixed lots of views related schema issues in #2381973: View wizard creates 'invalid' views out of the box, missing plugin_ids, insecure permissions and #2325269: Test and fix views in test_views directories against their configuration schema but there are still some issues here and there.

Proposed resolution

Find and fix the issues. All the affected views tests are all built off of either ViewTestBase or ViewUnitTestBase, so we should find and fix the issues by going strict on config schemas there.

Remaining tasks

Fix the issues.
Review.
Commit.

User interface changes

None.

API changes

Views schemas might change and new ones may be added as appropriate.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Gábor Hojtsy’s picture

Status: Needs review » Needs work

The last submitted patch, 1: 2385805-strict-schema-check.patch, failed testing.

Status: Needs work » Needs review
effulgentsia’s picture

Priority: Major » Critical

Reading #2183983-140: Find hidden configuration schema issues, this looks like a hard blocker of that critical issue, so raising this one to critical as well, but correct me if I misunderstood.

Status: Needs review » Needs work

The last submitted patch, 1: 2385805-strict-schema-check.patch, failed testing.

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
9.08 KB
10.22 KB

Fixes based on the fails:

- Somehow some test views still included bugos items. That questions if #2325269: Test and fix views in test_views directories against their configuration schema did really test all views.
- views.access.none has no options, should not pretend it has.
- there is a 'deleted' flag on displays that should be in the schemas (see core/modules/views_ui/src/Form/Ajax/ReorderDisplays.php and core/modules/views_ui/src/ViewEditForm.php)
- Views also has a views_exposed_filter_block block type, which extends from views blocks, need the same settings (took the chance to define a base views_block type that people can extend)
- In core/modules/views/src/Tests/Plugin/DisplayTest.php when adding new displays manually to the view, we need to at least specify a valid display plugin.
- Tests use views_test_data.tests and define an access plugin, 2 display plugins, a query plugin, a row plugin and a style plugin with test settings. These need schemas in the test module.
- CustomBooleanTest.php changes the field id to boolean but not the plugin_id, so the boolean related settings were invalid later in the test.

These should fix most of the fails. There are still other fails that are more interesting :) Especially display extenders are suspicious that the way they may alter views settings may not even be possible to explain with views schema! Hah.

Wim Leers’s picture

Wow, amazing detective work!

Status: Needs review » Needs work

The last submitted patch, 6: 2385805-views-strict-fixes.patch, failed testing.

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
10.22 KB
3.35 KB

The stakes are getting higher but the fails that still remain will be the real interesting ones :)

- BlockDependenciesTest.php has an outdated copy of drupalPlaceBlock() that put visibility settings in the wrong place in the config structure, that was still failing on views blocks; this is not based off of WebTestBase, so that is why we cannot use that as-is; also there is no actual use of visibility testing with views blocks but the empty array that crept in is enough to be a problem

- My fix in DisplayTest.php was wrong, the new displays should also be instances of 'display_test', not 'page', the test later on relies on being able to set the test setting on it.

- My fix to set the boolean plugin_id in CustomBooleanTest.php was not correct. We need to set it where we override it, this is not actually stored in views data.

- In QueryTest.php, althgough the views_data is modified with a different query plugin for this table, the view already has a query plugin assigned from earlier and views only uses that table based default for new displays. So we need to modify the value on the display since we are testing an existing display.

Status: Needs review » Needs work

The last submitted patch, 9: 2385805-views-strict-fixes-9.patch, failed testing.

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
11.92 KB

Duh rolled that one the same as 6. Now rolled with the actual changes from the 9 interdiff. No change to whaat 9 should have been.

catch’s picture

- there is a 'deleted' flag on displays that should be in the schemas (see core/modules/views_ui/src/Form/Ajax/ReorderDisplays.php and core/modules/views_ui/src/ViewEditForm.php)

Should that ever make its way into config? I'd expect that to be state that views tracks internal prior to actually saving.

Status: Needs review » Needs work

The last submitted patch, 11: 2385805-views-strict-fixes-11.patch, failed testing.

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
12.83 KB
1.62 KB

@catch: Good find! So what ViewStorageTest does is the following:

    $displays = $view->get('display');
    $displays['default']['deleted'] = TRUE;
    $view->set('display', $displays);
    $view->set('id', $this->randomMachineName());
    $view->save();

Looking at the code flow, ViewEditForm::save() is the one that reacts on a 'deleted' value on display when saving, and when saving as pure config, that will not have any effect. View::save() just inherits fromConfigEntityBase::save(). I think this test code assumed it would change the view ID, so effectively saving a new view, so the deleted flag would be adhered to. But that only seems to be implemented in the form.

So we can remove the default display with the display handler and save it like that. Attached an attempt for that.

Also note that addDisplay() seems to be perfectly capable to overwrite the default display, if we want to add a default display it will not complain if there was already one. So technically we don't even need to delete the default display to ensure it is still position 0 although then we would only know if the new display was loaded back if we also assert the title. I added one assert for that for being extra sure.

Status: Needs review » Needs work

The last submitted patch, 15: 2385805-views-strict-fixes-15.patch, failed testing.

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
13.28 KB
2.34 KB

Worked on fixing some of the things I thought I fixed already...

- ViewStorageTest.php: I did not find a reliable way to remove the default display that did not have side effects in this test, so I just documented we rely on the fact that it is set again; It is not possible to create views out of the box without a default display, that is added in creation.
- CustomBooleanTest.php: I have several unrelated unserialize() fails locally but the problem seems to be that whatever we set for the field, since the view itself has a plugin_id already, we need to update the view (not just the views data); so saving the view after doing the config update will hopefully work :) again cannot run this test locally
- in QueryTest.php I attempted to update displays with the simpler array syntax, that assumes the view is loaded as the config entity not as the executable (that getView() would return), so I used that for simpler test code.

Status: Needs review » Needs work

The last submitted patch, 17: 2385805-views-strict-fixes-17.patch, failed testing.

jibran’s picture

several views and views_ui tests fail strict config schema checking

What is the difference between "config schema checking" and "strict config schema checking"?

Gábor Hojtsy’s picture

@jibran: StorableConfigBase::castValue() validates the value for storage. Ie. if you try to store an int where a mapping should be, it will throw an exception. It does allow for holes in the schema (where the type is Undefined). Strict schema checking on the other hand will find all incompatibilities with schema *including* missing schema coverage. Tests currently use $strictConfigSchema = TRUE; which makes all config use strict checking or SchemaCheckTestTrait which requires some more manual integration in tests and is used for testing specific things. We are migrating all tests over to $strictConfigSchema = TRUE; and hope to reach a state where all tests pass with strict config schema :) We found **so many** major and critical bugs on the way.

dawehner’s picture

  1. +++ b/core/modules/views/src/Tests/Plugin/BlockDependenciesTest.php
    @@ -100,13 +100,14 @@ protected function createBlock($plugin_id, array $settings = array()) {
           ),
         );
    -    foreach (array('region', 'id', 'theme', 'plugin', 'weight') as $key) {
    +    $values = [];
    +    foreach (array('region', 'id', 'theme', 'plugin', 'weight', 'visibility') as $key) {
           $values[$key] = $settings[$key];
           // Remove extra values that do not belong in the settings array.
           unset($settings[$key]);
         }
    -    foreach ($settings['visibility'] as $id => $visibility) {
    -      $settings['visibility'][$id]['id'] = $id;
    +    foreach ($values['visibility'] as $id => $visibility) {
    +      $values['visibility'][$id]['id'] = $id;
         }
         $values['settings'] = $settings;
    

    HAHA, so the block visibility patch failed, nice!

  2. +++ b/core/modules/views/src/Tests/ViewStorageTest.php
    @@ -233,14 +233,12 @@ protected function displayMethodTests() {
         // Ensure the 'default' display always has position zero, regardless of when
    -    // it was created relative to other displays.
    -    $displays = $view->get('display');
    -    $displays['default']['deleted'] = TRUE;
    -    $view->set('display', $displays);
    -    $view->set('id', $this->randomMachineName());
    -    $view->save();
    +    // it was set relative to other displays. Even if the 'default' display
    +    // exists, adding it again will overwrite it, which is asserted with the new
    +    // title.
         $view->addDisplay('default', $random_title);
         $displays = $view->get('display');
    +    $this->assertEqual($displays['default']['display_title'], $random_title, 'Default display is defined with the new title');
         $this->assertEqual($displays['default']['position'], 0, 'Default displays are always in position zero');
    

    That is a bit confusing to be honest. Now that we don't longer remove the default display, how does adding a new display works? It feels like we should maybe drop the entire test coverage then, because I doubt that we should support $view->addDisplay('default') then at all.

  3. +++ b/core/modules/views/src/Tests/Plugin/BlockDependenciesTest.php
    @@ -100,13 +100,14 @@ protected function createBlock($plugin_id, array $settings = array()) {
    -    foreach (array('region', 'id', 'theme', 'plugin', 'weight') as $key) {
    +    $values = [];
    +    foreach (array('region', 'id', 'theme', 'plugin', 'weight', 'visibility') as $key) {
           $values[$key] = $settings[$key];
           // Remove extra values that do not belong in the settings array.
           unset($settings[$key]);
         }
    -    foreach ($settings['visibility'] as $id => $visibility) {
    -      $settings['visibility'][$id]['id'] = $id;
    +    foreach ($values['visibility'] as $id => $visibility) {
    +      $values['visibility'][$id]['id'] = $id;
         }
    

    OOOH so the recent visibility failed to change those?

Gábor Hojtsy’s picture

@dawehner:

1/3 looks like the same code :) This test just copies WebTestBase::drupalPlaceBlock, so all I did was update the copy-pasted code. The exact code exists in WebTestBase.

2. addDisplay() does overwrite the default display when you try to add a default display, does not matter if there was or was not a default display before. So the result of the test is the same. Do you have a good suggestion for removing the default display reliably? $displays['default']['deleted'] = TRUE; was a no-op, it did not actually delete the display because that flag only has effect in the views *form* not the API. So it was already testing overwriting the default display I just removed code from the test that did not do anything else but set an invalid "deleted" key on the default display.

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
12.65 KB
1.43 KB

Further fixing:

- Found out that there is no reason to set the display plugin again on those test displays, those are already set.
- However, we can define the fake invalid display plugin for the sake of schemas (to inherit from page displays as they are used), so we don't get schema fails. The invalid display of course should not be defined in PHP that is the point of the test.

Status: Needs review » Needs work

The last submitted patch, 23: 2385805-views-strict-fixes-23.patch, failed testing.

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
12.65 KB
656 bytes

Helps if I don't make typos in the schema.

Status: Needs review » Needs work

The last submitted patch, 25: 2385805-views-strict-fixes-25.patch, failed testing.

dawehner’s picture

2. addDisplay() does overwrite the default display when you try to add a default display, does not matter if there was or was not a default display before. So the result of the test is the same. Do you have a good suggestion for removing the default display reliably? $displays['default']['deleted'] = TRUE; was a no-op, it did not actually delete the display because that flag only has effect in the views *form* not the API. So it was already testing overwriting the default display I just removed code from the test that did not do anything else but set an invalid "deleted" key on the default display.

Okay, opened up a follow up #2386971: View::addDisplay('default') should fail

Gábor Hojtsy’s picture

So we are down to the two most complicated fails here.

1. Display extenders are fundamentally incompatible with config (and therefore schema) assumptions. In config, the idea is every part of config has an owner that is somehow statically stored in the config as a dependency. The config entity shell itself is provided by views, but then all fields used will have a provider and therefore dependencies, etc. Display extenders are allowed to just alter the display with any additional keys in a non-identifiable way. It is not possible to tell if key X or Y is of the actual display_plugin or a display extender (by looking at the config). I think this needs a fundamental change, namely that display extenders would either be allowed to alter existing display settings or employ a ThirdPartySettingsTrait based solution where each 3rd party setting has clear ownership. I think that is a fundamental change and not sure this issue should also do that. We can exempt that test from strict schema checking for the scope here and open one more critical. Opinions?

2. When duplicating displays, what ViewEditForm::submitDuplicateDisplayAsType() does is it removes the display_title and display_plugin but otherwise clones ALL the settings of the display. That is ending up with all kinds of cruft in the new display setting, such as in the test a path for a block. Not good. So what I thought we can do there is to ask the display for their options and then filter the options from the old display with the possible options of the new. That is not possible because defineOptions() is protected on PluginBase and not intended as a public method. I previously also thought display extenders would be a problem here, but getting their stuff integrated is part of defineOptions() on DisplayPluginBase. The $view->options array is actually public, but sounds like it would be a major sin to use that directly, haha... #evilgabor

We can either choose to exempt these two tests from the strict checking in this issue and solve these in a separate issue or solve them all here. In the first case, we have even more issues, in the later we'll have wildly unrelated fixes combined. I am fine with either.

Gábor Hojtsy’s picture

dawehner’s picture

Thank you for your great work!

1. Display extenders are fundamentally incompatible with config (and therefore schema) assumptions. In config, the idea is every part of config has an owner that is somehow statically stored in the config as a dependency. The config entity shell itself is provided by views, but then all fields used will have a provider and therefore dependencies, etc. Display extenders are allowed to just alter the display with any additional keys in a non-identifiable way. It is not possible to tell if key X or Y is of the actual display_plugin or a display extender (by looking at the config). I think this needs a fundamental change, namely that display extenders would either be allowed to alter existing display settings or employ a ThirdPartySettingsTrait based solution where each 3rd party setting has clear ownership. I think that is a fundamental change and not sure this issue should also do that. We can exempt that test from strict schema checking for the scope here and open one more critical. Opinions?

I'm fine with defering a solution for that. I had an issue to basically use hook_config_schema_info_alter()to merge in config schema from the schema defined by display extenders, see interdiff, but I doubt that this is a good solution.

Sure, here is a follow up for the display extenders, see #2387149: Display extenders are not possible to describe with config schema

2. When duplicating displays, what ViewEditForm::submitDuplicateDisplayAsType() does is it removes the display_title and display_plugin but otherwise clones ALL the settings of the display. That is ending up with all kinds of cruft in the new display setting, such as in the test a path for a block. Not good. So what I thought we can do there is to ask the display for their options and then filter the options from the old display with the possible options of the new. That is not possible because defineOptions() is protected on PluginBase and not intended as a public method. I previously also thought display extenders would be a problem here, but getting their stuff integrated is part of defineOptions() on DisplayPluginBase. The $view->options array is actually public, but sounds like it would be a major sin to use that directly, haha... #evilgabor

Sounds like another critical, but fixable, to be honest: #2387157: Cloning display into another display also stores options that are not supported by the new display type

Gábor Hojtsy’s picture

Priority: Critical » Major
Status: Needs work » Needs review
FileSize
13.95 KB
1.3 KB

Thanks for opening those issues. Then all is left to do here is to exempt those two tests from the strict schema checking.

Also I think that removes the critical component of this one, so lowering to major.

dawehner’s picture

Status: Needs review » Reviewed & tested by the community

Thank you so much to push these issues forward.

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Committed dedaf33 and pushed to 8.0.x. Thanks!

  • alexpott committed dedaf33 on 8.0.x
    Issue #2385805 by Gábor Hojtsy: Views tests don't pass strict schema...
Gábor Hojtsy’s picture

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.