Problem/Motivation

TL;DR: Fields using the "Many to one" helper filter are considered as "required", even when they are in an OR group which should make it optional, due to a somewhat too aggressive optimization.

Steps to reproduce the problem:

  • create a content type, named Issue, with one "list" field: let's say a field named "Version", of type List (integer), with allowed values 1, 2 and 3
  • create a second type, named "Article", without any custom field
  • create one Issue node with version 1 and one Article node
  • create a view using the content as base table
  • add a filter on the Version list field, choose "Is one of", pick the option "1"
  • add a filter on Content: Type restricting it to content type 'issue'
  • add a filter on Content: Type 'article'
  • re-arrange the filter to put them into two groups, like so:
    group 1 (AND each filter)
    Content type: Article
    
    OR
    
    group 2 (AND each filter)
    Content type: Issue
    Version: 1
    

The expected result would be to get all the articles and all the issues affecting version "1".

The actual result though right now is that you only get issues nodes.

Looking at the query, the join clauses for the version field is an INNER JOIN, which explains why all the article nodes are disappering. This is coming from views_many_to_one_helper::ensure_my_table and views_many_to_one_helper::summary_join, which are trying to use INNER JOINs to get a speed benefit "when we know we can".

Proposed resolution

Treat the cases where the filter is in an OR-group, or where all the filter groups are OR-ed together as unsafe for INNER JOIN, falling back to LEFT JOIN.

Remaining tasks

Review, write tests.

User interface changes

None.

API changes

None.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

DeFr’s picture

Status: Active » Needs review
FileSize
815 bytes

Attaching a quick patch that's not covering all the cases, but is at least a start.

Might be good to get the related #1989588: Many-to-one filters do not respect the filter group when excluding values on subsequent filter for same field issue moving too, because both will try to bring in group information about the filter.

DeFr’s picture

Side note: I haven't tested yet, but looking at the D8 ManyToOneHelper class, it seems likely that the bug is present there too.

Not completely sure of the backport policy here ? Does the bug need to be fixed in D8 core first ?

dawehner’s picture

Not completely sure of the backport policy here ? Does the bug need to be fixed in D8 core first ?

I don't really care as long we have tests for the changed code. This code is fragile so we can't just change it without knowledge that things still work.

Even more important is some documentation in the code why this particular piece of code is like it is.

DeFr’s picture

FileSize
13.75 KB
11.75 KB

There you go for a bunch of test coverage and inline doc comments.

There's still quite a few cases that are not tested, but hopefully now that the plumbing is done, it'll be easier to add.

It would be kinda neat if the result of ViewsSqlTest::dataSet was keyed by the table name… It would remove the hardcoded 'views_test' table name in db_insert call of ViewsSqlTest::setUp, and would make it easier to test things needing to JOIN different tables. I think that should definitely go in D8 first though.

Status: Needs review » Needs work

The last submitted patch, 4: 2215703-4-fix-group-OR-test-only.patch, failed testing.

DeFr’s picture

Status: Needs work » Needs review

The test only version failed as expected, the test + patch version came back green as expected too. Moving back to NR.

DeFr’s picture

FileSize
15.08 KB
17.26 KB

Worked on this a bit more this morning on the train.

- Added tests for pretty much all the operators, putting them in an OR group.
- Reverted the summary_join part. This is only used for arguments, which can't be put in groups. Added more comments to clarify. Perhaps its getting too verbose.

The last submitted patch, 7: 2215703-7-or-groups-test-only.patch, failed testing.

Dean Clayton’s picture

not sure if this is a happy accident, but in the issue I've just related, we have found a workaround to this issue. to do it for the exampel given in this issue it would be as follows:

Content: Type (= Article) AND
Content: Version (empty)
OR
Content: Type (= Issue) AND
Content: Version (= 1)

The ordering of the groups and associated filters appears to have an impact as well, so the "empty" value only seems to work when in the first group.

Chris Matthews’s picture

The 5 year old patch in #7 does not apply to the latest views 7.x-3.x-dev and if still relevant needs to be rerolled.

Checking patch includes/handlers.inc...
error: while searching for:
  }

  /**
   * Provide the proper join for summary queries. This is important in part because
   * it will cooperate with other arguments if possible.
   */

error: patch failed: includes/handlers.inc:849
error: includes/handlers.inc: patch does not apply

Checking patch tests/handlers/views_handler_filter_many_to_one.test...

Checking patch views.info...
Hunk #1 succeeded at 283 (offset 11 lines).
Andrew Answer’s picture

Status: Needs work » Needs review
Issue tags: -Needs reroll
FileSize
17.25 KB

Patch rerolled.

peetrovich’s picture

Patch from #11 worked for me (Views 7.x-3.23).
Thanks a lot!