Problem/Motivation
TL;DR: Fields using the "Many to one" helper filter are considered as "required", even when they are in an OR group which should make it optional, due to a somewhat too aggressive optimization.
Steps to reproduce the problem:
- create a content type, named Issue, with one "list" field: let's say a field named "Version", of type List (integer), with allowed values 1, 2 and 3
- create a second type, named "Article", without any custom field
- create one Issue node with version 1 and one Article node
- create a view using the content as base table
- add a filter on the Version list field, choose "Is one of", pick the option "1"
- add a filter on Content: Type restricting it to content type 'issue'
- add a filter on Content: Type 'article'
- re-arrange the filter to put them into two groups, like so:
group 1 (AND each filter) Content type: Article OR group 2 (AND each filter) Content type: Issue Version: 1
The expected result would be to get all the articles and all the issues affecting version "1".
The actual result though right now is that you only get issues nodes.
Looking at the query, the join clauses for the version field is an INNER JOIN, which explains why all the article nodes are disappering. This is coming from views_many_to_one_helper::ensure_my_table and views_many_to_one_helper::summary_join, which are trying to use INNER JOINs to get a speed benefit "when we know we can".
Proposed resolution
Treat the cases where the filter is in an OR-group, or where all the filter groups are OR-ed together as unsafe for INNER JOIN, falling back to LEFT JOIN.
Remaining tasks
Review, write tests.
User interface changes
None.
API changes
None.
Comment | File | Size | Author |
---|---|---|---|
#11 | views-many_to_one_filters_or_groups_and_inner_join-2215703-11.patch | 17.25 KB | Andrew Answer |
|
Comments
Comment #1
DeFr CreditAttribution: DeFr commentedAttaching a quick patch that's not covering all the cases, but is at least a start.
Might be good to get the related #1989588: Many-to-one filters do not respect the filter group when excluding values on subsequent filter for same field issue moving too, because both will try to bring in group information about the filter.
Comment #2
DeFr CreditAttribution: DeFr commentedSide note: I haven't tested yet, but looking at the D8 ManyToOneHelper class, it seems likely that the bug is present there too.
Not completely sure of the backport policy here ? Does the bug need to be fixed in D8 core first ?
Comment #3
dawehnerI don't really care as long we have tests for the changed code. This code is fragile so we can't just change it without knowledge that things still work.
Even more important is some documentation in the code why this particular piece of code is like it is.
Comment #4
DeFr CreditAttribution: DeFr commentedThere you go for a bunch of test coverage and inline doc comments.
There's still quite a few cases that are not tested, but hopefully now that the plumbing is done, it'll be easier to add.
It would be kinda neat if the result of ViewsSqlTest::dataSet was keyed by the table name… It would remove the hardcoded 'views_test' table name in db_insert call of ViewsSqlTest::setUp, and would make it easier to test things needing to JOIN different tables. I think that should definitely go in D8 first though.
Comment #6
DeFr CreditAttribution: DeFr commentedThe test only version failed as expected, the test + patch version came back green as expected too. Moving back to NR.
Comment #7
DeFr CreditAttribution: DeFr commentedWorked on this a bit more this morning on the train.
- Added tests for pretty much all the operators, putting them in an OR group.
- Reverted the summary_join part. This is only used for arguments, which can't be put in groups. Added more comments to clarify. Perhaps its getting too verbose.
Comment #9
Dean Clayton CreditAttribution: Dean Clayton commentednot sure if this is a happy accident, but in the issue I've just related, we have found a workaround to this issue. to do it for the exampel given in this issue it would be as follows:
The ordering of the groups and associated filters appears to have an impact as well, so the "empty" value only seems to work when in the first group.
Comment #10
Chris Matthews CreditAttribution: Chris Matthews commentedThe 5 year old patch in #7 does not apply to the latest views 7.x-3.x-dev and if still relevant needs to be rerolled.
Comment #11
Andrew Answer CreditAttribution: Andrew Answer as a volunteer commentedPatch rerolled.
Comment #12
peetrovich CreditAttribution: peetrovich commentedPatch from #11 worked for me (Views 7.x-3.23).
Thanks a lot!