The attached testcase verifies the following problem:

If a user has access to a node because of multiple rows the comment count query SELECT COUNT(*) FROM comment ... is altered as such: SELECT DISTINCT COUNT(*) FROM comment LEFT JOIN node_access .... This results in twice the number of comments for two roles, three times as many comments for three roles and so on.
The visible impact this has, is that the comment pager get's broken, because it adds superfluous pages for the non existing comments.

By hacking core like this, i.e. removing the optimized count query and using SELECT COUNT(*) FROM subquery, the problem goes away. But that's not so nice and might have a performance penalty.

diff --git a/modules/comment/comment.module b/modules/comment/comment.module
index 239cd5b..4da98cc 100644
--- a/modules/comment/comment.module
+++ b/modules/comment/comment.module
@@ -828,17 +828,8 @@ function comment_get_thread($node, $mode, $comments_per_page) {
     ->addMetaData('node', $node)
     ->limit($comments_per_page);
 
-  $count_query = db_select('comment', 'c');
-  $count_query->addExpression('COUNT(*)');
-  $count_query
-    ->condition('c.nid', $node->nid)
-    ->addTag('node_access')
-    ->addTag('comment_filter')
-    ->addMetaData('node', $node);
-
   if (!user_access('administer comments')) {
     $query->condition('c.status', COMMENT_PUBLISHED);
-    $count_query->condition('c.status', COMMENT_PUBLISHED);
   }
   if ($mode === COMMENT_MODE_FLAT) {
     $query->orderBy('c.cid', 'ASC');
@@ -851,7 +842,6 @@ function comment_get_thread($node, $mode, $comments_per_page) {
     $query->orderBy('torder', 'ASC');
   }
 
-  $query->setCountQuery($count_query);
   $cids = $query->execute()->fetchCol();
 
   return $cids;

Comments

xjm’s picture

Status: Active » Needs review

Setting NR so the bot runs the test suite. We're expecting a fail here.

Status: Needs review » Needs work

The last submitted patch, taxonomy-access-comment-count.patch, failed testing.

Niklas Fiekas’s picture

Status: Needs work » Needs review
StatusFileSize
new2.83 KB

Fixed the notices. Hope it fails for the reason it should, now.

Niklas Fiekas’s picture

StatusFileSize
new3.08 KB

Experimenting ...

Edit: This makes sure that it's actually viewing the node page and has access to the comments section. Getting crazy here ... but it passed again. Now I can only hope it fails on your machine, too.

Niklas Fiekas’s picture

StatusFileSize
new3.17 KB

One last try for tonight: Create 60 comments so that we can assert that two pages exists, but no third.

In the mean-time: These look similar: #135722: Taxonomy Access Control causes paging error on front page (Drupal 5) closed as duplicate of #668268: Weird behavior of pager for non-admin users (Drupal 6), where someone explains, that it has to do with the role count.

Status: Needs review » Needs work

The last submitted patch, taxonomy-access-comment-count-3.patch, failed testing.

Niklas Fiekas’s picture

Status: Needs work » Needs review
StatusFileSize
new3.17 KB

Hehe, think I've got it now. I am using clean URLs locally, so testing for ?page= is fine, but the testbot has no clean URLs and thus &page=. Using just page= now.

Status: Needs review » Needs work

The last submitted patch, taxonomy-access-comment-count-4.patch, failed testing.

Niklas Fiekas’s picture

Yay! Just as it should.

Niklas Fiekas’s picture

Title: Having multiple node access grants within the same realm breaks comment pager query » Having access because of multiple roles breaks comment count query
Project: Drupal core » Taxonomy Access Control
Version: 8.x-dev » 7.x-1.x-dev
Component: node.module » Code

This is the query after all altering:

SELECT DISTINCT COUNT(*) AS expression
FROM 
{comment} c
INNER JOIN {node_access} na ON na.nid = c.nid
WHERE 
  (c.nid = 1) AND (c.status = 1)
  AND (
    ( (na.gid = 0) AND (na.realm = all) )
    OR( (na.gid = 2) AND (na.realm = taxonomy_access_role) )
    OR( (na.gid = 8) AND (na.realm = taxonomy_access_role) )
    OR( (na.gid = 9) AND (na.realm = taxonomy_access_role) )
  )AND (na.grant_view >= 1)
Niklas Fiekas’s picture

StatusFileSize
new3.2 KB

Port of the test to Drupal 8 core with a little debugging in the pager attached.

And the debug output:

'SELECT DISTINCT COUNT(*) AS expression
FROM 
{comment} c
INNER JOIN {node_access} na ON na.nid = c.nid
WHERE  (c.nid = :db_condition_placeholder_0) AND (c.status = :db_condition_placeholder_1) AND(( (na.gid = :db_condition_placeholder_2) AND (na.realm = :db_condition_placeholder_3) )OR( (na.gid = :db_condition_placeholder_4) AND (na.realm = :db_condition_placeholder_5) )OR( (na.gid = :db_condition_placeholder_6) AND (na.realm = :db_condition_placeholder_7) ))AND (na.grant_view >= :db_condition_placeholder_8) '


array (
  ':db_condition_placeholder_0' => '1',
  ':db_condition_placeholder_1' => 1,
  ':db_condition_placeholder_2' => 0,
  ':db_condition_placeholder_3' => 'all',
  ':db_condition_placeholder_4' => '2',
  ':db_condition_placeholder_5' => 'node_access_test_author',
  ':db_condition_placeholder_6' => 8888,
  ':db_condition_placeholder_7' => 'node_access_test',
  ':db_condition_placeholder_8' => 1,
)
xjm’s picture

Title: Having access because of multiple roles breaks comment count query » Having multiple node access grants within the same realm breaks comment pager query
Project: Taxonomy Access Control » Drupal core
Version: 7.x-1.x-dev » 8.x-dev
Component: Code » node.module
Status: Needs work » Needs review
StatusFileSize
new3.08 KB

Alright, here's a modified version of the test in #11 that reproduces the bug by adding a duplicate grant realm for the permission. Might want to create a separate grant realm for this test, but for now it's just a demonstration of the core bug. We expect a fail here.

Thanks @Niklas Fiekas for all your work on this!

Status: Needs review » Needs work

The last submitted patch, node-access-pager.patch, failed testing.

xjm’s picture

Title: Having access because of multiple roles breaks comment count query » Having multiple node access grants within the same realm breaks comment pager query
Project: Taxonomy Access Control » Drupal core
Version: 7.x-1.x-dev » 8.x-dev
Component: Code » node.module
Niklas Fiekas’s picture

Thought about that one. No problem since we're building our own count query anyway.

xjm’s picture

Assigned: Unassigned » xjm

revoof a.k.a @Niklas Fiekas confirmed that adding GROUP by realm, gid to the query resolves the issue. However, the interesting thing here is that I believe the base functionality is provided by pager.inc, the queries are in comment_admin_overview() and comment_get_thread(), and the node access alteration is coming from _node_query_node_access_alter(). Too many cooks...

The best I can come up with is adding a query alter in comment.module to alter queries tagged with node access and using the pager. I'm not sure if there is a more generalized problem or solution for node access queries with the pager that I'm not seeing at the moment, but it's a start.

xjm’s picture

Status: Needs work » Needs review
Issue tags: +Needs backport to D7
StatusFileSize
new4.13 KB

Alright, here's a relatively simple fix. It is targeted only at this specific query.

Other things to check for:

  1. Does the comment admin form pager at admin/content/comment have a similar bug?
  2. Are there any other pagers affected by node access that show this bug?

Other possible solutions (depending on above answers):

  1. Somehow change the query I have tagged in comment_get_thread() so it doesn't need the GROUP BY.
  2. Do something clever in _node_query_node_access_alter() to automatically add these GROUP BY when appropriate.
  3. Make pagers "better."
  4. ?

Also, @Niklas Fiekas and I discussed ways to make the test coverage more complete. But for now... sending to the bot.

Edit: Mmm, just thought of a way that this might break everything... the alter is still running even when _node_query_node_access_alter() is returning without adding node access joins. Durr. Well, bot should tell.

Status: Needs review » Needs work

The last submitted patch, comment-pager-1390046-17.patch, failed testing.

xjm’s picture

Yay, I broke every single comment test except the one added by this issue. :)

xjm’s picture

Status: Needs work » Needs review
StatusFileSize
new0 bytes

A patch which is slightly less likely to break everything. Kinda a hack, though... I added a special tag that says "please group meh!" in the node access query alter.

xjm’s picture

StatusFileSize
new4.09 KB

Weird?

Niklas Fiekas’s picture

Status: Needs review » Needs work

I like it. This is a hack for sure, but not a bad one to solve the issue in D7. In D8 #778050: Add support for database hints and make PagerDefault properly pluggeable might change paged queries, so that we no longer need a special count query for MySQL and PostGres and might fallback on subqueries for SQLite.

About the position of the groupBy: Even if we have the node_access_groupby tag, can we be sure the node_access table is always joined?

Having admin/content/comment, I am looking how that is build now. On a default setup, where you have the privileges to see that, you often have bypass access privileges, too, so problems there would be hard to notice.
No problem there, that view is using the automatically generated subquery solution, instead of a custom count query.

xjm’s picture

Per revoof, we're at least going to check for $access_alias to be set before adding the GROUP BY, to make it a little more robust. I still wonder if we can fix the base query instead, though.

Kjartan also pointed out this issue: #1237380: Forum count incorrect when using access control modules, now for D7

xjm’s picture

Status: Needs work » Closed (duplicate)
Niklas Fiekas’s picture

StatusFileSize
new11.27 KB
new11.58 KB
new10.44 KB
new10.71 KB

Wrong issue again ... uploading the files anyway to avoid having to redo it in the other issue.

xjm’s picture

Assigned: xjm » Unassigned
xjm’s picture

Issue summary: View changes

Add workaround.