The attached testcase verifies the following problem:
If a user has access to a node because of multiple rows the comment count query SELECT COUNT(*) FROM comment ... is altered as such: SELECT DISTINCT COUNT(*) FROM comment LEFT JOIN node_access .... This results in twice the number of comments for two roles, three times as many comments for three roles and so on.
The visible impact this has, is that the comment pager get's broken, because it adds superfluous pages for the non existing comments.
By hacking core like this, i.e. removing the optimized count query and using SELECT COUNT(*) FROM subquery, the problem goes away. But that's not so nice and might have a performance penalty.
diff --git a/modules/comment/comment.module b/modules/comment/comment.module
index 239cd5b..4da98cc 100644
--- a/modules/comment/comment.module
+++ b/modules/comment/comment.module
@@ -828,17 +828,8 @@ function comment_get_thread($node, $mode, $comments_per_page) {
->addMetaData('node', $node)
->limit($comments_per_page);
- $count_query = db_select('comment', 'c');
- $count_query->addExpression('COUNT(*)');
- $count_query
- ->condition('c.nid', $node->nid)
- ->addTag('node_access')
- ->addTag('comment_filter')
- ->addMetaData('node', $node);
-
if (!user_access('administer comments')) {
$query->condition('c.status', COMMENT_PUBLISHED);
- $count_query->condition('c.status', COMMENT_PUBLISHED);
}
if ($mode === COMMENT_MODE_FLAT) {
$query->orderBy('c.cid', 'ASC');
@@ -851,7 +842,6 @@ function comment_get_thread($node, $mode, $comments_per_page) {
$query->orderBy('torder', 'ASC');
}
- $query->setCountQuery($count_query);
$cids = $query->execute()->fetchCol();
return $cids;
Comments
Comment #1
xjmSetting NR so the bot runs the test suite. We're expecting a fail here.
Comment #3
Niklas Fiekas commentedFixed the notices. Hope it fails for the reason it should, now.
Comment #4
Niklas Fiekas commentedExperimenting ...
Edit: This makes sure that it's actually viewing the node page and has access to the comments section. Getting crazy here ... but it passed again. Now I can only hope it fails on your machine, too.
Comment #5
Niklas Fiekas commentedOne last try for tonight: Create 60 comments so that we can assert that two pages exists, but no third.
In the mean-time: These look similar: #135722: Taxonomy Access Control causes paging error on front page (Drupal 5) closed as duplicate of #668268: Weird behavior of pager for non-admin users (Drupal 6), where someone explains, that it has to do with the role count.
Comment #7
Niklas Fiekas commentedHehe, think I've got it now. I am using clean URLs locally, so testing for
?page=is fine, but the testbot has no clean URLs and thus&page=. Using justpage=now.Comment #9
Niklas Fiekas commentedYay! Just as it should.
Comment #10
Niklas Fiekas commentedThis is the query after all altering:
Comment #11
Niklas Fiekas commentedPort of the test to Drupal 8 core with a little debugging in the pager attached.
And the debug output:
Comment #12
xjmAlright, here's a modified version of the test in #11 that reproduces the bug by adding a duplicate grant realm for the permission. Might want to create a separate grant realm for this test, but for now it's just a demonstration of the core bug. We expect a fail here.
Thanks @Niklas Fiekas for all your work on this!
Comment #14
xjmSee: #1196292: Allow Pager count queries to utilise GROUP BY
Comment #15
Niklas Fiekas commentedThought about that one. No problem since we're building our own count query anyway.
Comment #16
xjmrevoof a.k.a @Niklas Fiekas confirmed that adding
GROUP by realm, gidto the query resolves the issue. However, the interesting thing here is that I believe the base functionality is provided bypager.inc, the queries are incomment_admin_overview()andcomment_get_thread(), and the node access alteration is coming from_node_query_node_access_alter(). Too many cooks...The best I can come up with is adding a query alter in comment.module to alter queries tagged with node access and using the pager. I'm not sure if there is a more generalized problem or solution for node access queries with the pager that I'm not seeing at the moment, but it's a start.
Comment #17
xjmAlright, here's a relatively simple fix. It is targeted only at this specific query.
Other things to check for:
admin/content/commenthave a similar bug?Other possible solutions (depending on above answers):
GROUP BY.GROUP BYwhen appropriate.Also, @Niklas Fiekas and I discussed ways to make the test coverage more complete. But for now... sending to the bot.
Edit: Mmm, just thought of a way that this might break everything... the alter is still running even when _node_query_node_access_alter() is returning without adding node access joins. Durr. Well, bot should tell.
Comment #19
xjmYay, I broke every single comment test except the one added by this issue. :)
Comment #20
xjmA patch which is slightly less likely to break everything. Kinda a hack, though... I added a special tag that says "please group meh!" in the node access query alter.
Comment #21
xjmWeird?
Comment #22
Niklas Fiekas commentedI like it. This is a hack for sure, but not a bad one to solve the issue in D7. In D8 #778050: Add support for database hints and make PagerDefault properly pluggeable might change paged queries, so that we no longer need a special count query for MySQL and PostGres and might fallback on subqueries for SQLite.
About the position of the groupBy: Even if we have the node_access_groupby tag, can we be sure the node_access table is always joined?
Having admin/content/comment, I am looking how that is build now. On a default setup, where you have the privileges to see that, you often have bypass access privileges, too, so problems there would be hard to notice.No problem there, that view is using the automatically generated subquery solution, instead of a custom count query.
Comment #23
xjmPer revoof, we're at least going to check for
$access_aliasto be set before adding the GROUP BY, to make it a little more robust. I still wonder if we can fix the base query instead, though.Kjartan also pointed out this issue: #1237380: Forum count incorrect when using access control modules, now for D7
Comment #24
xjmchx suggested this might be a dupe of: #681760: Try to improve performance and eliminate duplicates caused by node_access table joins
and referenced: http://www.xaprb.com/blog/2006/04/30/how-to-optimize-subqueries-and-join...
Comment #25
xjmrevoof and I both confirmed that #681760: Try to improve performance and eliminate duplicates caused by node_access table joins resolves this issue.
Comment #26
Niklas Fiekas commentedWrong issue again ... uploading the files anyway to avoid having to redo it in the other issue.
Comment #27
xjmComment #27.0
xjmAdd workaround.