I've been seeing inconsistent indexing results and after 2 days of digging into the wrong end of Solr, I tracked it down to bad SQL...

Here's test cases and fix that works for me patched against 5.x-dev (0917).

Test case 1: sort order BY GREATEST(n.changed, c.last_comment_timestamp) ASC

nid timestamp
1 1219256347
2 1219256340
3 1219256348

Node 2 will not be indexed.

Test case 2: where ((GREATEST(n.changed, c.last_comment_timestamp) = %d AND n.nid > %d) OR

nid timestamp
1 1219256347
2 1219256347
3 1219256348

Node 2 will not be indexed.

CommentFileSizeAuthor
apachesolr_sql_fix.patch1.44 KBjohn.money

Comments

john.money’s picture

john.money commented

Note that the test cases above assume indexing 1 node per run.

JacobSingh’s picture

JacobSingh commented
Status: Needs review » Closed (duplicate)

I'm guessing this is a dupe of:
http://drupal.org/node/302378

We need to do some work on the patch from:
http://drupal.org/node/42277#comment-846833

If you're keen, please go ahead, if you think this is a separate issue, please close.

Thanks!
Jacob

john.money’s picture

john.money commented

First link fixes case 1 (order by) but does not fix case 2 (where)... I like my implementation better because there is no need to recalculate last_change when ordering.

Second link (node.module patch) has essentially the same SQL, fixing both cases... I like my implementation better because there is no need to recalculate last_change when ordering.

I'd be happy to mark this as closed, as long as we get _some_ solution into dev ASAP. I wasted a day+ on this (as apparently other people have to). If we can't trust that every node is being indexed, then I know my employer will be hesitant to use ApacheSolr on production and instead fallback on the current Google site search (yuck).

john.money’s picture

john.money commented
Status: Closed (duplicate) » Closed (fixed)

Moving suggested patch to other thread.