-9% of the site has been indexed

Is this before or after running cron? Do you get to 100% eventually?

hmm. What about a bigger dataset, do you get negative or wrong values when only partially indexed as well?

I seem to remember seeing this one time when I first tried out D5. I haven't seen it since.

I have been unable to reproduce this. I've tried several combinations of the number of content items which needed to be indexed. I also changed the number to index on each cron run to try leaving more or less items remaining. Before cron ran, I always saw 0%. After, the numbers were always correct.

To reproduce, create some unpublished nodes.

ahh, unpublished nodes. That'd make sense.

In that case, search.module should just suppress displaying any number lower than 0 on that page then I guess?

The node module counts the total items to be indexed like this:

SELECT COUNT(*) FROM {node} n WHERE status = 1

and the remaining items like this:

SELECT COUNT(*) FROM {node} n LEFT JOIN {search_dataset} d ON d.type = 'node' AND d.sid = n.nid WHERE d.sid IS NULL OR d.reindex <> 0

What about changing the last query to

SELECT COUNT(*) FROM {node} n LEFT JOIN {search_dataset} d ON d.type = 'node' AND d.sid = n.nid WHERE status = 1 AND ( d.sid IS NULL OR d.reindex <> 0)

The {node} table has an index (node_status_type) on the status,type,nid columns, so performance shouldn't be a problem. I can provide a patch a soon as I convinced PDT not to eat up all my memory.

Changed query to

SELECT COUNT(*) FROM {node} n LEFT JOIN {search_dataset} d ON d.type = 'node' AND d.sid = n.nid WHERE n.status = 1 AND ( d.sid IS NULL OR d.reindex <> 0)

Tested with 124 nodes, 50 of which where unpublished:

0% of the site has been indexed. There are 74 items left to index.
33% of the site has been indexed. There are 49 items left to index.
83% of the site has been indexed. There are 12 items left to index.
100% of the site has been indexed. There are 0 items left to index.

Should be RTBC if no one comes up with a different solution.

Marked http://drupal.org/node/233941 as a duplicate.

I would think its better to change the first part of the code to

SELECT COUNT(*) FROM {node}

As opposed to the second part - during the cron task, the items that have a status not equal to 1 are still going to be indexed in the site.

@BTMash#12: only items will status equal to 1 will get indexed, so the patch in #10 makes complete sense.

However, bugs have to be fixed in HEAD first, than backported to DRUPAL-6. Here is the same patch for HEAD.

The patch is tiny, and makes perfect sense (it was a regression introduced in DRUPAL-6, DRUPAL-5 has the good WHERE condition), so marking this as ready to go in.

If that is the case, then node_update_index should also be changed so the code is also:

$result = db_query_range("SELECT n.nid FROM FROM {node} n LEFT JOIN {search_dataset} d ON d.type = 'node' AND d.sid = n.nid WHERE n.status = 1 AND (d.sid IS NULL OR d.reindex <> 0)", $limit);

Since right now, it indexes regardless of the status.

Similarly in your patch, it should be n.status = 1 as opposed to simply status = 1.

Ok, thanks for the clarification.

Indeed, node_update_index() index nodes regardless of the status, while node_search($op = 'search') only search published nodes.

So we really need to change the total node count query.

Fixed here: http://drupal.org/node/239196