In the search module there is a part with this code:

// Find words that were deleted from search_index, but are still in
// search_total. We use a LEFT JOIN between the two tables and keep only the
// rows which fail to join.
$result = db_query("SELECT t.word AS realword, i.word FROM {search_total} t LEFT JOIN {search_index} i ON t.word = i.word WHERE i.word IS NULL");
while ($word = db_fetch_object($result)) {
db_query("DELETE FROM {search_total} WHERE word = '%s'", $word->realword);
}

I found the first used query (SELECT ...) in my mysql-slow-queries-log with a execution time of more than 600 seconds on my server... The while loop was never executed due to timeouts...

I suggest to replace it with a set of queries that is much faster:

$sql = 'DROP TABLE IF EXISTS tmpsearch';
db_query($sql);
$sql = 'CREATE TABLE tmpsearch(word VARCHAR(50), del INTEGER(1) UNSIGNED)';
db_query($sql);
$sql = 'INSERT INTO tmpsearch (word, del) SELECT word, 1 FROM {search_total}';
db_query($sql);
$sql = 'ALTER TABLE tmpsearch add index i1(word)';
db_query($sql);
$sql = 'UPDATE tmpsearch t, {search_index} i SET t.del = 0 WHERE t.word = i.word';
db_query($sql);
$sql = 'ALTER TABLE tmpsearch add index i2(del)';
db_query($sql);
$sql = 'DELETE FROM tmpsearch WHERE del = 0';
db_query($sql);
$sql = 'DELETE FROM {search_total} WHERE word IN (SELECT word FROM tmpsearch)';
db_query($sql);

Comments

killes@www.drop.org’s picture

Version: 4.7.5 » 6.x-dev

I think I like this. Can you provide a patch?

Not sure we will backport this, but we should try to get it into Drupal 6.

FiReaNGeL’s picture

Note that we plan to drop mysql 4.0 support in 6.x; could we use a subquery instead of creating a temporary table?

chx’s picture

I like the principle, maybe not the exact queries. But. UPDATE tmpsearch t, {search_index} i SET t.del = 0 WHERE t.word = i.word this one is wrong, multitable update is mysql specific and totally unneeded as you change only one table, use a subselect (new feature of D6).

Noki’s picture

>Can you provide a patch?

Sorry, I won't. I'm new to drupal.

>could we use a subquery instead of creating a temporary table?

Sorry, but that's the point, subqueries on big datasets use lot's of memory and slow things down.

chx’s picture

Status: Active » Closed (duplicate)
sushiGer’s picture

Version: 6.x-dev » 5.7

Hi,

is there any plan to solve that problem in Drupal 5?
We have exactly the same problem and as we run several drupal installations it really kills our server from time to time...

naught101’s picture

subscribing