Search optimization

I have run into a problem with the current search design.

First to the background. Running a small community with a forum, it often happens that people create forum threads like "The never ending story".
The author starts to tell a story and others continue it.

Well such threads will fast reach 10.000 comments, actually 60.000 for us. And by nature these threads will contain virtually any possible search term you could think of, so a search will almost always hit this node.

Actually the search engine is well designed and returns the result pretty fast, but still the search result won't display and you will end up with a white page (of death) after a while.

I found the reason for this being in the node_search() function, to be more precise here:

        // Fetch comments for snippet.
        $node->body .= module_invoke('comment', 'nodeapi', $node, 'update index');

This will call check_markup() for all of the 60.000 comments and will either run out of memory (based on how huge the comments / the installed filters are) or hit the PHP timelimit.

I think the better option would be to index the comments along the nodes into the search tables and let the search results display both nodes and comments separatly (if they hit the search terms).