Problem/Motivation
I had "Cron run exceeded the time limit and was aborted." errors in log file.
I found that one node is used as CCK filed select more than 40000 times. And in table search_dataset in field "data" I could see long string with title of node - almost 40000 times.
search_cron() -> module_invoke($module, 'update_index') -> node_update_index() -> _node_index_node($node) -> node_invoke_nodeapi($node, 'update index') -> search_nodeapi(&node, 'update index', ...)
In search/search.module function search_nodeapi line 634:
case 'update index':
$result = db_query("SELECT caption FROM {search_node_links} WHERE nid = %d", $node->nid);
$output = array();
while ($link = db_fetch_object($result)) {
$output[] = $link->caption;
}
if (count($output)) {
return '<a>('. implode(', ', $output) .')</a>';
}
As result of that $output has ~40000 captions from same node!
After that search trying parse all 40000 words in search_index() - here is server is slowly dying.
Proposed resolution
Simple solution - add DISTINCT in SELECT and live happy with 1-2 captions instead of 40000 after that field "data" in table search_dataset have captions only once:
In search/search.module function search_nodeapi line 634:
case 'update index':
$result = db_query("SELECT <strong>DISTINCT</strong> caption FROM {search_node_links} WHERE nid = %d", $node->nid);
$output = array();
while ($link = db_fetch_object($result)) {
$output[] = $link->caption;
}
if (count($output)) {
return '<a>('. implode(', ', $output) .')</a>';
}
| Comment | File | Size | Author |
|---|---|---|---|
| #3 | 1403070-search-index-timeout.patch | 708 bytes | cameron tod |
Comments
Comment #1
Lev_A commentedCorrection:
$result = db_query("SELECT DISTINCT caption FROM {search_node_links} WHERE nid = %d", $node->nid);
Comment #2
jhodgdonI'm looking through a lot of old issues... This one appears to be valid and could be addressed in Drupal 7.x and then backported to Drupal 6 maybe. The code involving link indexing is not present in Drupal 8.
Comment #3
cameron tod commentedLet's see what testbot says!
Comment #4
cameron tod commentedComment #5
jhodgdonThis does not seem to break anything, but the patch would be more convincing if there was a test as well. Please make a test, and upload a patch with just the test so we can verify that the test fails. Then upload (or at the same time) a patch with both the test and the code patch, so we can verify that the test passes.
Comment #6
cameron tod commentedI think that we need to keep the original functionality, as it adds the anchor text keywords to the linked-to node's indexed text. That's important for search relevancy.
A better fix would be to limit the maximum amount of nodes that can be indexed in one run, via using the search_cron_limit variable.
The code you have pasted appears to be from Drupal 6 - can you confirm the version please?
Comment #7
jhodgdonThe original report was for 6.x, but we need to fix it in 7.x.... actually I think the comment on #6 perhaps was meant for a different issue? There are about 20 issues involving the mostly-broken link functionality. :(