Problem/Motivation

I had "Cron run exceeded the time limit and was aborted." errors in log file.
I found that one node is used as CCK filed select more than 40000 times. And in table search_dataset in field "data" I could see long string with title of node - almost 40000 times.

search_cron() -> module_invoke($module, 'update_index') -> node_update_index() -> _node_index_node($node) -> node_invoke_nodeapi($node, 'update index') -> search_nodeapi(&node, 'update index', ...)

In search/search.module function search_nodeapi line 634:

    case 'update index':
      $result = db_query("SELECT caption FROM {search_node_links} WHERE nid = %d", $node->nid);
      $output = array();
      while ($link = db_fetch_object($result)) {
        $output[] = $link->caption;
      }
      if (count($output)) {
        return '<a>('. implode(', ', $output) .')</a>';
      }

As result of that $output has ~40000 captions from same node!
After that search trying parse all 40000 words in search_index() - here is server is slowly dying.

Proposed resolution

Simple solution - add DISTINCT in SELECT and live happy with 1-2 captions instead of 40000 after that field "data" in table search_dataset have captions only once:
In search/search.module function search_nodeapi line 634:

    case 'update index':
      $result = db_query("SELECT <strong>DISTINCT</strong> caption FROM {search_node_links} WHERE nid = %d", $node->nid);
      $output = array();
      while ($link = db_fetch_object($result)) {
        $output[] = $link->caption;
      }
      if (count($output)) {
        return '<a>('. implode(', ', $output) .')</a>';
      }
CommentFileSizeAuthor
#3 1403070-search-index-timeout.patch708 bytescameron tod

Comments

Lev_A’s picture

Correction:
$result = db_query("SELECT DISTINCT caption FROM {search_node_links} WHERE nid = %d", $node->nid);

jhodgdon’s picture

Title: (SOLVED) Large database - Cron run exceeded the time limit and was aborted. » Node with many links to it causes cron to run out of time in indexing
Version: 6.22 » 7.x-dev
Status: Needs review » Active
Issue tags: -search, -performance nodeapi, -cron error +Needs backport to D6

I'm looking through a lot of old issues... This one appears to be valid and could be addressed in Drupal 7.x and then backported to Drupal 6 maybe. The code involving link indexing is not present in Drupal 8.

cameron tod’s picture

Status: Active » Needs review
Issue tags: -Needs backport to D6 +search, +performance nodeapi, +cron error
StatusFileSize
new708 bytes

Let's see what testbot says!

cameron tod’s picture

Issue tags: +Needs backport to D6
jhodgdon’s picture

Status: Needs review » Needs work

This does not seem to break anything, but the patch would be more convincing if there was a test as well. Please make a test, and upload a patch with just the test so we can verify that the test fails. Then upload (or at the same time) a patch with both the test and the code patch, so we can verify that the test passes.

cameron tod’s picture

Version: 7.x-dev » 6.x-dev
Category: bug » support
Status: Needs work » Postponed (maintainer needs more info)

I think that we need to keep the original functionality, as it adds the anchor text keywords to the linked-to node's indexed text. That's important for search relevancy.

A better fix would be to limit the maximum amount of nodes that can be indexed in one run, via using the search_cron_limit variable.

The code you have pasted appears to be from Drupal 6 - can you confirm the version please?

jhodgdon’s picture

Version: 6.x-dev » 7.x-dev
Category: support » bug
Status: Postponed (maintainer needs more info) » Needs work

The original report was for 6.x, but we need to fix it in 7.x.... actually I think the comment on #6 perhaps was meant for a different issue? There are about 20 issues involving the mostly-broken link functionality. :(

Status: Needs work » Closed (outdated)

Automatically closed because Drupal 7 security and bugfix support has ended as of 5 January 2025. If the issue verifiably applies to later versions, please reopen with details and update the version.