Improper results coming from hook_nodeapi, "update index" $op.

You are right. Its not that part of the code that make te duplicates in my text variable.

I found out, that the text in the table "search_dataset" contains the same duplicated data. This means, that this is not related with the apache solr update index method.

I'm debugging since 3 hours but its extremly slow. I'm waiting the whole time. If I find out, how these duplicates arise, I will post it.

I finally found it, but i have no solution. The code from above invokes the hook_nodeapi.

        // Fetch extra data normally not visible
        $extra = node_invoke_nodeapi($node, 'update index');
        foreach ($extra as $t) {
          $text .= $t;
        }

While debugging I found out, that the search_nodeapi returns the duplicated values.

In my case the following is returned:

<a>(node title node title node title node title node title )</a>

What is this update index function for? The result makes no sense. I have an author node and books node. On the books node is a link to the author node. This leads probably to this senseless output. If an author has 100 books, this will generate 100 repetitions because there are 100 links from book nodes.

Any ideas?

/**
 * Implementation of hook_nodeapi().
 */
function search_nodeapi(&$node, $op, $teaser = NULL, $page = NULL) {
  switch ($op) {
    // Transplant links to a node into the target node.
    case 'update index':
      $result = db_query("SELECT caption FROM {search_node_links} WHERE nid = %d", $node->nid);
      $output = array();
      while ($link = db_fetch_object($result)) {
        $output[] = $link->caption;
      }
      return '<a>('. implode(', ', $output) .')</a>';
    // Reindex the node when it is updated.  The node is automatically indexed
    // when it is added, simply by being added to the node table.
    case 'update':
      search_touch_node($node->nid);
      break;
  }
}

Dear Robert

I studied your search index documentation on the acquia.com website. It was very helpful for understanding the method pasted above.

The original idea was, that every link from other nodes is a helpful information for the search index. This is normally also true, but if you are using views with lists, you have normally a lot of links to other nodes. In other word, it will highly enlarge the search index with duplicated data.

My example: the author node is linked on every 100 book node with leads to 100 duplicated data.

If I understand your description on acquia.com correctly, every link is stored in the search_node_links table.

If we want to solve this problem from its roots, we should check if a nid/caption combination for a node already exists before inserting a new one.

sid | type | nid | caption
-----------------------
201 node 200 rowling
202 node 200 rowling

Means: on the node 201 and 202 is a link names "rowling" tothe node 200 = duplicated data.

Am I right?

I'm using embedded views. But also a lot of reference fields to other nodes.

@JakobSingh

I think this is a general issue and not specific to my configuration.

I try to make an example, that make it more clear:

1. In your database you have a parent node. Lets call this node "Author".

2. Furthermore you have child nodes called "Books" that belongs to an "Author"

3. On every "Books" node, you have a link to the "Author" node.

You can imaging, that there are a lot of such one-to-many relation in the real world.

If you create now the index, drupal search is very clever and consider every link title, that has been made from a child node to a parent node as an important information for the node.

That so far correct. But unfortunately if you have always the same link title, it will generate a lot of duplicated data.

To come back to my example: For every childnode, that has a link to the parent node, it will add the link title. Unfortunately, in one-to-many relations, the link title is always the same. So drupal will add a lot of duplicated data.

I'm not sure, but maybe this problem has to be addressed in the drupal core, because it affects also the normal search index.

To bypass or to solve this issue, link titles have to be checked for duplication. But I have no idea how to achieve this. Mabye with temporary hashtable or something like this.

Could someone think about my example and confirm it? If its true, we should change the title and move this to the core search queue.