Render $node->teaser before sending it to Solr

ApacheSolr users the Drupal search module api.

You will have to implement this to do what you are asking AFAIK.

http://api.drupal.org/api/function/template_preprocess_search_result/6

HTH,
Jacob

I am reopening this because I don't really understand why we are storing a different teaser in solr.

Instead of storing the teaser as created by Drupal, we create a simplistic teaser by making a copy from body, as specified in schema.xml:

<copyField source="body" dest="teaser" maxChars="300"/>?

This is a problem because the drupal teaser can be very different from the first part of the body, either because specified as a "split summary" or because of cck fields that might appear in the body but not in the teaser, or whatever other reason. So basically the teaser can be completely unrelated to the body.

The thing is that it is actually arguably simpler to store the regular teaser in solr. The drupal teaser is already being retrieved with everything else during indexing, in the apachesolr_node_to_document funtion in apachesolr.index.inc, so all we have to do to get the drupal teaser in the solr database is not specify the above copyField in the schema.xml, and populate $document->teaser in apachesolr_node_to_document just as we populate the $document->body, $document->type, etc.:

...
$document->body = apachesolr_clean_text($text);
$document->teaser = apachesolr_clean_text($node->teaser); // We just add this to populate the teaser
$document->type  = $node->type;
...

I am new to this so am I missing something here? Is there another reason why the developers of this wonderful module chose to go the copyField route?

I think, we are only using this "fake" teaser in the case where there is no highlighting result.

If you want to normally return some other teaser, you can use a dynamic field.

Yes, I understand that it is only used when results have no highlighting...

My question is really why create and store a "fake" teaser when we can just store the "real" one that most people (I think) would expect to see when the teaser is returned instead of the snippet with highlights.

Obviously this is a small matter, and easily customized, so feel free to mark it "by design"...

@dawansv: Maybe this can help:

You can write your own theme function in order to make search results look like you want them too, no need to modify ApacheSolr module.

Here's what we have in our template.php in our theme: (change "mytheme" in the function name to your theme's name, and remember to clear caches before trying it out)

function mytheme_preprocess_search_result(&$variables) {
  if ($variables['result']['node']->nid) {
    $node = node_load($variables['result']['node']->nid);
    $variables['snippet'] = node_view($node, TRUE, FALSE, FALSE); // node_view($node, $teaser = FALSE, $page = FALSE, $links = TRUE)
    $variables['info'] = '';
    $variables['title'] = '';
  }
}

This way we are showing a full node fetched from the DB. (we are not doing any highlighting).

By the way, ApacheSolr is *just* following Drupal core's functionality =) You'd need to do the above in Drupal's core search too.

Janusman:

Thanks for sharing your code to show another way to get the teaser in the search result. Note that this forces accessing the database instead of just getting the data from solr, so it adds some overhead over just getting the teaser from the solr index.

That said I still stand by the fact that I don't see the point of storing a "fake" teaser in solr instead of just the regular drupal one...

Anyway, I should stop flogging that dead horse ;-) so I am marking this by design...

I'm reopening this as i'd also like to know why the need to store a fake teaser rather than the real one?

This also prevents Apache Solr Views from having a good teaser field handler since the data isn't available for it.

Loading the teaser from the database for each node in the result seems too expensive an operation when there is no real need for it.

The patch to change this behavior is very simple - is there any reason why it shouldn't be done?

@smoothify - where is the patch?

I'm not necessarily opposed to this change, but it means sending over more data and thinking about the handling of the highlighting config.

@pwolanin, I will upload a patch shortly, I have made the change on my dev machine and just need to get back to a vanilla apache_solr.

It does of course require a change to the schema.xml file, to avoid the copying from the body field.

I will also require a change to the solrconfig.xml if you want the teaser versus the leading body text when no highlighting is found - in fact we likely should have changed this already:

   <!-- instructs Solr to return the field itself if no query terms are
        found -->
     <str name="f.body.hl.alternateField">body</str>
     <str name="f.body.hl.maxAlternateFieldLength">256</str>

It would be nicer to have a conditional copyField. oh well.

Ok, here is the patch for version 6.x-1 from CVS.

I purposely didn't run the teaser through apachesolr_clean_text() as since it is not being indexed for querying its good to keep any tags for display purposes.

I removed the copyField from schema.xml as currently a conditional copyField is outside my Solr knowledge :)

I also included the config change in solrconfig.xml.

I'm cleaning the teaser: we do clean everything else, after all. http://drupal.org/cvs?commit=359608

I believe that is a bug. It is possible to edit your content type and remove the 'body' label. By doing this you remove the teaser and the body field from the node.

http://img.skitch.com/20100502-jpe4h1m2d1pr8wsw925ypywbaj.jpg

Ok, this should fix it. http://drupal.org/cvs?commit=362634

    if (isset($node->teaser)) {
      $document->teaser = apachesolr_clean_text($node->teaser);
    }

This is wrong because $node->teaser contains the raw teaser which has not been built via node_build_content($node, TRUE). When using modules providing filters like img_assist or codefilter, it causes snippets like [img_assist|nid=1|title=img|desc=|link=none|align=left|width=100|height=78] to appear in the search results.

Here is an attempt at fixing this. To reproduce, I've installed img_assist, placed an image in a teaser, and did a search on a keyword contained in the title only. I didn't see the raw img_assist snippet (like before) but instead the title of the image. Searching for a word contained in the node lead to the highlighted search result. I wish there was a way to use node_build_content() without dirtying the whole $node object but for now we have to live with drupal_clone()...

Looks reasonable - but we don't cache the built teaser?

also - if the teaser is only for highlighting, do we want to use a different build mode?

This change would mean that the teaser is always set, right, so we need to change this code:

85 	 if (isset($node->teaser)) {
86 	$document->teaser = apachesolr_clean_text($node->teaser);
87 	}
88 	else {
89 	$document->teaser = truncate_utf8($document->body, 300, TRUE);
90 	} 

Some alternative solution:

/** 
 * Implementation of hook_apachesolr_update_index(). 
 */
function hook_apachesolr_update_index(&$document, $node) {
  $document->teaser = $node->teaser;
}

seems like there are unanswered questions

+1 thanks...using alternative solution in the meantime.

Note issues raised in #20 and #21. D7 seems to suffer the same issue. In D6, why do we replace the $node with node_build_content()? See the issues raised around this practice in #852186: delete index + re-index: taxonomy filters (facets) no longer display. D7 doesn't replace $node like that. #21 is the patch to review for D6. Fix in D7 first.

Is this issue still relevant ?

Current code :

//  $entity = entity_load($entity_type, array($item->entity_id), array(), TRUE);
//  $entity = $entity ? reset($entity) : FALSE;
//this entity is being sent to the following function for nodes as $node

function apachesolr_index_node_solr_document(ApacheSolrDocument $document, $node, $entity_type, $env_id) {
  // None of these get added unless they are explicitly in our schema.xml
  $document->label = apachesolr_clean_text($node->title);

  // Build the node body.
  $build = node_view($node, 'search_index');
  // Why do we need this?
  unset($build['#theme']);
  $text = drupal_render($build);
  $document->content = apachesolr_clean_text($text);
  if (isset($node->teaser)) {
    $document->teaser = apachesolr_clean_text($node->teaser);
  }
  else {
    $document->teaser = truncate_utf8($document->content, 300, TRUE);
  }

Closing this issue since it is not relevant anymore and nobody seemed to reply or have additional interest

Just cross posting a related issues here, if someone comes across this: #2111363: Support for entity view modes