Including Taxonomy Terms in search results.

-        $text = check_plain($node->title) .  $node->body;
+        $text = check_plain($node->title) .' '.  $node->body;

This part (and similar parts) have to get in, one way or another.

The taxonomy terms part needs to be dealt with separately.

Probably best to handle what is to be queried using a Dismax Solr query? Then it is not a matter of adding it during index, but at query time. This would work using the qf parameter.

@janusman: I've read the Dismax docs several times now. But. Could you please for me explain in detail why this is true?

Probably best to handle what is to be queried using a Dismax Solr query? Then it is not a matter of adding it during index, but at query time. This would work using the qf parameter.

Thanks!

I've addressed the space issue in DRUPAL-5 and DRUPAL-6--1

-        $text = check_plain($node->title) .  $node->body;
+        $text = check_plain($node->title) .' '.  $node->body;

The taxonomy terms issue remains.

Ok, let me rephrase this =)

I think there is no harm in actually adding the taxonomy terms to the "text" portion--as we already are adding all the node body, including probable "noise" like CCK field names; so I guess I approve the current patch...

What I meant was that, additionally, we can get much better relevancy scoring of results, by using dismax, and specifying per-field boosts. However this is a separate discussion and already in progress at #284923: Boost title relevancy (so you can ignore my previous comment, IMO proceed with the patch, and continue the other discussion in parallel) =)

MAYBE in the future we'll realize that to maximize what admins can control about what is searched, how any field can affect relevancy, etc. then we might realize that putting stuff (title, body, taxonomy... etc) into the "text" field might not be best. But for now at least we want 100% recall.

@janusman: got it. Assigning issue to self.

This needs a re-roll. What's this bit for?

+        if (isset($_GET['qt'])) {
+            $params['qt'] = $_GET['qt'];
+        }

Also, what is the goal of the patch? Is it to aid in the finding of results when searching for term names? Or is it the optical display of the search results? My feeling is that we already do just the right amount for terms and term names (especially with janusman's hierarchical taxonomy patch that will likely get committed soon), so this patch should be dealing with the optics. Perhaps we should simply (optionally) show taxonomy term names on search results?

I'm reconsidering whether this is critical (ie for 1.0 release). Please opine.

The goal seems to be to add the term names to the solr text field. Which puzzles me, because afaict terms already are included and found when searching the default field. Maybe this gets done via the hook_nodeapi('update index') call.
This would mean that this patch was rather useless in the current form, but I might be wrong.

In any case, I don't think this is critical for 1.0 (apart from the whitespace-between-strings-thing, which already got committed).

Agree with the monkey.

I second the monkey, although I would like to bring up the (not 1.0) conversation:

When people start wanting to index a site and then retrieve that index from another site, or from something else non-drupal, or when using a federated search, it would be nice to provide our rationalized data in way this is more portable. I realize that storing the term as {VOCAB}/{TERM} also has a downside because renaming terms becomes a Royal PITA, but something to consider...

Best,
Jacob

Does $node->body depend at all on theming functions? If so, should we ensure a "minimum" set of fields to index? (e.g. terms. =) I vote yes for highly-relevant fields: title, terms, etc.

In my case, on my current theme, $node->body does NOT contain any terms... here's a sample from a Solr Query (only the "body" field from one node):

<str name="body">
<div class='millennium coverimage'><img src="http://biblioteca.mty.itesm.mx/mty/portada/v2.php?id=3527301100&type=100&tit=Adhesives+and+adhesive+tapes&aut=Gierenz%2C+Gerhard%3B+Karmann%2C+Werner" alt="Adhesives and adhesive tapes" width=80 height=110/></div>
<div class='millennium authors'>por Gierenz, Gerhard; Karmann, Werner</div>
<div class='millennium extra'>Libro / 2001</div>
<div class='millennium holdings'><a href="/pasteur/es/node/124990">Mostrar disponibilidad</a></div>
<div class="fivestar-static-form-item"><div class="form-item">
 <div class="fivestar-widget-static fivestar-widget-static-5 clear-block"><div class="star star-1 star-odd star-first"><span class="off">0</span></div><div class="star star-2 star-even"><span class="off"></span></div><div class="star star-3 star-odd"><span class="off"></span></div><div class="star star-4 star-even"><span class="off"></span></div><div class="star star-5 star-odd star-last"><span class="off"></span></div></div>
 <div class="description"><div class="fivestar-summary fivestar-summary-"></div></div>
</div>
</div>
</str>

... and further down, the indexed taxonomy_name field:

<arr name="taxonomy_name">
<str>Chemical Engineering</str>
<str>Engineering</str>
<str>Engineering & Applied Sciences</str>
<str>Book</str>
<str>Adhesives</str>
<str>Adhesivos</str>
</arr>

So if we are searching only the Solr field "text" (the default) for "Engineering", this node wouldn't show up, because that's not in the node body (therefore, not in $text)

I vote for this other patch however, which not only includes terms but all parent terms in a hierarchic vocabulary (perhaps this case should still be marked "wont fix" and the other one will go through) =)

I can see ppl asking for support because terms aren't indexed because are using this or that theme... =)