request only needed information to reduce bandwidth/response-time

I talked about this with Jacob some - and ideally we would also use the Solr highlighting feature to return a snippet, rather than the full body. That would be a big help in terms of bandwidth - returning a ~1k snippet, rather than potentially 10, 100, or 1000 kB of the body.

The Solr highlighting wasn't working, for some reason - needs investigation.

we can make some changes to solrconfig.xml like so:

  <requestHandler name="standard" class="solr.SearchHandler" default="true">
    <!-- default values for query parameters -->
     <lst name="defaults">
       <str name="echoParams">explicit</str>
       <str name="hl">true</str>
       <str name="hl.fl">body</str>
       <str name="hl.mergeContiguous">true</str>
     <!-- instructs Solr to return the field itself if no query terms are
          found -->

       <str name="f.body.hl.alternateField">body</str>
       <str name="f.body.hl.maxAlternateFieldLength">256</str>
       <str name="f.text.hl.fragmenter">regex</str> <!-- defined below -->
      </lst>
  </requestHandler>

and

   <!-- Configure the standard formatter -->
   <formatter name="html" class="org.apache.solr.highlight.HtmlFormatter" default="true">
    <lst name="defaults">
     <str name="hl.simple.pre"><![CDATA[<strong>]]></str>
     <str name="hl.simple.post"><![CDATA[</strong>]]></str>
    </lst>
   </formatter>
  </highlighting>
  

to highlight with strong rather than em tags

looking at the highlighting results, I think we should be running strip tags before indexing the body. Otherwise Solr can return broken markup.

I think we do have a handler that removes HTML for Solr - but it seems like Solr uses that html-stripped version for indexing, not for the results it returns. Need to double-cehck the solr docs on this.

http://markmail.org/message/qflbmxps4d5eiih2#query:solr%20HTMLStripWhite...
basically as above - stripped for indexing, not for results

The suggestions here is to strip html before indexing:

http://www.mail-archive.com/solr-user@lucene.apache.org/msg06465.html
http://markmail.org/message/qflbmxps4d5eiih2#query:solr%20HTMLStripWhite...

So, looks like we could use a default fl parameter something like fl=id,title,comment_count,type,changed,score,url,uid,name and get a Solr-generated snippet of the body back in the highlighting.

the PHP Response class doesn't specifically handle highlighting, so we'd have to grab it out of the parsed data

$response->highlighting

which is keyed by document id. So, something like:

$snippet = implode(' ... ', $response->highlighting->$id->body);

there are a lot of whitespace changes which make it a little hard to read

@Jacob:

I think we should put all the highlighting defaults into solrconfig.xml:

+        //Highlighting settings
+        $params['hl'] = 'true';
+        $params['hl.fragsize']= variable_get('apachesolr_textsnippetlength', 100);
+        $params['hl.simple.pre'] = variable_get('apachesolr_highlightpretag', '<strong>');
+        $params['hl.simple.post'] = variable_get('apachesolr_highlightposttag', '</strong>');
+        $params['hl.snippets'] = variable_get('apachesolr_numsnippets', 3);
+        $params['hl.fl'] = 'body';

Is overriding them going to be a common use case?

I think we should simply avoid sending in the URL anything that's going to match the defaults in solrconfig.xml to reduce the bansdwith per search request.

patch in #17 is empty

Patch to cleanup - mostly whitespace/style changes, except this also populates $doc->body with the snippet if that's not one of the fl fields.

Also adds nid as a fl field.

another cleanup needed perhaps - we don't actually have a $node

committed this patch