Hi all

I have a severe problem with the latest version of apachesolr.

Normally when I search a wrong typed word, for example "spel", the search returned a spell suggestion called "spell" aka "Did you mean" "spell".

But now, the word "spel" is not found and solr return nothing, not even spell suggestions. I think, spell suggestions should be added to the result even if there was no matching word found.

I tried to to find the source of the problem but was not successful until now. Maybe someone has an idea how to resolve this.

Solr: 3.6.2
Drupal 7, Apachesolr 1.2
Solr config 3.x copied over to solr

The spellchecker index works: If I call this url, I get the correct spell suggestion:
localhost:8983/solr/select/?q=spel&spellcheck=true&spellcheck.build=true&version=2.2&spellcheck.extendedResults=true

Comments

ayalon’s picture

Ok I think, the problem is, that the spellchecker index is never build.
I see, that the spellchecker files in the data folder are empty.

I added the following parameter to the search query:
function mymodule_apachesolr_query_alter(&$query) {

$query->addParam('spellcheck.build', true);

}

With the next query, the spellchecker index was build.

Of cource, the build parameter should only be added once.
I checked the code of the apache solr module and I think, the spellchecker.build parameter is only added if the index is deleted.

But there must be a somewhere a switch that builds the spellchecker index after indexing some documents.

nick_vh’s picture

apachesolr_search_build_spellcheck is the function. Normally solr builds the spellcheck itself. Any suggestions on how to improve it?

ayalon’s picture

Hi Nick_vh

All my analysis is based on a new solr installation without a data directory. People who have a spellchecker, should delete the whole data directory and then have a look, if it is rebuild. With the latest version, the spellchecker is never built.

I also found the function "apachesolr_search_build_spellcheck" but I think, this function does not work. Because there is a try / catch or because it never gets executed. If you use the "Delete index" function in the UI, the spellchecker never gets deleted.

I made a small module, to manually rebuild the spellchecker index. I wanted to use the code from the function "apachesolr_search_build_spellcheck", but the code throws an error.

The Api has changed:

Code from apachesolr module:
$response = $solr->search('solr', 0, 0, $params);

Working code:
$response = $solr->search('solr', $params);

I made a custom module, that adds a button to the backend, to rebuild the spellchecker index:

function mymodule_form_alter(&$form, $form_state, $form_id) {
    case 'apachesolr_index_action_form':
      $form['action']['spellcheck'] = array(
        '#prefix' => '<div>',
        '#type' => 'submit',
        '#value' => t('Rebuild spellchecker'),
        '#submit' => array('mymodule_rebuild_spellchecker'),
      );
      break;
   }
}

function mymodule_rebuild_spellchecker(array $form, array &$form_state) {
   try {
    $solr = apachesolr_get_solr();

    kpr($solr);
    $params['spellcheck'] = 'true';
    $params['spellcheck.build'] = 'true';
    $response = $solr->search('solr', $params);

    $destination = array();
    if (isset($_GET['destination'])) {
      $destination = drupal_get_destination();
      unset($_GET['destination']);
    }
    $form_state['redirect'] = array('admin/config/search/apachesolr');

    drupal_set_message(t('Spellchecker rebuild'), 'warning');

   }
   catch (Exception $e) {
     watchdog('Apache Solr', nl2br(check_plain($e->getMessage())), NULL, WATCHDOG_ERROR);
   }
}

But this is not a real solution. If you add new documents to the index, the spellchecker index will not be updated. I searched for a hook in apache solr but there is only an hook_index_document that is not suitable for rebuilding the spellchecker index.

I searched for a possibility to rebuild the spellcheck index from the solr side but I did not found a solution.

ayalon’s picture

The is a possibility to rebuild the spellchecker index on commit:
http://wiki.apache.org/solr/SpellCheckComponent#Building_on_Commits

But I don't see a way how to add this line to the config file:
<str name="buildOnCommit">true</str>

There is no configuration available for this.

ayalon’s picture

Title: Spellchecker not working, if there is no result » Spellcheck index not built, no spellcheck suggestions available.
Priority: Normal » Major

Ok I got it. It's not in solrconfig.xml, it's in solrconfig_extra.xml...

<str name="buildOnCommit">true</str>

You have to add this line to the file, the it works as expected. The index is built on commit. I think this is a severe bug.

<lst name="spellchecker">
  <str name="name">default</str>
  <str name="field">content</str>
  <str name="spellcheckIndexDir">spellchecker</str>
  <str name="buildOnCommit">true</str>
  <!-- uncomment this to require terms to occur in 1% of the documents in order to be included in the dictionary
    <float name="thresholdTokenFrequency">.01</float>
  -->
</lst>
WoozyDuck’s picture

I still have the problem, using Solr 4.2.0 and installed 7.x-1.2 version of this module on Drupal 7.
Your solution didn't help me

pwolanin’s picture

Ah, the past setup was to rebuild the spellcheck index on optimize, and to do that once per day.

Running it per commit can be costly

ayalon’s picture

Hi Peter

If you take the current 7-x1.2 version:

Could you show me, how the spellchecker index is rebuilt once per day? Building on commit might be costly. But I don't see any code, that rebuilds the spellchecker index even once a day. I also noticed this on Acquia search. I have a customer where the spellchecker index is not built.

nick_vh’s picture

Priority: Major » Critical
nick_vh’s picture

Status: Active » Needs work

pwolanin has committed this to the common configs and we should verify if it works as expected and also verify 6.x-3.x

nick_vh’s picture

Status: Needs work » Fixed
nick_vh’s picture

Status: Fixed » Closed (fixed)

Closing to clean the issue queue a bit

theamoeba’s picture

+1

janusman’s picture

Issue summary: View changes
Priority: Critical » Major
Status: Closed (fixed) » Needs review
StatusFileSize
new776 bytes

Reopening. Problem from #3 is still present, meaning, deleting the index from the UI does not properly rebuild the spellchecker because of wrong arguments passed to search().

Attached patch has fix.

diff --git a/apachesolr_search.module b/apachesolr_search.module
index a94c0a9..bee5d9c 100644
--- a/apachesolr_search.module
+++ b/apachesolr_search.module
@@ -1607,9 +1607,11 @@ function apachesolr_search_form_apachesolr_delete_index_confirm_alter(&$form, $f
 function apachesolr_search_build_spellcheck($form, &$form_state) {
   try {
     $solr = apachesolr_get_solr();
-    $params['spellcheck'] = 'true';
-    $params['spellcheck.build'] = 'true';
-    $response = $solr->search('solr', 0, 0, $params);
+    $params = array(
+      'spellcheck' => 'true',
+      'spellcheck.build' => 'true'
+    );
+    $response = $solr->search('solr', $params);
   }
   catch (Exception $e) {
janusman’s picture

New patch, this needed more tweaking.

  • 043e2cf committed on 7.x-1.x
    Fixes #1973678: Spellcheck index not built
    
janusman’s picture

Status: Needs review » Fixed

Committed.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.