The current implementation of DrupalApacheSolrService uses solr's JSON Response Writer to receive the response as json.
Unfortunately the json response does not contain all information compared to the other response writers available. That's a documented behavior (see section "JSON specific parameters"):
Using a JSON object (essentially a map or hash) for a NamedList results in the loss of some information.
In Apache Solr Multilingual we support multiple spell checkers in different languages. If you use the standard response writer the corresponding part in the XML looks like this:
<lst name="spellcheck">
<lst name="suggestions">
<lst name="intolerans">
<int name="numFound">1</int>
<int name="startOffset">0</int>
<int name="endOffset">10</int>
<arr name="suggestion">
<str>Intoleranz</str>
</arr>
</lst>
</lst>
</lst>
<lst name="spellcheck">
<lst name="suggestions">
<lst name="intolerans">
<int name="numFound">1</int>
<int name="startOffset">0</int>
<int name="endOffset">10</int>
<arr name="suggestion">
<str>intolerance</str>
</arr>
</lst>
</lst>
</lst>
The example shows a German and an English correction of "intolrans". Using XML it's no problem to access both suggestions.
If you use JSON, all suggestions are present in the result, but json_decode() collapses them and the latest element wins:
"spellcheck":{
"suggestions":{
"intolerans":{
"numFound":1,
"startOffset":0,
"endOffset":10,
"suggestion":["Intoleranz"]}}},
"spellcheck":{
"suggestions":{
"intolerans":{
"numFound":1,
"startOffset":0,
"endOffset":10,
"suggestion":["intolerance"]}}},
An interesting point is, how the available PHP Response Writers deal with that. Both, the php and phps responses automatically add counters to identically named elements:
'spellcheck'=>array(
'suggestions'=>array(
'intolerans'=>array(
'numFound'=>1,
'startOffset'=>0,
'endOffset'=>10,
'suggestion'=>array('Intoleranz')))),
'spellcheck 1'=>array(
'suggestions'=>array(
'intolerans'=>array(
'numFound'=>1,
'startOffset'=>0,
'endOffset'=>10,
'suggestion'=>array('intolerance')))),
a:1:{s:11:"suggestions";a:1:{s:10:"intolerans";a:4:{s:8:"numFound";i:1;s:11:"startOffset";i:0;s:9:"endOffset";i:10;s:10:"suggestion";a:1:{i:0;s:10:"Intoleranz";}}}}s:12:"spellcheck 1";a:1:{s:11:"suggestions";a:1:{s:10:"intolerans";a:4:{s:8:"numFound";i:1;s:11:"startOffset";i:0;s:9:"endOffset";i:10;s:10:"suggestion";a:1:{i:0;s:11:"intolerance";}}}}}
So I see different ways to solve this issue:
1. Use a better response writer than json in general. (Heavy rewrite and API changes)
2. Create patches for solr itself to let the json response writer behave like the php response writers. (Might be the best solution in the future but might not solve the issue for older solr 1.x and 3.x versions)
3. Fake a valid json result on our side.
I see two ways to implement solution 3 until solution 2 will be available. I will post a patch soon.
| Comment | File | Size | Author |
|---|---|---|---|
| #5 | 1924448_safe_json_response.patch | 3.1 KB | mkalkbrenner |
| #4 | 1924448_save_json_response.patch | 3.1 KB | mkalkbrenner |
| #2 | 1924448_save_json_response.patch | 3.1 KB | mkalkbrenner |
| #1 | 1924448_save_json_response.patch | 3.1 KB | mkalkbrenner |
Comments
Comment #1
mkalkbrennerHere's my quick workaround to make just search requests "save".
Comment #2
mkalkbrennerenable test bot
Comment #4
mkalkbrennerComment #4.0
mkalkbrennerfixed indention
Comment #5
mkalkbrennerfixed typo: safe instead save
Comment #6
nick_vhSo, looking at this I don't really advocate the phps.
Other issues about this :
http://drupal.org/node/1338342
http://drupal.org/node/396548
Quote :
Solution?
Here's some random code I'm making to overcome this issue. Since json is plain text and solr seems to not respect the json standards, we can fix this issue ourselves before parsing the json.
Comment #7
nick_vhComment #8
mkalkbrennerUsing xml will solve the issue as well, but my edge case is not worth that effort and we will lose performance.
If we only deal with the specific spell checking problem, we should move and adjust the code that is currently used in Apache Solr Multilingual to this module. That will increase performance a little because we avoid an additional json_decode:
Comment #9
nick_vhComment #10
pwolanin commentedI'd say this is really an issue with Solr, but the problem is documented in terms of the format.
Why not try a change to json.nl=arrarr ? The steps to reproduce this are also unclear - can you create a simple test case?
I'm rather opposed to taking in serialized PHP.
Comment #11
mkalkbrennerI think we should stay with json as it is. The spell checker is a real edge case. I introduced a new class DrupalApacheSolrMultilingualService that solves that issue for Apache Solr Multilingual individually:
http://drupalcode.org/project/apachesolr_multilingual.git/commit/1b98526
Comment #11.0
mkalkbrennertypo