Grant Ingersol (Solr Maintainer) announced that they were scheduling a January release of 1.4

Since it is mostly stable and has several features (http://issues.apache.org/jira/browse/SOLR/fixforversion/12313351) that we want, and since this module is not looking at a 1.0 release before Jan, it seemed prudent to switch to trunk now.

In addition, there is a new version of the SolrPHPClient which supports 1.4 and has new APIs for boosting and balancers. In order to stay in sync, I've included a patch for that here. Note that the patch contains huge white space changes, making it almost impossible to read. This is because when we added the client to the module, I think the spaces were switched to drupal style.

I propose in the future that we keep the external library external and do not modify its spacing / style so that we can read diffs that come from the author.

CommentFileSizeAuthor
#8 diff_between_stock_solrconfig_and_drupal.diff10.78 KBJacobSingh
#8 solr_14_and_new_solr_php_client.diff128.14 KBJacobSingh
#3 new_php_client_and_solr_trunk.diff118.14 KBJacobSingh
new_php_client_and_solr_trunk.diff113.63 KBJacobSingh

Comments

robertdouglass’s picture

robertdouglass commented

For the attachments I think it might be worth it. We should at least maintain this patch.

JacobSingh’s picture

JacobSingh commented

There is a rather strange bug in the new SolrPHPClient

public function setMultiValue($key, $value, $boost = false)
	{      
		$this->addField($key, $value, $boost);
	}

For some reason, even when $boost is not passed, in addField receives boost as a null (not a false) and therefor pukes on indexing.

See the backtrace

#0  Apache_Solr_Document->addField(tid, 6, ) called at [/private/tmp/apachesolr.old/SolrPhpClient/Apache/Solr/Document.php:171]
#1  Apache_Solr_Document->setMultiValue(tid, 6) called at [/Users/jacob/work/drupal/ad1/sites/all/modules/apachesolr/apachesolr.module:545]
#2  apachesolr_add_taxonomy_to_document(Apache_Solr_Document Object ([] => ,[] => Array ([id] => bde8bf81b6057a5ebf974837c87a0896/node/84,[site] => http://ad/,[hash] => bde8bf81b6057a5ebf974837c87a0896,[url] => http://ad/node/84,[nid] => 84,[status] => 1,[uid] => 39,[title] => Saluto Interdico Accumsan Paratus Roto Iusto Vicis Immitto,[body] => node (page) - Quia eros cui capto. Utinam minim tego luptatum utrum dolore. Roto occuro antehabeo quae validus quae distineo. Mauris secundum caecus euismod neo autem ibidem iaceo. Venio cui utrum veniam adipiscing paulatim. Hos modo proprius eligo duis luctus. Magna wisi qui roto inhibeo lobortis nibh verto. Volutpat quis iusto praemitto nimis sit.

I can of course fix this with a conditional to turn null into false, but 

public function addField($key, $value, $boost = false)

Looks like it shouldn't be needed.

Anyone know what the deal is here? If I make null = false in that function, indexing works, if not, it fails. This needs to be fixed at the client level, so when we identify why, we should pass a patch upstream.

JacobSingh’s picture

JacobSingh commented
StatusFileSize
new new_php_client_and_solr_trunk.diff118.14 KB

Okay, here is another patch (which seems to work).

I had to make a few modifications to the client. After we get this in, we should consider fixing / using the client implementation of field boosts instead of re-creating it in Drupal if it makes sense.

JacobSingh’s picture

JacobSingh commented
Status: Active » Needs review

status change.

pwolanin’s picture

pwolanin commented

@Jacob - those are index-time boosts or query-time boosts?

I doubt we will make much use of index-time boosts, especially not for individual fields (vs. the whole doc).

JacobSingh’s picture

JacobSingh commented

They are index time boosts...

I don't know if we will actually use them, but that isn't the point. The null values were creating:

<field name="title" boost="">

which was causing Solr to throw a nice cryptic error I had to search the source for :) so I just fixed that problem. We may want to have index time boosts at some point, but I don't know when and I don't know the advantages other than performance... I'm not sure if the client provides for query time boosts (don't think so).

We should get this in ASAP otherwise it will become a royal PITA to continue merging. Are you seeing any problems after applying this patch? I guess for BC sake, we should also provide the old solrconfig.xml as well. The new one contains FastLRUCache which the old one didn't, so it might not start (didn't test).

Best,
Jacob

vladimir.dolgopolov’s picture

vladimir.dolgopolov commented

Concerning this lines:
//@note: JacobSingh changed this to is_int because off null / false confusion

I believe

if (is_int((float)10)) echo 'GOTCHA!';

produces nothing.

We see many (float) casts in Apache_Solr_Service class. So we have to check is_float() instead of is_int().
Now fields are not boosted in xml POST.

JacobSingh’s picture

JacobSingh commented
StatusFileSize
new solr_14_and_new_solr_php_client.diff128.14 KB
new diff_between_stock_solrconfig_and_drupal.diff10.78 KB

Okay, here is another go at this.

I've merged in the new stuff from the solr trunk into solrconfig.xml and also added the handler for attachments (as of yet unimplemented). It checks out and seems to work great!

I've attached a diff between our new solrconfig.xml and the one from the Solr trunk for reference as well.

pwolanin’s picture

pwolanin commented

committed the above as the basis for further testing and verification.

pwolanin’s picture

pwolanin commented

Something a little odd in the patch here:

-  <requestHandler name="drupal" class="solr.SearchHandler" default="true">
+  <requestHandler name="partitioned" class="solr.SearchHandler" default="true">
pwolanin’s picture

pwolanin commented
Status: Needs review » Fixed
pwolanin’s picture

pwolanin commented
Status: Fixed » Closed (fixed)