Geospatial searching with solr 3.x

Well, yes that would be nice, but need some solid and tested configuration stanzas to support geo fields in the context of the rest of our configuration.

Any work around this will require use of Solr 3.2+ on the server side.

Here is a Solr 3.2 war file (rename it from .zip to solr.war) built from the ASF lucene svn 3.2 tag with patches using javac 1.6.0_24. For issues/patches see https://issues.apache.org/jira/browse/SOLR

Note the patch for SOLR-2462 I generated from the commit, since it seems the final patch was never posted to the issue.

I've been planning on adding the geospatial capability to Solr starting in a couple weeks (when I finish a current project), so please keep me in the loop if you start anything.

FWIW I was only able to get this running with the following: http://drupal.org/node/1197212

Hi

SOLR 3.3 has been released which has the fixes to the bug mentioned in #3 and as well some performance optimization and new features

It also has new features like

"automaton-based, implementation of suggest (autocomplete) component, offering an order of magnitude smaller memory consumption compared to ternary trees and jaspell and very fast lookups at runtime "

The Complete release notes is in the link : http://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_3_3/solr/CHA...

So I think we should start integrating SOLR 3.3 with Drupal

Unfortunately, https://issues.apache.org/jira/browse/SOLR-2535 was not committed for 3.3

Since Solr 3.3 was released we can skip at least the spellcheck patch. The file handler patch is nice to have, but won't prevent Solr from otherwise working with Drupal.

Hi

I tried to implement the GEO spatial with Drupal using the PATCH #3 and with SOLR 3.2 and I was able to see GEO spatial working . Following are the Schema Changes and the Process I followed to make it work

SCHEMA Changes

     <fieldType name="point" class="solr.PointType" dimension="2" subFieldSuffix="_d"/>
     <fieldType name="location" class="solr.LatLonType" subFieldSuffix="_coordinate"/>
     <fieldtype name="geohash" class="solr.GeoHashField"/>
     <field name="coordinates" type="location" indexed="true" stored="true"/>
     <dynamicField name="*_coordinate"  type="tdouble" indexed="true"  stored="false"/>

STEP 1:

Next, I added a hook_apachesolr_update_index() function to a custom module to index Location CCK data into the Solr Document.

           function MYMODULE_apachesolr_update_index(&$document, $node) {
                // check for location data
                if (empty($node->field_location[0]['latitude']) || empty($node->field_location[0]['longitude'])) {
                    return;
                }    
                $document->coordinates = $node->field_location[0]['latitude'] . ',' . $node->field_location[0]['longitude'];  
          }

STEP 2 :

I then add a hook_apachesolr_modify_query() function to my custom module to override the Solr query params.

        function MYMODULE_apachesolr_modify_query(&$query, &$params, $caller) {
            if ($caller != 'apachesolr_search') {
               continue;
           }
           // NOTE: Hardcoded the Co-ordinates 
           // You can also get the user co-ordinates and send along with this
           $coordinates = '21.346617,-82.098747';  
           $distance = 5;  
           // modify params
           $params['fq'][] = "{!geofilt sfield=coordinates pt=$coordinates d=$distance}";  
       }

// The spatial query syntax has an augmented "fq" value, but the apachesolr Drupal module sets the fq parameter automatically (via: $query->get_fq()) , So for this I need to make a change to the apachesolr contrib module

     function apachesolr_modify_query(&$query, &$params, $caller) { 
         // TODO: The query object should hold all the params.
        // Add array of fq parameters.
       if ($query && ($fq = $query->get_fq())) {
           $params['fq'] = array_merge($fq, $params['fq']);
      }

IF STEP 2 isn't working For Testing you can change in the apachesolr views query.inc

      /**
       * Executes the query and fills the associated view object with according
       * values.
       * 
       * Values to set: $view->result, $view->total_rows, $view->execute_time,
       * $view->pager['current_page'].
       */
       public function execute(&$view) {
               // NOTE: hardcoding to check whether Distance Works  
	      $coordinates = '21.346617,-82.098747';
	      $distance = 5;
              $this->_params['sfield']='coordinates';
	      $this->_params['pt']= $coordinates;
	      $this->_params['d']= $distance;
	      $geo_params = "{!geofilt}";
	      array_push($this->_params['fq'],$geo_params);
       } 

And then I was able to see the Results show up based on Distance and I was able to do a SORTING based on Distance . The new spatial query syntax (documented here: http://wiki.apache.org/solr/SpatialSearch)

Hope this Helps

Thanks
Balaji

Right, so we cannot do:

<dynamicField name="*_coordinate"

because our other dynamic fields use prefixes. So we need to flip this around, maybe:

<dynamicField name="coordinate_*"

Hey there,

I was indeed able to get this to work. Thank you very much pwolanin and mcabalaji, this is beautiful. I changed some small things that allows apachesolr_modify_query() in apachesolr.module to remain as it was, however, a small unobtrusive change elsewhere in apachesolr.module AND apachesolr_search.module is needed.

MYMODULE.module

function MYMODULE_apachesolr_update_index(&$document, $node) {
  // check for node location
  //if (!empty($node->location['latitude'])&&!empty($node->location['longitude'])) {
  //  $document->coordinates = $node->location['latitude'] . ',' . $node->location['longitude'];
  //}

  // check for cck field location
  if (!empty($node->field_location[0]['latitude'])&&!empty($node->field_location[0]['longitude'])) {
    $document->coordinates = $node->field_location[0]['latitude'] . ',' . $node->field_location[0]['longitude'];
  }
}

function MYMODULE_apachesolr_modify_query(&$query, &$params, $caller) {
  if ($caller == 'apachesolr_search') {
    $params['fq'] = '{!geofilt}'; // filter query
    $params['sfield'] = 'coordinates'; // document field
    $params['pt'] = '21.346617,-82.098747'; // latitude,longitude 
    $params['d'] = 5; // distance
    $params['sort'] = 'geodist() asc'; // sort by distance
  }
}

Changed against 6.x-2.x-dev

apachesolr.module - line 1693

--  if ($keys == '' && isset($params['fq'])) {
++  if ($keys == '' && isset($params['fq']) && (is_array($params['fq']) || is_object($params['fq']))) {

apachesolr_search.module - line 261

--  if (isset($params['fq'])) {
++  if (isset($params['fq']) && (is_array($params['fq']) || is_object($params['fq']))) {

http://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_3_3/solr/CHA...

is out now. Let's target that as our next baseline from 1.4.x

Attached is a 3.3 war (rename as above) which includes patches SOLR-2535 and SOLR-232. The other bug is fixed in 3.3.

I personally think it would be awesome if Spatial Search became part of apachesolr itself, perhaps as a separate submodule but perhaps otherwise orchestrated via module settings. One thing I did not include in my post was a module that I've yet to release called Locator which will automatically try to determine the user's location and store it to their session. It gracefully degrades through HTML5 GeoLocation, Maxmind's GeoIP, and simple user input.

/me needs to finish and release that module one of these days.

@drupal_sensei - I'm not sure that schema snippet is correct.

Look in more detail at the Solr 3.3 example.

The _d suffix appears to refer to:

 <dynamicField name="*_d"  type="double" indexed="true"  stored="true"/>

we should not use a suffixed field like *_d. This will certainly conflict at some point with a prefixed field, though in theory all our existing prefixes are more specific.

Look at the example schema, instead of specifying a suffix, apparently you can specify a field type like:

<fieldType name="point" class="solr.PointType" dimension="2" subFieldType="double"/>
<fieldType name="location" class="solr.LatLonType" subFieldType="tdouble"/>

Though this will create a suffixed synamix field, it will be even more specific.

Also, I got some feedback about this from Yonick on the solr-user list, so you might read that exchange.

I must not have been at the same BoF - none of those summary points match what I remember discussing or suggesting.

I said we will not build any kind of schema builder, but rather add some markers to the schema to facilitate the multilingual module.

Solr 3.4 returns results but wont return a mix of location nodes and non-location nodes

Update: using 6.2.x and the modifications above Solr is now returning location and non-location nodes in its search result.
Still cant get proximity results only results based on relevancy
following #12 gave some errors and field_location latlong is not being indexed.

Update2: I got this working with 6.2.x.dev I documented my effort here
http://drupal.org/node/1334960

Here's a possible schema.xml that at at least allows me to index lat/lon.

The dynamic field prefixes aren't great, but not sure if I can come up w/ better.

How is this patch applied? Doesn't seem to match up with Drupal instructions for applying a patch.
Any help would be greatly appreciated

Thanks!

Oh okay, found it here: http://drupal.org/node/620014

3.4 is out...

Committing this patch as the basis for further refinement.

@drupal_sensei - that's for Solr 1.4 using an external geo library, not directly relevant to this issue.

Making this generic for the solr 3.x branch.

#1334960: SOLR 3.4 & D6.x indexing location nodes more info can also be found in this (closed) issue

Adding this to Contrib Component. Good for someone to pick up to develop a contributed module and marking postponed until someone picks it up

Won't fix here. See apachesolr_geo or apachesolr_location for more information on geo searching