Proximity searches are fundamental to working with location data. It would make sense then that Geofield provide at least basic functionality to find/filter based on distance from a known point. In discussion at the Drupal Camp Ohio code sprint, it seems like the best way to handle this initially would be by providing a Views filter, and implementing a simple point and distance calculation at the SQL level. The results of the view would then be filtered based on what does or does not fit within the supplied distance radius.

CommentFileSizeAuthor
#28 1360260_geofield_distance_sort_28.patch4.64 KBLetharion
#26 1360260_geofield_distance_sort.patch4.29 KBLetharion

Comments

nrambeck’s picture

nrambeck commented

This should also include a sort handler to sort by distance.

foo’s picture

foo commented

Here's an example of the math from Google: http://code.google.com/apis/maps/articles/phpsqlsearch.html#findnearsql

They basically implemented the Haversine formula in SQL. My initial tests with MySQL showed that it's actually pretty quick.

SELECT id, ( 3959 * acos( cos( radians(37) ) * cos( radians( lat ) ) * cos( radians( lng ) - radians(-122) ) + sin( radians(37) ) * sin( radians( lat ) ) ) ) AS distance FROM markers HAVING distance < 25 ORDER BY distance LIMIT 0 , 20;

Where 3959 is the estimated radius of the Earth (in miles; 6371 for kilometers), lat and lng are columns, and (37, -122) is the center point of the search.

nod_’s picture

nod_
Primary language French
Location Lille
commented

Mysql and solr both have an extensions for that, are you proposing using that or old fashionned php/sql to filter the results ?

foo’s picture

foo commented

We're proposing doing it without other extensions or server requirements. Just a simple way to filter based on proximity. Real geospatial extensions are probably better/faster, but something that works out of the box would be super-awesome-5000. In the long term, @Brandonian suggested a progressive enhancement approach, where things like this would take advantage of native libraries and things when they're available.

nod_’s picture

nod_
Primary language French
Location Lille
commented

Yeah that'll be really cool indeed.

Better be careful then, people will start expecting a lot out of it, I keep getting asked how to implement this for every single project I'm using maps on, and it's pretty big clients with pretty big datasets. If we don't warn them enough it'll be our fault it's slow and messy with lots of data.

foo’s picture

foo commented

Agreed. I think that proximity functionality is critical out-of-the box, but the docs need to be very clear about how it will perform.

As long as it can handle a few hundred to a few thousand nodes in a Views result set without barfing or timing out, I think it's acceptable for baseline functionality.

A Solr-based solution might be a good next step for large datasets. Most people experienced with Drupal know that vanilla installs aren't meant for extremely large content catalogs, so I don't think it's unreasonable to expect those users to find advanced solutions. Maybe they'll even submit patches :-)

Also, we haven't even started talking about complex polygon searches or anything more GIS-like. A simple point and radius filter is a great start, though.

nod_’s picture

nod_
Primary language French
Location Lille
commented

Well turns out i needed that for yesterday. I believe you gave it a thought, any direction you'd like to go or is it just up to me ?

Brandonian’s picture

Brandonian commented

@nod_, I'm not sure when I'm going to have time to work on this. I'm not planning on pushing another release without something done for this issue, and if you get to it first, that's fantastic. Otherwise, I (or anybody else with an interest in this, for that matter) can take a crack at it.

rickvug’s picture

rickvug commented

There has been work on a D7 port of OpenLayers Proximity, which relies on geofield (see #1013450: OpenLayers Proximity port Drupal 7). For Drupal 7 it sounds like it would make a lot of sense to drop "OpenLayers" from the project name as the proximity functionality is generic to any mapping displays (or view for that matter). What do you think about combining forces on this issue? Is proximity something that belongs in geofield or should the module only concern itself with storing the field data?

nod_’s picture

nod_
Primary language French
Location Lille
commented

I guess it wouldn't hurt to have a proximity filter for views in geofield. Beyond that I don't know.

rickvug’s picture

rickvug commented

@nod_ Perhaps develop the functionality separately in the Proximity project and then decide if it should be merged back into Geofield?

jpstrikesback’s picture

jpstrikesback commented

Hey All, indeed there is now a sandbox for Proximity (which will hopefully become the successor to OL Proximity in D7):

http://drupal.org/sandbox/jpstrikesback/1367194

It currently has field, filter and sort, you could test it to see if it meets your needs and if you do please let me know any issues you find over at the sandbox.

Cheers,
JP

rickvug’s picture

rickvug commented

Ya, this is the perfect use case of module_exists(). Really the UI in Views should say something like "enable Geocoder to support string based input".

I also don't think that having the module based around nodes makes sense anymore (for views filters, for example). A more general approach of choosing entity type and then selecting the geofield to use would be more flexible.

rickvug’s picture

rickvug commented

Sorry, posted #13 on the wrong issue. Meant to say that in response to http://drupal.org/node/1013450#comment-5354036.

nrambeck’s picture

nrambeck commented

jpstrikesback, I ran some initial tests and all 3 handlers seem to work fairly well. I'll post one minor issue to your sandbox project issue queue. Great work!

jantoine’s picture

jantoine commented

I just started getting into this today. Thanks for all the hard work from those involved! I think we can mark this as a duplicate of #1013450: OpenLayers Proximity port Drupal 7 and close this issue no? I'll leave the actual closing of the issue to those already involved, but I think a separate proximity module is the way forward!

Brandonian’s picture

Brandonian commented

@jpstrikesback, Proximity looks like a nice start. I'm confused by some of the architectural decisions of the module at first glance (why do we need an index table that only stores lat/lon? We already store that info with geofields.), but I suspect that these might have been made when the module originated in D6.

I am very, very interested in getting this functionality into Geofield proper, rather than through an additional module. To me, Geofield needs to be useful out of the box, and I think having at least a basic proximity search would cover a pretty typical use case for our users. After all, what good is having geodata if you can't search it in a way that's meaningful? Yes, in theory you could probably fake your way through a square filter with views on the lat/lon fields, but if we can get a basic proximity filter in (and document it as such), I think that would go a long way into making Geofield much more useful.

That's not to say, of course, that there isn't room for a more feature-complete/scalable/awesome proximity module outside of Geofield, I just want to make sure the basics are covered here.

phayes’s picture

phayes commented

I have to say, the use of the Haversine formula *implemented in SQL* in pretty genius. I would have never thought of that.

nod_’s picture

nod_
Primary language French
Location Lille
commented

Indeed you should have made a proper post instead of an edit, we'd have seen this earlier.

Anyway I hope we'll use proper international units in whatever formula is used :)

phayes’s picture

phayes commented

We should provide options on which unites they want to use. Ahem: http://en.wikipedia.org/wiki/Unit_of_length

cubits? parsecs?

nod_’s picture

nod_
Primary language French
Location Lille
commented

Haha, what I meant to say was : please, don't use imperial units. I'm talking about hardcoded values in the formula.

Of course there should be proper options to allow any relevant unit to be used client side. And the defaults should be SI units.

phayes’s picture

phayes commented

Seriously though, I think we should aim for the following:

Kilometer (default)
Meter
Mile
Yard
Foot
Nautical Mile
Degree

Brandonian’s picture

Brandonian commented

+1 on #22.

Letharion’s picture

Letharion commented

I built a fairly hacky views handler that allows for sorting based on distance from an argument, and I found this issue. I haven't looked at the sandbox mentioned above, since I'm unlikely to switch mechanism for that, this late in the project cycle. My question is, does it make sense in anyway for me to clean up and post my code, or has the proximity project fulfilled this need?

Regarding #21-23, I'm in the SI corner, but there are people using imperial. Perhaps options could be derived from the locale?

nod_’s picture

nod_
Primary language French
Location Lille
commented

I'm no geofield maintainer but I'd be really interested to see your code anyway :) And I'm all for the use of locale to deduct units.

Letharion’s picture

Letharion commented
StatusFileSize
new 1360260_geofield_distance_sort.patch4.29 KB

I never decided took the time to clean this up of continue, but I figured a crappy patch that can be improved is far better than no patch, so here goes.

foo’s picture

foo commented

I agree with Brandonian that proximity needs to be a baseline feature for Geofield, out of the box. As much as I love Drupal's traditional "code it yourself, whippersnapper!" mentality, some out of the box functionality is good :-)

Also, thumbs up for selectable or locale-sensitive units. The code snippet I pasted above was just pulled from Google's example code for the gmaps API; hardcoded units are obviously a bad idea.

If other projects implement more sophisticated geometry, awesome. Basic proximity will coverer a lot of common use cases, and at least get people started.

@Brandonian: have you taken a look at Letharion's patch?

Letharion’s picture

Letharion commented
StatusFileSize
new 1360260_geofield_distance_sort_28.patch4.64 KB

Slightly fewer wtfs now. Still needs lots of work.

Brandonian’s picture

Brandonian commented

Fewee wtf's are always appreciated, @Letharion. :-) Thanks for getting the ball rolling.

Brandonian’s picture

Brandonian commented
Version: 7.x-1.x-dev » 7.x-2.x-dev

Made some commits in the 2.x branch for this. Started with @Letharion's work and built out a sort, field, and filter. The filter definitely does not work at the moment, and I can't guarantee that the other two are 100% working, but it's a definite start.

Aldus’s picture

Aldus commented

absolutely killer feature :) unfortunately I can't help much nowadays, but I hope to see it available soon. Following!

Brandonian’s picture

Brandonian commented
Status: Active » Fixed

I've committed some more work on the 7.x-2.x branch. I've added a filter, field, and sort based on @Letharion's initial patch and the guidance from earlier in the ticket. At this point, I'm comfortable in saying that it's stable, but untested. I'm going to close out this ticket. Any issues that come up should be documented in separate tickets.

Thanks for the support and guidance, everyone.

phayes’s picture

phayes commented

Awesome!

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

mgifford’s picture

mgifford
he/him
Primary language English
commented

So for clarification, that should be in the 7.x-2.x-dev release that you did 2012-Feb-29?

grantlucas’s picture

grantlucas commented

Am I correct in understanding that this change presents a filter which allows you search with in the radius of a central point? From looking at what the changes add, can one not make the origin of the search flexible or must a fixed Lat/Long always be provided?

In it's current form, what are some use cases for these changes?

Thanks for all the hard work!

Letharion’s picture

Letharion commented

At #36. You probably want to see #1496354: Distance sorting from other entity..

r_wel’s picture

r_wel commented

I am having a difficult time figuring out if there is a conclusion to this thread... I have had openlayers + geocode up and running for awhile, hoping that some sort of proximity filter functionality will come along. Sounds like there is a sandbox project but can't find any info on a potential release date. Does anyone have some insight into this or another route. I am not a developer so I am a bit stuck looking for a solution (I am basically trying to to show locations on a map and allow users to find locations within a certain proximity). I've been really happy with openlayers and really hate to go the location + gmap direction.

grantlucas’s picture

grantlucas commented

The initial features of this thread were created and closed. Further functionality has been built over at http://drupal.org/node/1469956. The patch in progress adds a new module called geofield_proximity which adds a proximity search filter. We can definitely use more testing of the patch!