[Meta] - Improve Views-powered Geofield proximity searches

I would definitely consider including it as a submodule. Although rolling an independent one might also make sense and just use this module's API. Other geocoder maintainers have a preference here?

Wait, why would the proximity views filters go in Geocoder instead of Geofield? We've started building them out in the next version of Geofield (#1360260: Add views filter to provide proximity search of Geofields) with the idea of backporting them once they're stable. Is there something I'm missing?

That makes sense Brandonian. I think maybe the issue is more that proximity filters should use geocoder API instead of rolling their own. Is this correct jenlampton?

Also, note that if you are running a large scale web-project with lots of data, you should be using local-solr (apache solr) for doing proximity searches. Doing proximity searches in the DB are essentially a tablescan. Is this correct? Brandonian

@phayes: "proximity filters should use geocoder API instead of rolling their own" what do you mean here? geocoder api doesnt really provide anything related to proximity information so i dont know what we would provide or to whom we'd be providing it.

The proximity search provided by the DB/openlayers_proximity module is either coarse (lat/lon bounds) or slow (great-circle) but i still use it when i dont want to install and configure a solr 3.3+ instance for other search.

It is a useful low fruit for alot of sites that arent geolocation sites and just need to occasionally look up nearest stores or something.

Reassigning to geofield. As much as i like/want the functionality myself. I agree with brandonian that it belongs in geofield as it relies on its data to provide the filters. Happy to integrate here personally if he opts to punt. ;)

This was my mistake. I dont know how i misinterpreted jen but we cleared it up in irc.

She want to be able to provide "Austin, TX" or "78704" in a exposed views filter to have that geocoded and then used for proximity searching of the geofields. This isnt "find other geofield entities near this geofield" (althoguh this will be asked for next you can guarantee it ;) )

This still feels like it belongs in geofields and geofields should depend on geocoder to provide that functionality/filter. Ours is a simple API to utilize and we'd have to do alot of heavy lifting with geofield objects if we did it from our side. It also allows easy reuse of the views handlers for the entity to entity proximity we'll be asked for next.

@brandonian and @phayes agreed? If not please push it back to geocoder one last time sand ill pipe down.

That sounds great. Would it make sense to try to built some sort of pluggable system with our distance views plugin, so other modules like geocoder can add that different sources for origin points, or am I getting too complex? A pluggable system would be nice so that we could isolate distance formulas/implementations.

The more I think about it the more I agree with @michaelfavia. Proximity-search really doesn't belong in geocoder (even if it makes use of our API).

Ok then with that we'll punt it over to geofield. API call to us should looks something like:

geocoder('google','1501 Barton Springs Rd 78704');

You can get the list of available geocoding backends like any other ctools plugin of a type:

ctools_get_plugins('geocoder', 'geocoder_handler');

You could also use our fancy new ajax api if youd like to do it via exposed ajax filters (though i think that views will bridge that divide for you automatically).

@phayes a useful addition to our now custom google geocoder that DOES belong in geocoder, might be to leverage google places autocomplete fields like jens search box. We could optionally attach the ajax callback to single line text fields via form alter.

Demo: http://code.google.com/apis/maps/documentation/javascript/examples/place...
Docs: http://code.google.com/apis/maps/documentation/places/autocomplete.html

Let's make it a submodule on 2.x. Once it's stable we can backport to 1.x

Also note that geocoder has an AJAX interface: #1462364: Provide AJAX service

@jenlampton, I'm thinking about spending some time on the views integration in general so that we can start backporting this part back to 7.x-1.x. Do you have any code you can share from your work?

I am curious as to the progress of this issue and how it relates to #1360260: Add views filter to provide proximity search of Geofields. Would love to see this added to the module and if there's not much being done on it at the moment I could take a stab at implementing what is discussed in a patch.

@grantlucas, I've got some code locally that I'm going to push tonight to a separate branch for collab. I have a basic structure down for doing distance searches from geocoded values and different fields. Currently works with fields, haven't expanded it yet to filters or sorting, but should be relatively simple. I noticed on twitter today that you're taking an interest in working on this, and I'd love a second set of eyes on the final code. If you have any questions about geofield in general, I'm (usually) on IRC as Brandonian, or you can hit me up via my contact form/twitter.

Apologies for being off the grid during the last week, visiting family. Real life is crazy sometimes... :-)

Just posting a bit of an update. I've managed to get text proximity search working as a filter. So you can search for "montreal" and all nodes with in a currently fixes radius will be shown.

Things I still need to get working

• Choice of Geocoder source
• Autocomplete from Google places
• Allow to specify distance and units?
• Not sure if it's possible but if the distance filter Letharion created is present, it would be good to take the distance from there

Right now with the addition of the specifying the distance and units, it's coming pretty close to what Letharion had done, but with the added ability of setting the origin based on geocoded text. I'm not really sure of the best course of action at the moment. Have two filters which try to work together, or have one filter which can have either of the sides predetermined and hidden from the exposed filter. i.e.: Just have text search visible with the radius predetermined, or have the origin geocoded and predetermined with the radius visible.

Anybody have thoughts on this?

Attached is a patch which adds a filter for geofield allowing for text proximity search. I have it to a point that it's mostly presentable but there are still some major things to do like proper Geocoder implementation. Right now it's hard coded to use Google for geocoding.

Enable the filter in views and you will have the Origin which will be geocoded so enter a city, Radius with distance and units and a checkbox for whether the radius should be included in an exposed filter.

I would love to get some feedback on this implementation as soon as possible as I'm actively working developing this feature.

Let me know if you have any questions on make this filter work etc. I'm happy to help.

Might be useful to make use of the geocoder module, which has an AJAX api for doing geocoding and support for google, yahoo, yandex (russia) and more on the way.

@phayes Yup already using Geocoder. Will be adding the real support for the different services next. Will have it as an option in the views filter settings.

Oh! Another major thing I will be adding in the next few days is contextual filter support.

Just to be clear, the patch #18 is using the Geocoder module. I'll be fleshing it out more tomorrow to allow for the setting of a service to use instead of the hard coded Google.

Would love to get feedback on #18 for how it's implemented interface wise etc as the distance filter is someone incorporated etc.

Thanks

@grantlucas: I started looking at your patch last night, and you're on the right track (especially with the options for exposed filters, hadn't considered that yet). I'm going to fold your code into geofield_handler_filter.inc.

@jenlampton: I haven't had a chance to look at your module yet, but plan on doing so today. I'm sure there's some great things to work with in there.

To get an idea of where I'm heading, take a look at geofield_handler_field.inc in the views-integration branch. While the current repo's has a few bugs that I haven't pushed fixes for yet, it's basically what's in my head for dealing with this in general.

I'm heading to the airport in a few, but will be online off and on today hacking away at this. Hoping to commit something back to the 2.x (and porting to 1.x) by Monday-ish. :-)

@brandonian: Was going over my patch with @jenlampton a few nights ago on IRC and came across some pretty bad bugs in my patch. I'm looking to fix it up in the next day or so.

Oh. Am also going to switch the work i'm doing to be a separate submodule to keep the dependancy on geocoded out of the main geofield module

Alrighty. Did a bunch of work on the patch and now have it set up as a separate module under the modules directory named geofield_proximity so that the Geocoder dependency doesn't fall on Geofield itself.

It now features a new form item for the exposed filter in the format [distance] [unit] from [origin] and also provides a contextual filter / argument which can be used.

You will see the initial support for selecting which Geocoder service to use but it's disabled at the moment due to some things that need to be worked out when making the call to geocoder() and ensuring the string is in the right format. Figured I'd upload with that in there for now so you can see how it would work.

To add this filter to your view, search "proximity" when adding the filter and it should show up. Same thing for the contextual filter. From there I think it's pretty straight forward. Let me know if there are any issues or glaring omissions.

Grant

Sorry. Minor bug fix. Had set an option_definition default incorrectly. Should be good now.

Thanks for all the feedback! Your text suggestions make sense for sure. For the geocoded coordinates, I def noticed something weird but I didn't set up a ton of test places so it looked alright and was getting values for the coords array. Using getY() and getX() are indeed the right way to go though so good catch.

I've attached another patch off of #30. Just removed a validation function I forgot to remove in geofield_proximity.views.inc and fixed a typo in a validation function call on the custom form element.

Oh forgot to mention about the geocoder service selection. The one issue with geocoded selection site wise is that the service used, or provided by another third party module may not be applicable for the views filter. So for example if they choose a file/image based geocoding service site wide, that wouldn't be good for our text search.

Right now I have the selection limited to only services that accept text but the issue I ran into was knowing the format of text that the service wanted is unknown from our side. So for example if they select JSON as their service, then it expects a json string. With what geocoder send back, other than the service name, we have no way of knowing the needed format to pass. I was talking to @phayes about adding support to services to define their formats needed when dealing with plain text input.

So I guess for now that feature of this patch is on hold till I can get that stuff sorted out. At the moment, for the input we're expecting from a plain text box, Google is a pretty safe bet.

It seems like it would be realky easy to add static caching to geocoder

On the topic of using a relationship, would that work in a situation where the proximity itself is dependant on a relationship? For example right now the setup i'm working in is that all the distance calculation and proximity are based on a content entity reference and not the base node type.

I've actually run into an issue now where I'm mainly using Contextual filters for the main location but then want to add a distance filter whose location is based off of the contextual filter origin. Not sure of the best way to achieve that.

I'm wondering if in the options for a field/filter/sort proximity, we could add a drop down of other fields/filters which could be used as the origin. Not sure how feasible that would be or even possible.

Hi folks,

I've just added static caching to geocoder in the latest dev.

@phayes Great to hear! Does anything need to change for modules which use geocoder()?

nope. There's additional options if you *dont* want caching, but caching is enabled by default.

Just an idea... Sometimes it happens we have to present to the user the information which is easier reachable, not what has a shorter radius distance.

What about having the option to compute time proximity. Which is would not be radius distance, nor any other spatial distance. It would be the time needed to reach one point from another, taking into account the streets you have to walk, the speed limits, etc etc.

It could be computed using Google or Bing get direction webservice, which takes into account the streets you have to walk. We already have the Get Direction module here. Perhaps we could interface with it.

Hi cesareaugusto,

I think that's a great idea but likely out of the scope of this particular patch. It might make a great contrib module that also implements a general routing API.

I think that's a great idea but likely out of the scope of this particular patch. It might make a great contrib module that also implements a general routing API.

I don't think it would be out of the scope of this patch/module. The module which does the routing thing is already here: Get Directions.

It often happens that points which are the closest in radius distance are not the ones which take less to reach. It may be usefull to sort those points according to actual reachability.

What I suggest is: to use Get Directions to compute the time the route takes then to do the proximity sorting according to that computed time.

We would then have entities which take less time to reach from a given point (perhaps another entity location) first/last than the ones which take longer.

Get Directions module uses JavaScript, we would need some sort of RESTful service. So we can't use Get Directions.

Furthermore, there is no way to query on this. To get the "closest" 5 points out of a list of 1000, we would need to do an API call on every single one. Maybe we could use some sort of heuristic to reduce the list to 20 before doing the lookups, but that still FAR too many. Cesareaugusto, I just don't see any way to make this work.

If you think you have a solution to this problem I would certainly encourage you to submit a new patch in a new issue. This particular patch is very close to complete, so again, doesn't belong here.

Was sent over to this thread from http://drupal.org/node/1360260. I'm using openlayers and searching for proximity filter. After reading this thread, I'm guessing that geofield-proximity is not ready for a non-developer such as myself - I've never applied a module patch - and don't want to waste anyone's time with issues that I may come up with (assuming I could figure out how to apply this patch) because I don't know what I'm doing.

My needs are pretty simple...show me locations within xx distance from an inputted location

However, I'm hoping someone will tell me I'm wrong and give me a bit of direction on how to proceed. If not, is there any info on when the functionality may be available to the general public. thanks

I've reviewed and committed this!

Thank you so much to everyone who worked on this patch.

Awesome!

Actually I'm re-opening as there are two more things that need to be done before this is "done-done".

1. Fix the default length of the geocoding-input text field. It is currently set to 15 characters, which is too short.
2. Port geofield_proximity to the 1.x branch

yay!

I started to review the patch at #34 yesterday and felt like there's still quite a lot to do to get distance filtering wrapped up. @phayes, I don't know which patch you committed, but what I think would be a good course of action is to treat this issue as a meta-issue, and break out sub-issues so we can knock them out piecemeal. I can play project manager on this and re-review based on whatever you committed and start breaking out tickets so that others can play along. :-)

Part of the reason that I was having issues with earlier patches was that I had a separate geofield_proximity module that was floating around in a sandbox at some point. (namespaces++). That being said, I still think it's worth pulling together other issues that will improve this feature and get it ready for stable release.

Changing the title to reflect meta-ness of the issue. (How meta...)

Updated issue description to work as a meta-issue for our views integration.

How to do proximity searches based on a node location instead of a fixed lat-lon point? When setting up a view it seems I can only put static lat-lon data, no way to use a geofield of another node as a base point (let's say the current viewed node).

@cesareaugusto: That's not covered by this module yet, but I'm going to add a ticket to cover that use case, so we should hit it soon.

Added #1561190: Proximity Search - Remove dependency and/or provide alternative on filter for other views handles to hit list.

Added another sub-issue

Added task.

Added issue to hit list.

@Brandonian: Thanks for your quick reply! I'll wait to see it then! But I got a question... how does this module proximity filter relate to the Proximity module (ex Openlayers Proximity)? Are they totally separate filters? Or do the relate somehow? Which one shold I choose and how do they differ?

Added task to hit list

@cesaraugusto, It's my understanding that what we're doing here will supersede Openlayers Proximity. I'll let anyone involved with that project speak otherwise.

Also, you'll probably be interested in watching #1561336: Distance filtering from another geofield

@Brandonian: Perhaps I got it wrong... but from what I read on Openlayers Proximity pages, they are dropping the "Openlayers" in the module name in favor of just "Proximity". Their upgrade process to 7.x shall lead to a new Proximity Module. Did I miss anything?

Does anyone have a repo for their current progress so far? I'd like to get my hands dirty in this one.

The most current code at this point is the 7.x-2.x branch.

For testing i've been successful in applying the following filter, which I believe is what's being developed in this thread:

Content: Geo Location (field_data_field_location: Distance from point.)

However I cannot find the same filter in the contextual filters.

I basically need to contextually set the location (either WKT, or lat/long) for the view. I dont mind if the distance is a default value, but that would also be good.

From this thread it appears that contextual filters is already implemented, but i'm not having luck finding it from within the views panel. Thanks.

@stopshinal

If using the 7.x-2.x-dev version I see Proximity as a Contextual Filter, but not the Distance from point. From my thus-far limited engagement, looks like there isn't one for Distance from point.

That is correct. There is one contextual filter for the proximity. No distance from point. It's pretty much the same as the normal proximity filter but in argument form.

@stopshinal,

Please don't highjack this thread for support. Open up a new issue please.

Added another task.

Added an update section to the overview page. Patches/testing/issues welcome!

coming here from some discussion in the openlayers_locator #1418238: Roadmap

Please note that we now have a "freeform Latitude / Longitude" parser for geocoder. I think it might be quite useful for this issue!

Status update on overall project.

Updated notes, pointer link to proximity merger issue.

Status update for 5/22/2012

Added ticket for ajax issue in admin

I've committed a bunch of code to the 7.x-2.x branch for wider testing of our views-powered proximity search. At this point, I've made the following changes.

• Removed geofield_proximity
• Created a custom FormAPI element (mostly from prior geofield_proximity patch), which takes 3 data points, distance, units, and origin. Origin can be set to be any other Form API element. (Probably should set a validator to prevent somebody from setting it to be another geofield_proximity element).
• Added a function (geofield_haversine) which returns a SQL implementation of the Haversine formula, suitable for use outside of views calculations.
• Created geofield_handler_field.inc, which can calculate distance based on manual lat/lon entry, geocoded data (via geocoder), other geofields, and an exposed views filter.
• Created geofield_handler_filter.inc, which can calculate distance based on manual lat/lon entry, geocoded data (via geocoder), and other geofields.
• Created geofield_handler_sort.inc, which can calculate distance based on manual lat/lon entry, geocoded data (via geocoder), other geofields, and an exposed views filter.

While the handlers are (basically) functional at this point, they could use some serious testing/polish. Please dig in and file issues/patches! :-)

@Brandonian

I upgraded, the new option (in proximity exposed filter) for choosing the input type for origin is very cool, however, I could not get the proximity exposed filter to properly return results under any circumstances I tried. The previous version (2012-05-13) is working as far as I can tell.

I did not yet inspect the query to see what was going wrong but soon will compare it to the 05/13 version's query to see if that sheds any light on it.

Also, it seemed that it was mandatory to have a default distance & origin set in the exposed filter's settings. It threw errors and flagged the fields red if they were not. I noticed very odd behavior in the default settings on origin field as I toyed with and saved various settings.
I saw the value in that field change 3 times from being:

• A pair of lat/long fields (initial config), I just put 0 0 in each.
• Then when I set origin to be a Geocode input, (default settings fields stay as lat/long so I hit apply and re-opened setttings) origin became 1 field and had "0 0" in it. I cleared that out and saved view
• Reopened and where it was blank was traded with "field_geofield_xxx"

Hi everyone, sorry for the lack of...erhmm...formality here (not really a coder and don't really know what I'm doing for submissions) but I messed around with the current dev version 7.x-2.x and changes Brandonian made for my own project, taking the geocode coordinates from the user and running proximity. I tried to find relationship data to pull this but apparently it hasn't been implemented yet. So, here's my hack! Thought I would share.

I added this to both handler_field and handler_sort, then added the menu option and just selected it.

Top part:

case 'current_user':
global $user;
$user_fields = user_load($user->uid);
$position = field_get_items('user', $user_fields, 'field_geolocation');
$this->options['source_lat'] = $position[0]['lat'];
$this->options['source_lon'] = $position[0]['lon'];
break;

Menu array:

'current_user' => t('Current User'),

I think the next logical step is to allow a relationship with the current user's location, then we can just have proximity run reading the relationship. I don't really know the right code to make it work, but I think that's the right idea. That's basically what I did here but obviously it's very hacky, and probably only works for logged in users (suits my use, though).

Attached is a patch complete refactoring Geofield's Views proximity handlers. We've had many, many different requests for various data sources (geofield on current node, geofield on current user, html5 location of current user, etc), and continuing to extend our views handlers by adding switch statement after switch statement was going to turn into a maintenance nightmare, so I've scrapped all that and created a mini-plugin system for interfacing with Views.

Once committed into the 7.x-2.x branch, it will have better in-code documentation than what's currently there, but essentially I've mapped various common touchpoints within a views handler. In addition to the various geofield data sources we currently support, I've also added a plugin to support pulling a geofield from the currently loaded entity from a url.

A caveat, applying this patch means that you will have to rebuild any current proximity-based views on your site. I don't plan on creating an upgrade path between what our current dev branch does and this patch.

Before I dig into this - will this allow filtering by distance in views? I see your reference from the other issue where Master chief created a patch here http://drupal.org/node/1561336#71. Will all play nicely except I need to rebuild views?

Thanks

@IWasBornToWin, that's the plan. The patch in #69 provides sort, field, and filter plugins for proximity/distance calculations based on Geofields. Currently, it provides a way to enter data via a manual lat/lon input, a geocoded result, other Geofields referenced in the View, and a specified Geofield based on menu path (i.e., you can create a block that shows nearby nodes to your current node). Before committing, I want to add an additional plugin that supports a specified Geofield on the current logged in user.

I'm glad I asked, the logged in user is big to me. So I'll ask to ensure I understand. I want a user to be able to type in 25 and then the view will filter all other users who are within 25 miles of the current logged in user. Is that something that will happen in the next step you just described?

This is just the sorta thing I needed to add a handler that, for example, calculated distance from a taxonomy of zipcodes (zipcodes not in/on returned row content, just as a point to search from) with geofields isn't it? I'm gonna give it a whirl on that one, see how I fare and report back later today if I don't get dragged in some other direction.

Updated patch, including a new plugin for proximity searches based on a current user's position (via a field attached to the user entity, not HTML5 location).

Been delayed in my tests...the dreaded lurgy I'm afraid...Will attempt to gather my brain and get on with it

New patch to address #1810668: Views exposed filter does not work if filter identifier is changed.

Changes have been committed to the 7.x-2.x branch. Please open new tickets for any additional problems.

How far along are you with the solutions?

For reference, I added some simple instructions on how to create a view:
http://drupal.stackexchange.com/questions/54873/geofield-proximity-in-views

thank you for looking at it mototribe...however I cant recreate the results from your example.

Is there some permission things I could have overlooked? I am using the 7.x-2.0-alpha2 version

Getting this error when setting up a view.

Using version:
geofield 7.x-2.0-alpha2

SQLSTATE[42000]: Syntax error or access violation: 1582 Incorrect parameter count in the call to native function 'RADIANS'

Confirming #82, this is also in latest dev.
Update: After defining a "Source of Origin Point", the error is gone.

Tried playing with the latest dev, and I noticed the proximity filter is not available as an option for Search API based Views. Would love to use it there as well. Not yet sure what is required to expose the filter to search api views. Inquired over at Search API as well #1888272: How to expose a Views filter provided by a module (Geofield in my case) to a search api view?

Anyone know what I am doing wrong for this issue? http://drupal.org/node/1887122

Anyone else having this issue?

Thanks

@Brandonian: drunken monkey recently posted a refactor of Search API Location, which has geofield as a dependency. Since you've already come a long way on getting views exposed filters happening in geofield 7.x-2.x, I was wondering if you'd have a minute to look at the feasibility of integrating your good views integration work with search_api_location. This would really help to avoid duplicate effort, as noted in this comment.

Can the source of origin be the users current location secured from SmartIP?

Is anyone working on integrating autocomplete for proximity searches? Would be a fantastic feature for the module.

@brandercreative, pasada

I took search_api_location and forked the heck out of it with some custom JS.
The end result is a free-text entry field for location that gets geocoded from google on-the-fly.

http://rocklobby.drastikbydesign.com/map-search

Enter "New york" or "London" in the "Distance from" field. Then hit tab, click off, or try to click apply, and that is when it will quickly geocode and in the background fills in the coordinates to hidden fields. Then you can use the slider to choose proximity and hit apply.

It still uses apache_solr for the spatial data results.

The actual gmap widget (that comes w/ search_api_location) is still there, just hidden - but when I am complete I will cut that out of the module entirely.

@drastik , very nice, indeed.
Can you share the code?
I tested your website, using IE8:
- when entereing 'Amsterdam', I get error 'Stack overflow at line 25'.
- The 'radius' widget is not shown, only after entering a city, and the pressing the '3 layer thingie' in the upper right corner.

@johnv

Yes, the client in this case specifically said disregard IE entirely. Pretty shocking, yet definitely caused warm emotions in my heart :)

I may in the future make it work for older IE, but he needed it for SXSW on a tight deadline, and using CSS3 and other modern methods, there was no time for IE.

If there is enough positive response & interest, I will try to go by the book and contact the search_api_location maintainer and roll in new features w/ setting options to switch to this mode (after refining and cleaning it up, this was still just a concept stage).

Hi there, I've started using the "proximity" branch of this module to get proximity searches working in views. With two patches, to fix some small errors, it's working pretty well.

What are the plans now for merging the proximity branch with the 7.x-2.x branch? The latest commit to the proximity branch was 10 months ago (2012-05-22) so I'm wondering if that is still the best place to look.

.

.

.

@pinkonomy This is a bug report. If you have a support request, you need to open a new issue; this is not the right place to ask.

This meta issue has served us well, but I think it's more of a distraction than anything else at this point. If you have any improvements/suggestions/patches/hate mail for proximity searching and Geofield, please open a new issue. Otherwise, thanks all!