I have created a views argument handler for proximity.

You can use the argument in two ways, by postal code and distance or by coordinate and distance.

Example: view path is content-listing

content-listing/[postal_code]+[distance]
content-listing/75214+10

The above argument will restrict content within 10 miles of zipcode 75214.

Coordinates can also be passed.

content-listing/[lattitude],[longitude]+[distance]
content-listing/32.783,-96.8+25

Options for the argument handler are:

* Coordinate Description
Radio for either postal code that gets translated to rough coordinates or exact coordinates
* Unit Type
Miles or Kilometers. Unit type is set as an option, not encoded in the argument.
* Method
The views query is modified similar to how the proximity filter already works. And you can choose between a radial or rectangular search.

location.views.inc was modified to register the new handler in

location_views_handlers() and location_views_data()

A new file handlers/location_handler_argument_location_distance.inc was created.

I need to do a little cleanup and will post the code later this weekend. Bdragon, can we schedule a time for me to run some questions by you on IRC or email?

Comments

damienmckenna’s picture

Does it support a default on the second argument, so that it would work as just "content-listing/75214"?

Dustin Currie’s picture

Not currently. I didn't even think of that until I read your post. It's a good idea. I'll make that change before I post the code here.

damienmckenna’s picture

Great, I look forward to testing it :-)

Dustin Currie’s picture

I think that postal code only is not appropriate to use with a proximity argument handler. It should have its own argument handler.

Doesn't like me adding the new file as a .inc, so I just appended .patch. It's not really a patch file.

Anonymous’s picture

i can confirm that the patch and inc file works. very nice work, thanks you for your effort. karma+1

i noticed that the sorting by proximity/distance as well as the display of proximity do not work. the field is empty.

it's still very usable to show all marker inside a defined distance.

rc2020’s picture

Awesome. Excellent. Check out http://drupal.org/node/357295 which is the main queue for this problem....

What's the process on getting this committed? I'll install a fresh installation of drupal and load it on and write a little report on how it works/how it can be improved.

Thanks again for spending the time to do this!

dchampine’s picture

Can't get postal code to set
I've patched the -dev version and added the handler which exposes the functionality just fine, but I can't get the argument to work properly. Neither of the radios are checked for type of center point and the query seems to indicate that lat/long is selected even when I specifically select postal code and save? AND (location.latitude > 0 AND location.latitude < 0 AND location.longitude > 0 AND location.longitude < 0)
(it may just be for validation and not affecting the query - not sure). I'm not a php coder, but I'm pretty handy with drupal/cck/views/panels and happy to help with testing/config as I really need to get zipcode arguments to work for my production site.
Drupal 6.14
MySQL 5.1.36
Views 6.x-2.6

Dustin Currie’s picture

dchampine, email me dustinc@getlevelten.com so we can get together and resolve what you are seeing.

held69’s picture

Can someone confirm if the patches above can be used for use with cck location as well?

Thanks.

fletch11’s picture

subscribe

jereoms’s picture

I am not sure I understand how to implement once you have applied the patch and added the new .inc file. I believe I have done those parts correctly and was expecting to see a change available in my views configuration area. Where should I go to implement this feature?

Thanks

jereoms’s picture

Sorry for the mixup, I found my error and it is simply a matter of adding the Proximity/Location drop down to the arugments view area.

creativegenius’s picture

subscribe

jeff.k’s picture

I too am having an issue where I cannot set the postal code or Latitude/Longitude. The radio does not appear to set. Is this a known issue?

UPDATE:
This works even though the radio does not stick. There is a new parameter added "seperator". It only needs to be added efore the distance

ie
38.804835,-77.046921r25

Is there anyway to display the distance with the entry?

rc2020’s picture

What is the status on getting this committed to Dev? I'm using a hacked handler to input a lat/long, having this be in the official release and migrating to this solution would be killer!

Thanks!

socialnicheguru’s picture

How can you setup a default location argument based on the user logged in or php?

socialnicheguru’s picture

Is this similar or not: http://drupal.org/node/662892

ayalon’s picture

Category: support » task
Status: Needs review » Needs work

I used this argument handler on my test website and it works.

Basically I see 2 issues:

1. The radio button / selection ZIP vs lat/lon is not saved (this is cosmetic because it works)
2. There is no way to display the proximity of the location to the argument lat/lon and there is now way to sort the view reults according to the distance. (this need probably some more changes)

I think if these two issue can be resolved, the patch is ready to be commited.

rc2020’s picture

Will this argument handler accept php code for a default argument?

That way I can use module-defined functions to pass a lat/long from an HTML form search. ($_POST search passing city/state to query db to return a lat/long).

Thanks!

sagannotcarl’s picture

Subscribe

cyberwolf’s picture

Subscribing

cyberwolf’s picture

In the method calculate_cords of the argument handler there is a line that can be removed:

location_load_country('us');

gravisrs’s picture

StatusFileSize
new4.84 KB

Hello everyone.

I've created similar handler that supports proximity.

Features:
- accepts [latitude];[longitude] pair and then uses set distance in the configuration of the argument
- or acepts [latitude];[longitude];[distance] for different distance
- Miles, Kilometers or Meters
- configure which accuracy function to use (square or circle) for argument
- doesn't collide with proximity filter set
- doesn't collide with other arguments (also other proximity - argument can be used multiple times)
- php default argument setting
- all the other available no-argument handling (except summary)

rc2020’s picture

oh F'in right!!!!!!

Thanks dude!!

Nick Lewis’s picture

Re #23 -- this patch works great! Only one problem:
"; " gets stripped out by the url function. Probably "_" is the safest alternative i can think of. If you are able to use another character I think you should put this either on review or RTBC.

matt2000’s picture

Status: Needs work » Needs review
StatusFileSize
new4.84 KB

Here's a version that uses underscore. Please test.

Nick Lewis’s picture

Status: Needs review » Needs work

Okay gave this a try against some 2000 locations.
30.2671530_-97.7430608_500 (The lat/long is for Austin, Texas with a radius of 500 miles. )

The first result returned was: Tulsa, OK, 74104
The last result returned was: Houston, TX, 77024

So the sorting mechanism no worky.

The good news is that narrowing the search to 30.2671530_-97.7430608_10 (10 miles of Austin, Texas) only returned Austin, TX. If we can get this tied into the sort fields, and maybe even the proximity field like the exposed_view form, i think we're good to go.

comfused’s picture

I did the patch from #26

I get:
Error: handler for location > distance doesn't exist!

when I create a distance argument

rasumo’s picture

Is it possible to display the distance of each node from the location passed i.e. field? When I add a Location: Distance / Proximity field there's no value.
EDIT: If I plug in static coordinates a value shows up but what I'd want is the distance from the coordinates passed as the argument.

Anonymous’s picture

StatusFileSize
new5.77 KB

I've added to this the ability to use Decimal degrees for distance measurements. I find this useful when dealing with Google maps and wanting to get back only nodes that are in the viewable area of the map.

Now the argument handler can take in 4 parameters:

  1. latitude in decimal degrees
  2. longitude in decimal degrees
  3. latitude distance in decimal degrees
  4. longitude distance in decimal degrees

The handler will look for values that fall within latitude +/- latitude distance, and longitude +/- longitude distance.

When using a map you simply pass in the coordinates of the center of the map, and the the distance from the center to the top edge and the distance from the center to the left edge.

PS The patch is a diff vs. 6.x-3.x-dev
Hope someone finds this useful.

var bounds = map.getBounds();
var sw = bounds.getSouthWest();
var ne = bounds.getNorthEast();

var s = sw.lat();
var w = sw.lng();
var n = ne.lat();
var e = ne.lng();
var centerLng = map.getCenter().lng();
var centerLat = map.getCenter().lat();

var lon_distance = centerLng - w; // This is the distance from the center to the left edge in degrees.
var lat_distance = n - centerLat; // This is the distance from the center to the top edge in degrees
rc2020’s picture

Any chance this can be committed to dev? That way I don't need to keep patching releases and I can know what code version I'm using to keep current...

Thanks!

jupiterchild’s picture

Spent hours yesterday trying to get any of these patches to work but without success. I was receiving an error message 'Error: handler for location > distance doesn't exist!'. I was using the latest version of Views 6.x-2.x-dev and had tried clearing Views cache but to no avail. Today I noticed an update to Views 6.x-2.x-dev dated 9th March, I installed the update and the handler is now available and functioning.

I am using #30 p.patch. The handler seems to work OK but it fails to display the actual proximity distance, though I could be missing something here. Sorting also seems to be broken.

Just like to thank all the people working on moving this forward. Great work!

deepeddy’s picture

I'd like to second the comment in #29. I'm actually finding that it shows the distance from my user location.

I'm real happy to discover as I need missing functionality that it's been worked on in the past week.

yesct’s picture

Issue tags: +location views

So.. I'm confused, what is the review of #30? Does that work for people? Someone please post a summary of their review.
http://drupal.org/patch/review

Nick Lewis’s picture

Ha -- yeah it sure does. I've been actually using $globals['user']->location[0] = $location; as a work around in hook init to make views filters sort of work... yeah its ghetto.

// try this for judging distance 
    $latlon_a = array('lat' => $loc1['latitude'], 'lon' => $loc1['longitude']);
    $latlon_b = array('lat' => $loc2['latitude'], 'lon' => $loc2['longitude']);
    $distance = location_distance_between($latlon_a, $latlon_b);

Not proud of my final solution: inform node references by way of location proximity during an operation... qq

ankur’s picture

I'd personally like to combine #23 and #30 plus one more type of argument: a node id (and maybe a distance attached to it) where the first location of the argument node is used as the search point.

Also, there *has* to be a way of pulling the location as a field when we have an argument that causes a filtering by distance. I'll consult the views documentation.

I'm very interested in getting this in ASAP and will be watching this issue. I'll even (if I get the chance) try to put up a patch that combines #23, #26 ('_' instead of ';'), 30, and the argument format I described above.

yesct’s picture

I think working on this issue sounds like a good start for one of the larger issues.

teleted’s picture

subscribing

rfay’s picture

subscribe

q0rban’s picture

Subscribe

rc2020’s picture

Hey YesCT - here's the breakdown of what we have w/ handler attached. If you or any maintainers would like access to the DB or codebase so you can see what we got going on it can be arranged. If you have any more questions, please let me know.

My location searching system for views is for my primary project www.lifeundersun.com.

In short, it obtains the city and state of an area in question through HTML forms. The user enters the state from a text field and selects the state via a select dropdown, and the form action is a landing page with a bunch of views that use a hacked proximity handler that I have attached on this document, using $_POST as the method.

It works like this: Each view on the form action page (/proximitylaunch) uses a hacked location filter for a views filter, to accept PHP code. Each filter has 3 php input boxes – lat/long/dist. Respectively, for values, each is

print proximitysearch_lat();
print proximitysearch_long();
print proximitysearch_dist();

These functions are defined in a custom module called proximitysearch.module. All the module does is provide functions that take the $_POST values provided by the parent pages’s HTML form, run the user-input values through check_plain() for security, and then query the database’s location table. The query is basically Return Latitude/Longitude where city equals variable for $_POST[‘city’] and state equals variable for $_POST[‘state’]. Each function returns a latitude, a longitude, and distance just returns $_POST[‘distance’] into the filter. This way, users are able to enter in location information for querying on another page, and are directed to the results page, and since $_POST is global, each view takes it and queries it nicely.

Putting it into a panels layout, I can configure it any way I want.

This system works well, as you can check. I’d use BOSTON, MA, as the city to test on, since most of our content is from new England.
The challenges with an argument handler is that it needs to be able to provide the lat/long from other variables than custom modules, (such as node ID-based location info and url-based location info through the argument system) - although even if an argument handler was successfully built, we’d still use HTML forms as it gives us the most dynamic methods of searching. Drupals form API is great but for ad-hoc usage it’s cumbersome and HTML forms are quick and easy. For security, passing all user-submitted values through check_plain() breaks the balls of any would-be troublemakers.

I've attached the handler. Drupal doesen't like .inc files so I've added it as a zip. The magic starts at about line 88.

I'd be happy to post the proximitysearch module but honestly the maintainers know more about DB queries than I do and could write it better, and its a pretty simple set of queries.

Please let me know if there's anything else I can do to help. Thanks.

hutch’s picture

There are two kinds of proximity handler being discussed here, argument and filter. I am currently reviewing both.

Thanks for the example in #41, I will look it over.

There are various ideas in the patches in #4, #26 and #30. I am attempting to amalgamate some of these into one argument handler, probably called location_handler_argument_location_proximity.inc as it is related the filter handler in #41.

I'm thinking along these lines ATM
Keep lat/lon pairs as comma delimited as this is common practice elsewhere.
Use underscore as the default separator but allow this to be altered.
Distance can be expressed as miles, kilometers, meters or lat/lon pair (dd)
Postcode enabled in conjunction with country, but only if the required function exists for that country. No point in flogging dead horses.

more later.

hutch’s picture

A couple of files attached, a patch on location.views.inc and a .txt file to be renamed to location_handler_argument_location_proximity.inc and put in handlers
It is a rework of the ideas in #4, #26 and #30 as proposed in #42

Distance in Decimal degrees only works with Rectangular Proximity but it does not have to be square.
There is a good page on Decimal degrees at http://en.wikipedia.org/wiki/Decimal_degrees

So if your postcode is XYZ you can have arguments like
XYZ_10 where 10 is the distance in whatever unit you selected
or
XYZ_0.5,0.5 for when you set up with Decimal degrees. You can make long thin rectangles or short fat ones

Postcodes will only work with countries that have them set up with entries in the zipcodes table. UK users may want to look at #789426: uk postcodes

Please test folks and report back.

rc2020’s picture

@hutch - a problem with post codes as it's problematic for users to know. Lets say I want to find somewhere to eat in San Francisco, California. Who knows the zipcode of san fran other than people that live there? Zipcode based searching might be convienent from a systems point of view, but from a usability point of view it's highly problematic - city/state is the way most individuals understand how to specify a location.

Thoughts?

I'll try to put your patch on my development installation shortly.

Thanks!

hutch’s picture

I take your point about users not knowing postcodes, absolutely right.

However we are after a single point on which to base the proximity search.
I just ran the following SQL on zipcodes:

SELECT * FROM `zipcodes` WHERE `country`='us' AND `state`='CA' AND `city`= 'San Francisco'

result gave me 71 points, which one to use?

You could use decimal degrees perhaps picked up from elsewhere
37.78,-122.46_100

(the lat/lon is just a guess)

So you need to find the center point of the area to be searched, in lat/lon, that's what the proximity search runs on. The zipcodes table is not the right tool for searches by city/state.

My two bits worth :-)

rc2020’s picture

Yeah, why can't we just query the zipcodes table with the city/state values? Thats what my query is based off of - and then pass those decimal degrees from there? If it returns more than one, why not just use the first value?

hutch’s picture

You could just use the first value, but that's a bit rough.
There just might be an answer in here:
#578216: proximity filter wont work with german zips, the error is the missing location_latlon_rough_de()

The patch in #6 apparently makes a geocode search under certain circumstances, more tomorrow, it's late.

yesct’s picture

I just want to say the collaboration here is inspiring! Rock on!

hutch’s picture

A correction in location_handler_argument_location_proximity.inc2_.txt, use the attached instead

rc2020’s picture

@hutch - well, what I was doing in my proximitysearch.module was a foreach loop from all the values, so actually, I think it's the last value I'm using:


function proximitysearch_lat() {

if($_POST['city']) {
  $city = $_POST['city'];
} else {
  $city = $_GET['city'];
}
$city = check_plain($city);
if($_POST['state']) {
  $state = $_POST['state'];
} else {
  $state = $_GET['state'];
}
$state = check_plain($state);


$latitude = "SELECT latitude FROM {zipcodes} WHERE city = '$city' AND state = '$state'";

$latresult = db_query($latitude);

while ($lat = db_fetch_array($latresult)) {
foreach ($lat as $key => $value) {
	$latcoord = $value;
		}
	}
	return $latcoord;
}

Now, this probably isn't any better than using the first, since I wasn't aware initially that there was more than one value per city/state, but in reality we need some method of prioritization. Zipcodes are effectively useless, unless we run a sub-query to get the zipcode from a city/state, and then pass that into the query.

I'm all self-taught with php and sql so if there's some no-no's with my query please let me know - but there still needs to be some overarching method to get the most accurate coordinates from a designated area. Perhaps we should use a new zipcodes table?

In a perfect world, we could integrate with some 3rd-party tables that produce lat/long values - they certainly exist. I don't know how secure it would be to continually talk to this, or we could include updated tables with each new location release. The world is shrinking every day, so international codes are going to eventually become a must. I'm mirroring my development installation shortly, so I should be able to test your patches and respond back - but perhaps this might be a good opportunity to look at the system from a macro level and see if we can improve the lat/long queries out of the gate.

hutch’s picture

As you are already cycling through them all. why not collect the highest and lowest of each and use the average? That would give you the middle of town.
Something like

function proximitysearch_coords() {

if($_POST['city']) {
  $city = $_POST['city'];
} else {
  $city = $_GET['city'];
}
$city = check_plain($city);
if($_POST['state']) {
  $state = $_POST['state'];
} else {
  $state = $_GET['state'];
}
$state = check_plain($state);

$sql = "SELECT latitude, longitude FROM {zipcodes} WHERE city = '$city' AND state = '$state'";

$result = db_query($sql);

$maxlat = 0;
$minlat = 0;
$maxlon = 0;
$minlon = 0;
while ($row = db_fetch_array($latresult)) {
  if ($row['latitude'] > $maxlat) {
    $maxlat = $row['latitude'];
  }
  if ($row['latitude'] < $minlat) {
    $minlat = $row['latitude'];
  }
  if ($row['longitude'] > $maxlon) {
    $maxlon = $row['longitude'];
  }
  if ($row['longitude'] < $minlon) {
    $minlon = $row['longitude'];
  }
}
$lat = ($maxlat + $minlat) / 2;
$lon = ($maxlon + $minlon) / 2;
$coords = array('latitude' => $lat, 'longitude' => $lon);
return $coords;

The last 4 lines could be condensed into one for efficiency

rc2020’s picture

Nice - that's a genius idea! Perhaps we should include some variant of this module for either HTML forms and FAPI forms (I like using both since the HTML crowd is a little more accustomed to HTML forms and the Drupal developers crowd is all FAPI). I love where this is going...

yesct’s picture

I'm collecting bounty pledges to get this fixed (and thus hopefully a new release out). I can chip in $50. Who else? I'm guessing a figure like $400 (is that realistic?) might really get some movement to get this fixed in a good time frame. I'm optimistic and hoping for 2-4 weeks? I'm also totally OK with the bounty going to the maintainers, or splitting it among major contributors. I think some sponsorship of their time is totally OK (just think Earl Miles gets/got paid for all his awesome work by Sony).

So... what would be best here? Maybe people could reply with an amount they want to contribute, and their criteria. And also, anyone that wants to claim some of the bounty could reply and then we can coordinate and plan.

[had to take off a tag in order to add the bounties tag...]

farald’s picture

Ill hereby add $20 to that bounty.

hutch’s picture

@YesCT, I think you/we need to define a lot more clearly what the problem is and what the solution should do, the code I chucked in at #51 is a crude solution, works fine if a city is evenly spread around but might come up with weird results on a lopsided city or a uneven postcode.
If you search in googlemaps for instance using their get driving directions facility (or with my getdirections module) google returns a city center if you do a very general search eg 'London, UK' or 'New York, USA'. Wether there is a way of collecting this as lat/lon serverside and saving it to a database table I don't know.
The http://drupal.org/project/gmap_direx module attempts something like that using google's directions API functions.

Some grist for the mill ;-)

yesct’s picture

Yeah, I agree we need to define. Maybe Ankur can help define?

My definition: a patch (or contribution to a patch) that is RTBC'd and committed. :)

juicytoo’s picture

I'll support this effort by contributing $20.

Not much, but if everyone can contribute a bit, it will go along way.

As an aside, joomla sobi have a solution for this

http://www.sigsiu.net/addons/radius_distance_search_plugin.html

Hopefully the proximity search will also include javascript downdrop for suburb/city.

Ed

rc2020’s picture

I'm happy to work with hutch to implement a solution - I really like his solution based off my code in #51.

My caveat is that the argument handler accept raw php code for a lat/long resolution because I'm going to run this with HTML forms regardless, I could use FAPI but it'll just be too much rework for me. Since the majority of the work will be getting it integrated for an argument handler as opposed to a hacked proximity handler I think hutch (or whoever implements the actual argument handler) should get the lions share of the bounty, but I'm down for testing/moral support/design help and if it's a collaborative effort I'll take the smaller share as I'm unable to really put my code as a straight up views handler.

Hutch, my email is cmacpher at gmail dot com if you want to message me on how we can work together to tackle this I'm down.

Talk to you bud -

matt2000’s picture

For the record, no one should use the code in #50 and #51 as it is vulnerable to SQL injection attacks, due to the failure to use proper placeholder tokens in db_query(), unless I'm really missing something here.

And generally speaking, hacking around FAPI is a bad idea and prone to security flaws unless you're very careful.

hutch’s picture

Absolutely correct, the examples in #50 and #51 have not been escaped, here is how it should be done

$result = db_query("SELECT latitude, longitude FROM {zipcodes} WHERE city = '%s' AND state = '%s'", array($city, $state));
rc2020’s picture

This is true and I am/was aware of this but all values were passed through check_plain so I wasn't concerned but in practice, yes, all values should be escaped and any code thats committed should def. be escaped.

matt2000’s picture

check_plain does nothing to prevent SQL injection.

rc2020’s picture

@matt2000 -

if I take a value from user input, say,

$value = $_POST['value'];

and then pass that through check_plain() -

$value = check_plain($value);

and then query the database with the new $value, you're saying that does nothing to prevent SQL injection attacks? According to the API docs on it, it returns this: "An HTML safe version of $text, or an empty string if $text is not valid UTF-8." Is this not valid for SQL injection?

Please advise.

rfay’s picture

First rule of Drupal POSTs: You don't access $_POST with user code. (Use the form API)

First rule of database access: You always sanitize database queries with parameterized (as opposed to concatenated) queries.

rc2020’s picture

I have my reason for using HTML forms as opposed to FAPI, they're explained in earlier posts.

W/ DB access - for sure. I'll be escaping all code in the future if check_plain() doesen't secure me against SQL injection. Thanks for the heads up.

greggles’s picture

Subscribe.

@corona ronin - could you point to the specific comments where you indicate why FAPI won't work?

rfay’s picture

I think the security team will look very poorly on something that uses HTML forms and ignores one of the core security features of Drupal. Not a generally good idea, and opens up the module to security issues down the line.

I did ask coltrane, one of the members of the security team, and he definitely felt like this was questionable and would be an eyebrow-raiser for the security team.

coltrane’s picture

@corona ronin

If you're writing your own form and checking POST yourself, and you *do not* implement a token-based authorization model similar to FAPIs then your form will be susceptible to cross site request forgeries which is not acceptable.

Secondly, the use of check_plain() is the wrong way to protect against SQL injection. You are advised to follow Drupal best practices and use variable substitution for your queries.

The simple matter is to prolong the life of your contribution and aide users and other developers you should follow coding standards and best practices which means using the Form API and database variable substitution methods in db_query() and family.

rc2020’s picture

Okay. Sure - that's all well and good. I'm happy to implement a better and more secure solution but I'm a fast learner but not a born genuis so please bear with me.

1. I used HTML forms as opposed to FAPI is becuase FAPI is significantly more complicated than HTML forms. I started building websites knowing perhaps 4 HTML tags, and taught myself how to do this from the ground up. I build this myself and by myself alone I can only learn so much in a year and a half. I do appreciate the insight posted here and I will take note of it. The big reason reason I use HTML forms is because I need access to $_POST for my functions to work because I access the values on a secondary page. On my frontpage, www.lifeundersun.com, I use the HTML form action to redirect to the page /proximitylaunch, which takes the values of $_POST and passes them to hacked views filters that query the database to return a lat/long/distance respectively. If FAPI can do this, I'm more than happy to use it. Is the submit callback the same thing as form action? Thats a guess because it's not explained for people that didn't go to school for php development. I taught myself php, so, again, please bear with me.

Can I use FAPI to generate forms and access those values on a secondary page? Are those values in $_POST? If not, how do I access them on a secondary pass and pass those values to functions?

2. Understood with SQL injections. I will modify my code accordingly to escape the values and pass them in as arguments to avoid SQL injection techniques - I appreciate the heads up.

Thanks!

greggles’s picture

Great to hear, corona ronin. For general secure code tips see http://drupal.org/writing-secure-code

I learnedthe form_api from http://drupal.org/node/751826. Given the way the code is built so far you may also want to look at the search.module in core and in particular the functions search_box and search_box_form_submit. It uses the form_submit function to redirect to a particular URL using a GET request so that the page can be cached. Handy for expensive queries like distance.

yesct’s picture

I'm happy to see this conversation about security. Location really needs good code to help it live. I'm just learning about writing code the Drupal way too! Thanks to corona ronin for not getting angry and leaving (I might have gotten defensive, depending on how little sleep I had gotten *grin*), and thanks to everyone else for sticking with this and helping bring everyone to the same page. :)

marcoBauli’s picture

Howdy, as per correspondence with YesCT, i'd likely chip in with $50 for this fix. In case it goes in port, please count my bounty in and contact me back. Cheers!

yesct’s picture

summary of bounty: so far $190 (#72 $50, #57 $20, #54 $20, #53 $50).
Criteria seems to be general: A patch that gets committed, probably shared among contributors. How about someone claim this issue to work on it... Hutch, Corona Ronin, Ankur???)
Still opportunity to add to the bounty, if you can contribute, just reply and say how much, and what your criteria is. I'm hoping we can get another $210. Come on, I know you want to chip in $50!!!
Looking to get something by June 15th?

Edit: total $260 (#75 $50)

edit: total $310 (#78 $50)

hutch’s picture

Here is some more stuff towards the location arguments handler.
4 attachment:
1) a patch on location.views.inc. This registers the handler
2) The handler file. This should be renamed to location_handler_argument_location_proximity.inc and dropped into the handlers folder.
3 and 4 are views imports

To get the postcode/zipcode stuff to work you need to have the zipcodes table with your country's stuff in there.
Furthermore you will need to have a function location_get_postalcode_data_xx where xx is your country code or a function location_latlon_rough_xx where xx is your country code. This needs to be in your country's file in the 'supported' folder. Currently not many have that.
UK users may want to have a look at the patch I submitted at #789426: uk postcodes. By the way, if anyone does use this patch could you report back to that thread, I would really like to see it committed. It 'just works' :-)

The arguments can be in the form of
for latlon type view
nn,nn_d where the first nn is latitude, the second is longitude and d is the distance. eg 52.44,-3.54_10

for postcode type view there are two forms
pppppp_d where ppppp is the postcode and d is distance
or
cc_ppppp_d where cc is the countrycode. this will override the default country

d (distance) can also be expressed in decimal degrees, eg 0.1,0.1 would make a rectangular box to search of roughly 10km by 10km
All the settings for this are in the view

I'm sure there's more but I gotta go ;-)

rc2020’s picture

Even though I'm contributing on a few areas here (although hutch is really doing the lions share of the dev. work) and working with testing etc, my company, www.lifeundersun.com, would also be willing to toss in $50 - so @ YesCT we can add that to the tally.

I'm happy to take point working out some FAPI stuff but I have a few questions on how it works as all the FAPI stuff i've done is with form_alter(). YesCT - are you pretty comfortable with FAPI to answer a few questions I have? (basic generic stuff that I just can't seem to have answered in tutorials or documentation).

If so, I'll catch you on Gchat when you're around.

@ Hutch - great stuff, I'll load this on my dev site to test and see if I can migrate my original solution in the drupal way in a fashion that handles all security measures, the new code escapes the arguments anyway so SQL injection attacks are moot but I'd like to create some fields that integrate with this so we can have a more dynamic city/state/zip location query system I'm happy to work on.

Thanks!

vlad.k’s picture

I have the patch provided in #49. Initially the patch seemed to work, but in certain constellations the query results are not correct.

Could it be that there is an errror caused by the international dateline ?

When trying to find geotags near Auckland/New Sealand (which is pretty close to the international dateline) the system causes an error when searching in a radius of 1000 miles. When searching around geotags in Africa the error is only caused when searching between 7000 and 8000 miles.

So I guess that error could have something to do with the distance that a geotag has to the international dateline.

Does anyone have similar issues?

hutch’s picture

To be honest I have only done searches on a very local basis, eg 100 miles max and the patch (#49 and #74) was only tested in that way. I don't have a setup to test over such vast distances.
This looks like a longitude problem so I assume the problem lies in file earth.inc function earth_longitude_range()

My understand of trigonometry can't touch the stuff in there, if there is anyone who does grasp spherical trig if they could look and see if dateline crossing is being compensated for, my intuition is that it is not.

captainack’s picture

I'll gladly throw $50 into the bounty pot. Any instructions on this?

rc2020’s picture

@hutch - I got a math guy (differential equations and the like) I can run this by, can you provide a semi-detailed explanation of the problem for him to grasp? He's compsci and good w/ java but is fairly weak on the LAMP end and knows nothing about drupal/php - but if you can explain it well enough I'll see what he can do.

hutch’s picture

I *think* the problem is with the international dateline. Longitude runs from greenwich (0) eastwards up to +180 and westwards to -180 which is the dateline.
People are trying to calculate distances across this line and apparently it's not doing it right.
As far as I can tell function earth_longitude_range() in earth.inc is not correcting for this (but it might be). Show him the whole of earth.inc, if he can read java this should not be too difficult to follow.

rooby’s picture

StatusFileSize
new8.92 KB

Here is an updated version of the patch in #74

Changes are:
* Should have fixed the problem with longitude and that annoying date line
* Removed the options() function from the handler include file as it is not required and will be depreciated at some point according to views code
* A couple of minor coding standards fixes regarding incorrect spacing

- Also see #821628: earth_longitude_range() in earth.inc uses invalid values with asin() as that is sometimes a cause of the proximity argument not working.

When I get a bit more time I'll have a better look through.

Thanks for all the work everyone has put in :)

The patch includes the location_handler_argument_location_proximity.inc file so just apply the patch and that's it.

hutch’s picture

Looks good! Patch applies fine and the new handler looks good too, those of you with dateline problems should test this
Thanks rooby ;-)

yesct’s picture

Status: Needs work » Needs review

I think this needs review on
1) dateline situations
2) a TODO list for any thing left to fix (hopeful me thinks this might maybe be enough to be working?)

hutch’s picture

I have tested patch in #81, it works for me as expected.

The dateline issue solution lies in earth.inc and is being addressed in #821628: earth_longitude_range() in earth.inc uses invalid values with asin() , at least that's my understanding.

I have recently discovered that the PHP function split() is now deprecated in PHP 5.3.x, the attached patch fixes that by using explode() instead.

It should be applied *after* applying patch in #81. The patch should be copied to the handlers folder and applied there.

It also improves a description slightly. Perhaps some markup could be added explaining how the argument string should be constructed, possibly based on the stuff I chucked in to #74. In advanced_help perhaps?

Hope this helps

rooby’s picture

In regards to the dateline issue:

+++ handlers/location_handler_argument_location_proximity.inc	2010-06-09 00:00:47.000000000 +1000
@@ -0,0 +1,207 @@
+    // Add MBR check (always).
+    // In case we go past the 180/-180 mark for longitude.
+    if ($lonrange[0] > $lonrange[1]) {
+      $where = "$this->table_alias.latitude > %f AND $this->table_alias.latitude < %f AND (($this->table_alias.longitude < 180 AND $this->table_alias.longitude > %f) OR ($this->table_alias.longitude < %f AND $this->table_alias.longitude > -180))";
+    }
+    else {
+      $where = "$this->table_alias.latitude > %f AND $this->table_alias.latitude < %f AND $this->table_alias.longitude > %f AND $this->table_alias.longitude < %f";
+    }
+    $this->query->add_where($this->options['group'], $where, $latrange[0], $latrange[1], $lonrange[0], $lonrange[1]);

This is the part that handles crossing the dateline.
[edit] This code snippet is from the patch in #81

#821628: earth_longitude_range() in earth.inc uses invalid values with asin() is also related but is only the cause of edge cases in comparison to the main issue, which is what the above code fixes.

Powered by Dreditor.

hutch’s picture

Thanks for the clarification.

rooby’s picture

For further clarification (I didn't explain properly before)

The asin issue is unrelated to the date line but causes proximity calculations to be inaccurate for large values of distance.
I have documented the problem more thoroughly in #821628: earth_longitude_range() in earth.inc uses invalid values with asin()

rooby’s picture

StatusFileSize
new8.97 KB

Here is an updated patch which is the patches from #81 & #84 combined.
I was going to make some changes based on the changes in #718928: add options to set coordinates w/ user location, PHP code for field, filter but none of them really apply here.

hutch’s picture

The patch is #88 is working fine, I have also got it running with #718928: add options to set coordinates w/ user location, PHP code for field, filter patched in at the same time on a fresh copy of Location (13 June). This is using 'classic' location, no CCK location. I did test #88 using CCK location and only the lat/lon method worked (as I expected)

Woohoo! Good stuff.

rooby’s picture

I have successfully tested CCK locations with the post code argument.

Like with most other dealings with node views and CCK location fields it requires the 'Content: Location' relationship for the CCK location field you want to use.

If you are having any problems with the post code proximity stuff not working make sure you have uploaded the zipcodes database file for the country you are trying to use (see installation point 6 in INSTALL.txt for more information). Also note that the au & be files are wrong so they don't work properly for this use. See #830352: postcode database files for au & be have lat/lon reversed.

summit’s picture

Subscribing, greetings, Martijn

sewid’s picture

Subscribing

rooby’s picture

If any new subscribers have time to test the patch that would be cool.
It should be going in soon and the more testing we can get before then the better.
Thanks.

sewid’s picture

I've already updated to the latest dev-build (june 25), should the patch work with that build?

If yes, I will try it today in the evening.

hutch’s picture

patch works fine with current cvs

sewid’s picture

Hi,

I've just tried the patch with current dev version - everything works as expected, great job!

Best regards,
Sebastian

sewid’s picture

An additional nice feature would be, to allow only some ranges, so that I only may add a postal code and e.g. 5, 10, 25 and 50 as radius. If somebody modifies the url, so that a radius of 75 is set, an 404 (or error-page) is shown.

rooby’s picture

@sewid:
Thanks for testing the patch.
It would be possible to add that but I think it would be unnecessary extra code because views already has validation settings that could handle this.

For example, in your argument select 'PHP Code' for the 'Validator:' option.
Then in the 'PHP validate code:' field enter:

$arg_parts = explode('_', $argument);
$allowed = array(5, 10, 20, 25, 50, 75, 100, 125, 150);
return in_array(array_pop($arg_parts), $allowed);

In the first line change the underscore to whatever your separator is.

Admittedly it is made a little complicated by the complexity of the argument but it isn't too bad.
What do other people think?
Should we provide a simpler option like entering a comma separated list of allowed values or just leave people to use the views PHP Code validation?
Another possibility is to document this code snippet.

sewid’s picture

Thanks for the validation code.

Another point I found is the underscore as separator. In my opinion, the nicest solution would be a slash as separator, but I think that is not possible, because that would be two arguments. Or is it possible to make views accept two arguments as one? If not, why is "-" not allowed as separator? I don't like the underscore, that's old scool. But maybe this is only my opinion. :-)

rooby’s picture

No worries.

A slash separator is definitely not a good idea.
A dash though I can't see anything wrong with. Actually now that I think about it:
I think it was hutch that added that part so he can confirm but it could be related to post codes.
Are there any countries with dashes in their post codes?

@hutch:
Is the reason for excluding dashes related to post codes?

ankur’s picture

About dashes and underscores:

I think underscores may be the way to go. Commas get encoded when included in URLs. Dashes don't. Underscores aren't the best looking and my personal preference would be semi-colons, but I think those are reserved for special use for some protocols, not to mention that they also get url encoded.

hutch’s picture

The reason for excluding dashes is that you might have a minus value longitude. Commas are also in use, used to split lat and long.
The other characters in the $not_allowed list are either already in use in some context, might be considered a hack attack or just didn't work.

I have to admit that I didn't think of url-encoding at all, should have done.
@ankur, could url-encoded characters be un-encoded? Is it worth the hassle?

Personally I would be happy if underscores were hard-wired, it works and would reduce the code a bit.

ankur’s picture

URL encoded characters could be decoded. In face, they might already be decoded by the time the views handlers are called with the argument from the URL. However, encoded, they look less pleasant than the underscores, which don't need to be encoded.

rooby’s picture

Man, I can't believe I didn't think of negative coordinates :)
Also, all numbers will have to be blocked and alpha characters are common place in post codes around the world so if we have all these symbols blocked and all alpha numeric blocked there is not much need for an option.

I have nothing against hard coded underscores and no option.
Simplified code is a plus too.

rooby’s picture

Here is a version of the patch in #88 with the separator option removed.
It also adds some more description to the Coordinate Type option so people know the format of the argument.

+++ handlers/location_handler_argument_location_proximity.inc	2010-06-16 01:23:07.000000000 +1000
@@ -0,0 +1,207 @@
+    /*****FIX BEFORE COMMIT ****/
+    // What is group and why do I need to set this to work??
+    $this->options['group'] = 0;

I also removed this part.
I have seen not loss of functionality. Do you remember why that line was needed before?

Powered by Dreditor.

hutch’s picture

patch in #105 is working for me, tried with and without country preset. Also tried it in meters with rectangular and circular and postcode

$this->options['group'] = 0;

This has been there since at least February or March, no idea why.

BTW
I ran location through coder, lots of cleanup to do, mostly @file block missing

As far as this thread is concerned:
location_handler_argument_location_proximity.inc
* severity: normal Line -1: @file block missing (Drupal Docs)
* severity: normal Line 121: Use "elseif" in place of "else if"

Anyway this patch is the biz for me, go for it.

rooby’s picture

Status: Needs review » Fixed

Committed to 6 & 7.
http://drupal.org/cvs?commit=387884
http://drupal.org/cvs?commit=387882

Thanks everyone for all the work!

I also added the @file block.
I have a habit of using else if due to other programming languages and I don't feel it hurts readability (I would actually say the space enhances readability).
I don't mind committing patches with elseif or else if but I think it is a non-issue.

yesct’s picture

#73 shows a current total of $310 in bounty. Yeah! Who wants it? I suggest a split between rooby, hutch and ankur. Dont be shy, if you feel like you worked to get part of the bounty just say so! Use my contact form too if you'd rather.

itserich’s picture

New here, hope this is not the wrong place to post it, but the development download is not working.

http://ftp.drupal.org/files/projects/location-6.x-3.x-dev.tar.gz

Oops! This link appears to be broken.

yesct’s picture

@itserich, your instincts are right, it needs a new issue. :)
I made one: #850724: http://ftp.drupal.org/files/projects/location-6.x-3.x-dev.tar.gz not found

q0rban’s picture

yesct’s picture

Thanks q0rban!

rooby’s picture

Re the bounty: I'm happy to help out so I don't really mind, I think the only part I did for this patch was fixing the problem with areas crossing the date line.

Also, I'm going to be on holiday for two weeks as of this weekend so I won't be replying to issues during that time.
Then when I get back all fresh I'm going to get stuck into the issue queues.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

ankur’s picture

In reply to #108 above, I wanted to say that I honestly don't deserve any of the bounty as I only made suggestions and never uploaded any patches for this issue. I think the bounty should be split among any one that uploaded a patch to this thread as well as YesCT, for organizing and driving this thread. Perhaps it can be weighted in favor of hutch and rooby for their share of contributions towards the end.

yesct’s picture

I finally sent my bounties (via paypal)!
I'll try and contact others that put up bounties and provide you with the paypal email addresses to use. If anyone wanted to chip in, just contact me, and I'll get you the emails too.
Thanks!

vkr11’s picture

Subscribe

Vladimir K’s picture

more simple code, it calculates center based on all points, not only max and min

$center = db_query("SELECT avg(latitude) as lat, avg(longitude) as lon FROM `zipcodes` WHERE city = :city and state = :state", array('city' => $city, 'state' => $state))->fetchObject();
echo $center->lat;
echo $center->lon;
Vladimir K’s picture

Vladimir K’s picture

manoloka’s picture

mmm ... so how is this going as for today, did anyone sorted this out? As it seems that it's not working.

I'd be happy to chip in something for the bounty if that's not sorted yet.

After so many patches, could someone list the final working solution?

Thanks
Manuel

rooby’s picture

@manoloka:

This was fixed and committed to the module a very long time ago (see comment #107 for commits for what was done).

If you are still seeing some sort of problem when using the latest release please open a new issue and describe exactly what about it is not working.

Thanks.