integrate geonames location webservice - 6 million geographic locations and growing!

Hey

Has anyone else looked into linking Geonames with the location module? From what I can see, it would be ideal for projects that need to offer global location searching without needing to get hold of postcode or city databases for each country they're looking to provide location information for...

I'd be interested in trying to code something to get geonames and location.module working together.... but wanted to check:

a) If anyone else is working on this

b) If anyone can see any possible obstacles to this working...

I've got to look into it more (as this would be the first bit of drupal coding I've done) but I would propose adapting the location input form so that:

1) Users first select the country they are using
2) Users type in the city name they want
3) Ideally via AJAX a lookup is performed on geonames to return a list of possible cities
4) If gmap.module is available, they would then click a 'find on map' button, which would locate the chosen city on the map, allowing the final lat-long marker to be more finely positioned (possibly alternative cities would be displayed also...)
5) If gmap.module is not available, they would be presented with the option to clarify which city they meant and to manually tweak the lat-long settings...

This seems to me as if it would provide a simple way of offering relatively global location support... I could also wrap up the lookup code so that it would work with the MySociety Gazeteer as well (http://www.mysociety.org/?p=83)

Subscribing, this seems like a very valuable feature.

Ok,

I've had a first go and getting something implemented. Not quite what I described above yet, but a step towards quick (if not entirely neat) geocoding for anywhere covered by the Geonames service.

Progress & Files
Attached to this post is an updated location.uk.inc for the location/supported/ directory. This uses the geonames web service to pull back the first response for any combination of city name and postcode. Postcode look-ups through geonames on the UK only use the first part of the postcode (e.g. SW1) so pick out the centre of postcode districts only (which sometimes can be quite a long way from the place picked out by the full postcode...). This file also includes the mapping links submitted by geodan previously...

In the next two posts (as I can't work out how to tar the files together / attach multiple files...) I'll attach:

1. location.zz.inc - (to go in the location/supported/ directory ) - a generic location function that uses geonames webservices to perform a lookup for addresses in any country. The coverage of the geonames service is listed at http://www.geonames.org/statistics/ and should be workable for many projects...

and

2. location.inc.patch - (to patch against location.inc in the location/ directory) in order to a) Use the location.zz.inc functions above when no other geocoding exists for a countr - and b) to offer a new country called 'All Country Functions' in the location configuration screen, which needs to be enabled for the location.zz.in function to be available...

So far I've only tested this set-up with the user module (setting a location for a user) and only for a couple of global addresses - but it seems to work at finding (very) rough lat-long co-ordinates when, for example, when I enter a Paris address and postcode, even when France isn't enabled in the location configuration options...

Issues
Geonames webservices are licenced under 'Creative Commons Attribution 2.5 License' and I believe they require (though I can't find exact terms anywhere) a link back to their site in any application using the data (http://www.geonames.com)

The use of the location.zz.inc file isn't an ideal sollution... perhaps the location module needs to include a fall-back geocoding option by standard (perhaps it does already and I missed it?)

Next Steps
Unless there seem to be any major issues with the implementation so far... I'm going to look into work next on:

a) Offering google maps for more precise locations to be selected

b) Looking into an Ajax sollution so that users select which geonames lookup is correct, rather than opting for the first response geonames gives, which may be incorrect...

Second file... as mentioned above...

Third file.... as mentioned above

NB. Forgot to mention - I've been working with Drupal 4.7.3 and the location module 4.7 Last updated: September 18, 2006 - 04:01

Thanks for putting work into this obslogic - looking forward to getting a chance to try it out on my test install soon!

I've just tried it out too (only took me 3 months to get back to it :) and it's definitely a great start - I've had it working for the UK and for Australia, though not for Germany, presumably as it has its own geocoding function already.

If we can integrate this a little more cleanly it'd be great. It could probably be built directly into the module without too much more work.

Hey

Good timing. I've just returned to looking at this again after a couple of months on other projects....

What do you think would be the best way to integrate this cleanly with the module?

I could perhaps put the geonames fall-back function (i.e. the optionally use of geonames when some other geocoder isn't available) into the earth.inc file and then create an option on the standard Location settings page for whether or not the fall-back should be used?

If doing that - then I guess there should be options about what fall-back to use (e.g. so someone could plug in a local copy of the geonames database rather than using the webservice / so that the MySociety.org gazeteer could be used instead of geonames...) - but I'm not sure where those sorts of options should be exposed?

Any suggestions greatly appreciated... I should have some time to work on this a bit tomorrow...

I'd say maybe put the function directly into location.inc, where you've already got the check for the generic fallback. Then add an option to the location -> geocoding page to enable or disable the fallback option. To make it even more flexible, you could reach out for any generic geocoder hooks (geonames, google, yahoo, mysociety, freethepostcode, etc) and then select on the geocoding page which should be enabled (and maybe even in which order they're checked).

Subscribing - sounds brilliant guys

Subscribing

Ok...

I've had a first shot at a more integrated solution to this...

I've not quite worked out how to create the patches properly so I've attached the changed file to this post and the next with .txt extension added on (apparently .inc uploads aren't allowed directly...).

First off: location.generic.inc (to go in the supported/) directory...

Contains the geonames geocoder and should be easily expandable with other geocoders...

Secondly: location.inc

In here I've added inclusion of the generic functions if they are needed, and have added extra logic in function location_latlon_exact to fall back on generic geocoding

Thirldly: location.module

Here I've added the options to the location settings page so that below the list of supported countries there is the option to turn the generic geocoder on or off...

This also puts options for the generic geocoder at the top of the geocoding options page...

That should be it I think...

It seems to work fairly well in the limited testing I've done so far....

Sounds good! I should be testing it out in the next day or so, so I'll let you know how it goes.

Hi,

This all sounds great!
Is there also a way to get the geonames (country, city) in the normal vocabulairy so all other content can have access to it?

Greetings,
Martijn

I don't think you posted your new version of location.module.

I also suggest you go here and check out the instructions for creating a proper patch: http://drupal.org/diffandpatch

Thanks for spotting my failure to attach the updated location module...

Have got myself in a mess with versions of files... so will try and untangle it all next week and will create a proper patch....

Tim

Subscribing

Ok... I think I've managed to create a proper patch which will update all three relevant files...

This version is against location 5.x however...

Hope it works....

Tim

subscribing

tracking...

Sorry for the delay...

Here's an /untested/ patch against 4.7.

I don't have any 4.7 environments set up for testing at the moment, so I've not been able to check that this patch works alright... but there were no major differences between the 5.0 and 4.7 code that would indicate that there should be any problems...

Tim

tracking

tracking

not sure if anyone would be interested in this. i think it may solve most of your problems.

i have written a little module that adds a (configurable) 'map this location' link below the location fields and above the gmap map. the feature is as follows: once you fill in the location information, you can click on the 'map this location' link and it will automatically geocode the location in the map below. it will also populate the lat and long data in the fields below.

this module was written very recently and has not been rigorously tested, but it seems to be working for a few of us. i are using drupal 5.2 and have the most recent versions of the location and gmap modules (at least thats what i built it on and i think the other person who i built it for has a similar setup).

i am interested in feedback on this module and whether or not it solves the problem at hand. i have tried to make sure that it works for all countries, so if you have problems with a specific country please let me know. i will be adding it as a project soon so we can do all the tracking for it officially on the site.

here is the link to the code: http://drupal.org/node/165040

cheers...

Download the Geonames search API for Drupal 5 at the following address: http://geonames.edesign.no/files/geonames_search.tar.gz (until it is made available as a module in the geonames project).

The module supports the free- and commercial service, caching to speed up common requests among other things. The API should be complete (unless I missed something), it fully implements both the parameters and the returned data.

See a quick demo here: http://geonames.edesign.no/

Good luck.

What do we need to have obscurity/Tim's patch committed to head?

Should it be refactored a little to use the GeoNames API module?

It's important to us to try to have it committed because we would like to then modify the code further to use swill's "Improve UI for entering and geocoding of locations" (and have his changes use this patch.)

And I'm not so good at this that I want to be applying multiple patches at once to what will be production code :-P

Also -- and this would be a separate, I hope nonconflicting patch I would work on -- I would like to provide options to intelligently guess a starting location for the country selection and the intitial map location by IP address or from the user's already-entered location (if present, and when creating location-enabled nodes).

And I endorse Martijnhaan's request to use taxonomy for countries, provinces, and cities or to expose these as taxonomy (or in other ways) to the site. For the World Social Forum we need country and city portals.

Eh, so that's our goals, but the question is simply: anything need to be done to get this patch committed?

ben, Agaric Design Collective

for: http://agaricdesign.com/project/wsf_action

Changing component to Data update.

(This is definately something to look into further, +1)

It'd be great to have location module integrated with GeoNames through serosero's GeoNames API module. The less data location module ships with (e.g. the zipcodes for the US and PLZs for Germany), the more current it can be, the more flexible it can be and the less painful it'll be anytime any of that data has to change.

I could not agree more, I am currently on the hunt for just such a solution. It seems like the direction the location module is taking is exactly what we need; we are building a resort directory for our members to browse properties in our club, there will ultimately be many thousands of properties, so ease of adding and browsing properties is paramount.

I have tried many of the available mods and such but I'm thinking this setup would make life easy in so many ways. Mostly it would ensure clean entries of locative data that can be predictively browsed.

I think I am going to have to create a job over at groups because not only do I really want this, I can see many people are on a similar hunt. I have been thinking this over for some time and just about have the job fully outlined, I will post back here with whatever solutions come of it.

If you have some funding available to sponsor work - it might be a good idea to talk to bdragon as he seems to be running a meta project to try to turn location into what many are looking for - see [master][meta] Normalize location table / enable shared locations / fix location deletion / fix revisions / make forms not suck

I could not agree with you more ... ill be contacting him shortly. ^_^

subscribing

Take a look at this.. ^_^

http://drupal.org/project/addresses

subscribing

could someone please compare the approach here with the one in the issue below?

marking this a duplicate of #132541: refactor google geocoding
which might be functioning as a master issue to discuss approaches to geo-coding.

If this is a separate issue, please change the status back, and also re-roll the patch against the current version.

Subscribing.

I personally think that geonames can be used for a lot more than just retrieving the latitude and longitude, it could also be used to lookup other info (for example to accomplish autocompletion of zipcodes/cities, either to fill the zipcodes table on forehand - see also http://drupal.org/node/439638 - or on the fly).

Ideally the GeoNames module can be used as the API to query geonames.org