Posted by lyricnz on November 25, 2008 at 4:45am
7 followers
Jump to:
| Project: | GeoNames |
| Version: | 6.x-2.x-dev |
| Component: | Code |
| Category: | feature request |
| Priority: | normal |
| Assigned: | lyricnz |
| Status: | active |
Issue Summary
I am aware that this module currently supports invoking the Geoname webservices, rather than using the data. But it would be useful, and relavent for this module, to be able to import/search data from geonames.
This may be used to improve performance, reduce the number of hits on the provided webservice, etc.
Comments
#1
I do very much agree on this. Subscribing
#2
Could you outline your idea? Do you picture maintaining a copy of the GeoNames data using the daily dumps?
(with caching enabled, the webservices actually are quite efficient. We process thousands of GeoNames requests daily, with the large majority of the requests being delivered lightningfast from the cache... but performance and requests will of course depend on your config&implementation).
.. and btw I am also positive towards this idea - it will require a lot of work.... ;)
#3
Well, I was thinking about high-volume requests, especially for the really low-level things like country-info and cities. I wouldn't want to hammer their webservices unnecessarily, see http://geonames.wordpress.com/2009/01/25/what-to-do-against-ddos-effects/
No, I wasn't planning on keeping in sync with geonames. Perhaps provide some bash scripts to pull the data, and push it into the tables - could be done once, or every so often.
#4
I wonder if this might be easier to consider if we break this problem into smaller tasks. Simply copying the Geonames data to a local database table isn't enough. Geonames web service does proximity searching and hierarchy, which won't be supported simply by keeping a local copy. The downloadable data also doesn't represent hierarchy in a transparent "normalized" way that RDBMS would want.
I don't have a solution to this problem, but I can see some missing features that would make this a little easier to contemplate.
1. Write a normalized (Drupal) data schema for the data in allCountries.txt.
2. Write Views 2 integration for all these tables.
3. Write a function or script to convert the allCountries.txt file to SQL inserts on one or multiple related tables.
This does not resolve the problem of caching data locally, but I think it may be a good start.
#5
This could be quite easy with Feeds I assume :)
http://drupal.org/project/feeds
#6
Getting the data into Drupal isn't that big of a deal, but *using* it from Drupal is more problematic - since we'd have to reimplement the API functions provided by geonames.