Format of countryInfo.txt has changed, breaking geonames cache

Yes, of course we should have same format as the source data (but the issue above refers to the countryInfo format, while your posting is about the main geoname format).

Just open http://download.geonames.org/export/dump/countryInfo.txt in your browser, and look at the column names. These are not the same as the "main" geonames format.

We've thought about this, of course. These are two facets to this;

- making it possible to load the data-dumps from geonames, and then query/search within this dataset (which will work very different to the existing web-services)

- using this data as a "local cache" for some parts of some queries.

Can you reroll, post code-style cleanups just committed?

Applied, thanks!

hook_update_n required

jmary: why is this critical? just disable and uninstall the module, then reinstall it - and the table will be created correctly. The outstanding issue is just around the fact that the module doesn't update cleanly.

jmary: did you actually _try this_? (using 6.x-1.x-dev)

It works fine. Here is a log of my session: I downloaded geonames into a default drupal6 site, unpacked and enabled it (which creates the table). I then downloaded countryInfo.txt straight from geonames.org, removed the leading comments, then use mysqlimport to load the file into the database table (checking number of records loaded against the number of lines in the file). I then queried the DB to verify that the data looked okay.

simonmac:~ simonroberts$ cd htdocs/drupal6/sites/all/modules/
simonmac:modules simonroberts$ curl http://ftp.drupal.org/files/projects/geonames-6.x-1.x-dev.tar.gz | tar xvz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 28407  100 28407    0     0  16070      0  0:00:01  0:00:01 --:--:-- 19496
geonames/
geonames/LICENSE.txt
geonames/CHANGELOG.txt
geonames/GeoNames_API.txt
geonames/README.txt
geonames/geonames.info
geonames/geonames.install
geonames/geonames.module
geonames/geonames_config.inc
geonames/geonames_search_API.txt
geonames/geonames_tools.info
geonames/geonames_tools.module
simonmac:modules simonroberts$ drush enable geonames
Initialized Drupal 6.13 root directory at /Users/simonroberts/htdocs/drupal6                                                                                                                                              Initialized Drupal site default at sites/default                                                                                                                                                                          The following modules will be enabled: geonames
Do you really want to continue? (y/n): y
GeoNames API was enabled successfully.                                                                                                                                                                                 Command dispatch complete                                                                                                                                                                                                 simonmac:modules simonroberts$ curl -O http://download.geonames.org/export/dump/countryInfo.txt 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 29667  100 29667    0     0  14904      0  0:00:01  0:00:01 --:--:-- 18681
simonmac:modules simonroberts$ grep -v '^#' countryInfo.txt > geonames_countryinfo.txt
simonmac:modules simonroberts$ mysqlimport -u drupal6 -p drupal6 -v $(pwd)/geonames_countryinfo.txt 
Enter password: 
Connecting to localhost
Selecting database drupal6
Loading data from SERVER file: /Users/simonroberts/htdocs/drupal6/sites/all/modules/geonames_countryinfo.txt into geonames_countryinfo
drupal6.geonames_countryinfo: Records: 247  Deleted: 0  Skipped: 0  Warnings: 405
Disconnecting from localhost
simonmac:modules simonroberts$ wc -l geonames_countryinfo.txt
     247 geonames_countryinfo.txt
simonmac:modules simonroberts$ drush sql query 'select iso_alpha2, name, geonameid from geonames_countryinfo limit 10'
iso_alpha2	name	geonameid
AD	Andorra	3041565
AE	United Arab Emirates	290557
AF	Afghanistan	1149361
AG	Antigua and Barbuda	3576396
AI	Anguilla	3573511
AL	Albania	783754
AM	Armenia	174982
AN	Netherlands Antilles	3513447
AO	Angola	3351879
AQ	Antarctica	6697173
simonmac:modules simonroberts$ 

1) It is not possible to include the actual data from geonames with the module, as it is licensed under Creative Commons Attribution, which is incompatible with GPL, which Drupal+Contrib is released under.

2) The format of the table created by this module matches the current format of http://download.geonames.org/export/dump/countryInfo.txt - any problems with tools like phpMyAdmin etc are not the problem of the module, as such.

3) Likewise other contributed modules (you mention gmaps) aren't our responsibility. We updated the table structure to match geonames.org - they should use our API functions, or update their queries to match the new structure

4) Agree that geonames_metadata_update() is broken. It should be updated, along with a new hook_update_nn() to allow the module to be upgraded cleanly.

4. This module never claimed to fill in the geonames_countryinfo table at install time, and there is currently no code that would do that. However, if geonames_metadata_update() was working it would load this table on the first cron call (and every month thereafter). Arguably, there should be a more human-friendly way to trigger this refresh. Feel free to file that as a feature request.

Added hook_update support to 6.x-1.x-dev

Updated geonames_metadata_update() to handle new columns, and use schema API. Committed to 6.x-1.x-dev.

jmary: If you want an easier way to refresh this metadata (than deleting the variable that keeps track of the interval, and re-executing cron), please don't hijack this issue - file a separate feature request.