Country regions

Some of your concerns about the addressfield overlap are covered here: #1071552: Minor overlap and collaboration with Countries module. Location would have a similar argument. I'm more interested in a regimented DB for storing hierarchical data, and integration with either / both would be an awesome solution to adding simple address fields to Drupal!

For example, it would be as simple as typing 3 or 4 letters and using the arrow keys, to populate city, region and country data in one step.

I meant to have the hierarchical data with a number of required additional fields, namely the type of region stored (for example the top administrative region, second level, third, city, ...). I'm still playing around with the idea, but looking ahead for better integration with the address field-like modules

The addition of the core country list would make this unlikely to be the basis of the other modules, but IMHO, it would allow for easy to extend. Using cck_phone for example, the validation rules could be stored as a simple field on the country and this referred to validate the number field supplied by cck_phone. (IE: Least work would be required for that maintainer when attaching the data to the country bundle directly - no administration interface needs to be created for this.)

While I've got this working, there are far too many issues to resolve. Postponing, maybe I'll get to this for D8...

I've done a lot of development of this. The current version is not releasable as it contains about 35% non-related code for the project that I was playing with. But some points to date:

Latitude and longitude data
I added lat / lon columns to the table to enable geolocation searches. I actually implemented a simple element for this, but I do not think that it is the right place for it. I'm not sure what the status of the other related modules are to hook into their api atm.

If a field is used, there is no overhead on the module, but we can not supply a data source to the table that provides this info. Maybe we could supply the location data, but leave it to a submodule to deal with.

My thoughts:
Provide the base data with the default regional data.
Provide a submodule that directly alters the core table and adds this functionality using its own FAPI element, and hooking into GeoField module for additional functionality such as validation, etc.

Postcodes
The other related and common requirement is the postcode data. I think a varchar column that will allow regex would be the way to go here. Direct searches would pick up exact matches and validation could be done using the regex expressions
ie:
QLD, AU [0-9]{4,4}
Oxley, QLD, AU 4304

Again same issue. Adding it complicates things but provides seemless integration to the module.

My thoughts would be to have a separate module to handle this that alters the base table and has it's own country specific settings.

Hierarchical classification
It is not a good idea to classify the regions during data entry. This would prove very complex to validate and control the regional data. I haven't implemented this yet, but I was thinking that each country could define its own limits.

Something similar to this could be used to represent NZ.

Here is the base classification that I came up with. This was based on about 4 or 5 classifications that I found at the time (Google, Yahoo, GeoNames, Location module from memory).

COUNTRIES_REGIONS_C1 => t('Major country division'),
COUNTRIES_REGIONS_C2 => t('Minor country division'),
COUNTRIES_REGIONS_C3 => t('Region grouping'),
COUNTRIES_REGIONS_C4 => t('Region / State'),
COUNTRIES_REGIONS_C5 => t('Sub-region'),
COUNTRIES_REGIONS_C6 => t('District / City'),
COUNTRIES_REGIONS_C7 => t('Sub-district'),
COUNTRIES_REGIONS_C8 => t('Town / Suburb'),
COUNTRIES_REGIONS_C9 => t('Neighbourhood'),

This is needed to map different modules, (Address Field, Location, etc) with the data. So this enables the capability to provide an autocomplete to the Location field that is hooked to the regional table.

Population
While this seems like a pointless field, it actually is very useful in weighting the results of a search query.

I think that this is so trivial that it should be added directly into the base module.

Data sources
This is the biggest issue at the moment, albeit not at all limiting to development. The core ISO3166-2 data would be all that I would want to ship the module with, but a method for importing and sharing external data sources would be great.

I would like to see all importing functionality separated into separate modules with the exception to a CSV import that allows you to specify either the core ISO3166-2 file or an uploaded file.

We could even create a website that would allow users to enter and export this data themselves allowing it to grow with no import required by us. It took me about a day to import and clean up NZ for the module and the results were not that great, though one of these reasons is that NZ has districts that overlap over one or two regions (not something that I would want to support!!), and I was trying to map these as much as possible. While the import was easy, fixing up 5,000 records was painful.

Right, I'll keep cleaning things up and hopefully re-upload a zip latter today or tomorrow.

Here is a zip of the progress. I'm really happy with the way it is coming together. By default, country regions are fixed to four levels, but this is extendable to the full nine levels. After moving the type from the region object to the country settings, I instantly found the use cases where this didn't work. So by default, regions are mapped automatically to the correct regional type as defined by the country settings, and this info is stored against the region in the database. However, I added an override that meant that this region and its siblings could be decoupled from this restriction. Sibling children are then classified based on the overridden regions type.

On the todo list (mostly trivial now):

Create a new project Countries Regions
Create a region field and number of formatters.
Create a simple select list based widget. The autocomplete one is good, but it will get a number of issues.
Update the import procedure. I'm thinking that an XML based one is best.
To have a corresponding xml export. (no urgency)
Release alpha / beta release.

The code has bits n pieces of the old functionality of the fields and imports. Ignore this for the meantime.

And there were some major errors found in the countries configuration sub-module, work from the latest git version of the countries module

The zip this time

I've created a options list widget for the regions field.

There are two submodules, one for lat / lon and another for postcodes. These provide zero functionality except for adding the capability to store this data against the regions and country,

There is no internal data (apart from some default country coordinates), but I'm about to tackle a GeoNames integration module. This will be when the issues appear in the design (hopefully minor).

This one depends on the patch at #1292044: Code cleanup and filter changes.

Progress update.

Base import of some additional countries data and codes from the geonames server. Doesn't sound like much, but that the file downloading, processing of the zips and batch framework complete. Hoping to get the import done in the next day or two, and I'll post an updated zip then.

Yep a sandbox may be a nice idea. I think that this should be pushed into a separate project in the long run. I'm handling data that is coming in from the geonames project and hit additional requirements almost straight away, like the need for multiple parents. I really do not want to implement a true graph algorithm for this, instead sticking to the old tree model. I'm trying adding related regions, where these act as one while having different parents. I think that the benefits of denormalisation will outweigh the negatives here. Hoping to have a working prototype for this idea in a day or two.

Reasons why I think that this is important is that the first 3 out of 4 countries that I worked with had regions that has overlapping parental regions. Local politics do not make life easily sadly. :(

This looks like a great option for out of the box functionality and will probably cover most use cases for a regional database.

I think that this functionality will cover 95% plus of use-cases, but sadly not mine. I've created a sandbox project with my fully custom version of handling the database c/f location taxonomy that piggy backs on the core taxonomic system.

The sandbox is here: http://drupal.org/sandbox/aland/1311114

The sandbox project is there, but it probably needs some love and assessment if it should continue or not.

Cool. I never got past the initial developmental stage :)

I guess I should promote the project from sandbox?