Preventing display of extraneous delimiters in addresses

For example when an address with the form

[Street]
[City], [Province]

is being displayed and all of those fields are empty, addresses 1.06 will display just a single line containing a comma. Since the US address uses this form and ends up being the default format for a lot of cases, this would seem to be a common problem.

I've written a patch to fix this. It should handle all of the address formats distributed with version 1.06.
I have very little time for thorough testing right now, but it does pass preliminary testing on US addresses.
In particular it will not display any non-token text at the beginning and end of the address format, however none of the current formats use this.

The code rewrites the second part of addresses.inc's theme_address function -- meaning that if you want to use it just for your theme without patching the addresses module, that's an option. IE backup your copy of addresses.inc, apply the patch, copy the resulting theme_addresses over to your theme's template.php and rename it yourtheme_addresses. Then restore your backup. If there is a preferred way to post alternative theme functions, please let me know & accept my apologies.

It accesses the token API at a lower level in order to

• split the address into chunks of tokens and delimiters
• tokenize the tokens in the token chunks (ie convert them into the actual address data
• concatenate the resulting set of address parts and delimiters using "delimiter precedence". This means that as it moves over empty fields (ie address fields whose tokenized value is an empty string) it remembers the highest-precedence delimiter since it encountered a field with a value. It uses the highest-precedence delimiter thus encountered when it eventually reaches another address.

Newlines have the highest precedence, followed by delimiters that contain non-whitespace, followed by whitespace. This actually ends up being about the closest thing to the right way to present addresses that are missing fields.
So for example if someone has given a city but no province, you don't end up seeing a dangling comma at the end of the line; if there would be an empty line, this method sucks it up. (Note that empty lines are fixed as of 1.06 anyway.)

Really this is a hack, and something much more structured should be done if this module is really intended to be used to display incomplete addresses. But it does seem to get the job done for now at least. If it does end up getting integrated into the module, it should probably be optional, and preferably with a way to enable it for specific country formats.