Some oddities in number.module

The attached patch corrects the issue *on english systems* by stripping out all non-numeric characters other than periods before saving, and calling number_format() on the value before output. It's a starting point but anything longer term needs to be considered in terms of globalization.

Any thoughts? Anyone?

This issue makes number.module pretty much useless without extensive theming and *very* careful data entry.

Good idea. Numbers are numbers in all character sets I've worked with, even in Chinese, so I think this will work to clean up the input and it seems like an good initial pass at fixing the output. We probably need to create a group of formatters for the output to match up with various methods of marking groups and decimals. I'll do some testing.

Does Drupal use the php function setlocale() anywhere? The locale module doesn't do it. If that gets set to 'LC_ALL', we can use localeconv() to get the necessary number formatting info for number_format() and easily provide a group of locale-specific formatter options for the output.

On further thought, I'm not sure we can use the set_locale() option because of CCK caching, so instead I created a group of formatters to display a number in a number of different formats and added that to Eaton's patch to clean up the input, then did some testing. The input seems to correctly strip out things like dollar signs and the formatters seem to work to display the output in various ways. Patch attached.

There is one remaining problem. Large numbers still get turned into "1.5e+06", which it turns out is because the integer_decimal field is storing the value as a float. We need to store that value as a decimal (in MYSQL) which means we need to alter the table. So I need two bits of information:

1) How many decimals shall we use? I'm thinking 2 is enough.
2) What is the equivalent to DECIMAL(10, 2) for postgres?

I did some more digging and altering the data storage method is going to be a major patch since CCK has no provision for a decimal storage method, only float and integer, which means that we have to touch every place in the code that creates and manages storage methods to add a new method, make sure it works in both MYSQL and postgres, and create an update for the number.install to change the storage type and take the values stored as floats and convert them to decimals.

I propose we break this into two bits. Cleaning up the input and formatting the output would be addressed by the patch I attached previously. So I'm marking this patch ready to be reviewed, and I'll open a new issue about changing the database storage method.

I opened an issue for the database storage at http://drupal.org/node/109246.

Sorry for my ignorance, but what is the exact problem with the scientific notation? Calculations should still work, and for the output you could just write a function to convert to the normal representation. For storing decimals, couldn't you simply store them as integers multiplied by 10^(#dec) when you have a fixed number of decimals anyway?

Well, maybe I misunderstood. With this patch it is formatted correctly, just stored as scientific notation in the database. If that's sufficient, we're done. I was assuming, maybe wrongly, that we needed to actually store it as a decimal value in the database, maybe so the raw data can be used elsewhere.

...maybe so the raw data can be used elsewhere
I guess Views is the main possible issue here ?
The output goes through the formatters anyway, but we have to make sure that filters (<, >, etc) and sorts, that are made using the raw db value, work ok.

I went ahead and committed the fix to the formatter and Eaton's check to strip out non-numeric data. The discussion about whether a new storage type is needed can continue in that separate thread.