I have a situation where I have numbers entered by a user, but some of them are optional. If the user does not enter them, I want to insert NULLs into the database. I figured if I passed a null instead of an int, that a null would get inserted, but instead, the integer 0 gets inserted.
I've tracked this problem down to _db_query_callback in database.inc. It seems to be an intentional (if not beneficial, at least in this case) design decision, because the argument is cast to an int. The comment given is:
// We must use type casting to int to convert FALSE/NULL/(TRUE?)
Well, true and false I understand, but NULL has a meaning in an integer column; it means null!
I would purpose we change _db_query_callback so that if the arg is null, it returns 'NULL'.
For example:
case '%d':
$arg = array_shift($args);
return $arg === null ? 'NULL' : (int) $arg;
This works fine for integers and floats, but unfortunately, it won't work for strings, because the string would still be quoted in the sql, resulting in the string 'NULL' instead of NULL. For it to work on strings, we would need to make the regex match the quote marks as well as the %s.
It seems like it might be better to automatically insert the quote marks around the %s instead of having the user type them, e.g. db_query("INSERT INTO {table} VALUES(%s)") instead of db_query("INSERT INTO {table} VALUES('%s')"). With this patch the old usage would be supported, but deprecated.
So we could expand the regex:
define('DB_QUERY_REGEXP', "/(%d|'%s'|%s|%%|%f|%b)/");
Then the string portion of _db_query_callback would read
case '%s':
case "'%s'":
$arg = array_shift($args);
return $arg === null ? 'NULL' : "'" . db_escape_string(array_shift($args)) . "'";
I have modified my database.inc as such and it generally works fine, although I have encountered one case where the old behavior was relied upon:
user warning: Column 'link' cannot be null query: watchdog INSERT INTO watchdog (uid, type, message, severity, link, location, referer, hostname, timestamp) VALUES (3, 'page not found', 'test', 1, NULL, 'http://mysite.com/asdf', NULL, 'x.x.x.x', 1193253537) in /var/www/drupal/includes/database.mysql.inc on line 172.
The replacement code works properly, but watchdog was relying on the old behavior, where one could pass a null and an empty string would be inserted instead. Probably there would be a number of small changes required. They're simple, but they would have to be tracked down.
What do you all think about using "%s" instead of "'%s'" in query strings? It seems cleaner to me to write VALUES(%d,%s,%d) rather than VALUES(%d,'%s',%d), but maybe there is a reason for the old way that I don't know about.
This is the first time I've tried to submit a patch, so please let me know what you think. Give me advice! I look forward to your responses.
P.S. I patched this against HEAD, as requested, but I only actually tested it against Drupal 5.3. It looked like none of the relevant code had changed, so I think it should work the same on both versions.
Nick
| Comment | File | Size | Author |
|---|---|---|---|
| #4 | insert_nulls3.patch | 1.75 KB | Nick Urban |
| #2 | insert_nulls2.patch | 1.79 KB | Nick Urban |
| insert_nulls.patch | 1.61 KB | Nick Urban |
Comments
Comment #1
Nick Urban commentedOops I think I should have posted this on 6.x-dev, since obviously this wouldn't get incorporated into version 5.
Comment #2
Nick Urban commentedI realized that changing the old code is pretty inconvenient, and would probably hamper getting such a patch accepted, so I created a new version that leaves quoted '%s' tokens (old-style) functioning the same and only inserts nulls for unquoted (new-style) %s tokens. Hopefully that will take care of any backward-compatibility problems, unless someone is inserting null when they really want 0, which seems less likely.
Comment #3
Nick Urban commentedI think this is ready for review now.
Comment #4
Nick Urban commentedI discovered that %s substitutions are used for more than just inserting text. They're used for generating SQL code too, and thus we can't assume that there are supposed to be quotes around them.
The best solution seems to be avoid worrying about %s legacy issues at all, and instead to just add a new token type. I'm using %t for 'text'. This latest and hopefully last patch just adds a %t type that gets automatically quoted and that inserts the next 'NULL' for null values.
Comment #5
Nick Urban commentedOkay, it turns out that some of the code actually does insert nulls and want zeros.
My initial assumptions about how these were used seem to be all wrong :-/
Since nobody has taken interest in this functionality, I assume there would be little chance of other code being modified to insert 0 when it means 0 and null when it means null.
Maybe a better approach would be to create a new function that wraps/replaces db_query so that all old code will continue to function the same.
Does anybody have an opinion about this, either way?
Comment #6
Berto commentedI won't comment on the code because I'm new to Drupal, I just wanted to add that I'd love to see this functionality.
Right now the best workaround I've found is at
http://drupal.org/node/130200
I find it sort of ridiculous that you can't insert NULL using db_query without writing messy code.
If you use the workaround above, don't forget to db_escape_string('stuff that may or may not be NULL')
Comment #7
demeritcowboy commentedRan into this today (version 5 but seems to be identical). Another workaround is actually to use '%s' instead of %d. At least in MySQL it works because the database doesn't really care about the difference between
INSERT INTO mytable (someNumericField) VALUES (1)and
INSERT INTO mytable (someNumericField) VALUES ('1')So you can do something like
db_query("INSERT INTO mytable (someNumericField) VALUES ('%s')", $n ? $n : "NULL");Comment #8
alan d. commentedAny progress on this issue? With schema's, this problem is continued out into the function
drupal_write_recordthat ignores NULL values as if they were unset.If there is progress on this issue, can there also be checks added for the schema definitions that allow NULL values to be included in the update/insert statements.
The casting of NULL values can be traced back to at least version 4.7. Since NULL may mean do not update this field as it has not been set, maybe a safer option would be to clone the current set of options to include a NULL allowed option for each of the types; %dn, %fn, %sn and %bn.
Comment #9
Nick Urban commentedThe solution I finally went with was to use %s (without quotes, with quotes it will think you mean the string "null", which won't work) instead of %d of %f. I would then write queries like this:
db_query("INSERT INTO {my_table} (possiblyNullNum) VALUES (%s)", $val ? $val : 'null');
It's not pretty, but it works.
One question though: are there problems with using a method that doesn't ensure that the value is actually a number, as %d and %f do?
Comment #10
damien tournoud commentedNote that for
drupal_write_record(), there is a fix by yched waiting to get in (#227677).Comment #11
Nick Urban commentedI've never used drupal_write_record. Is that the preferred way to save records in Drupal 6?
Comment #12
alan d. commentedIf #227677: drupal_write_record can't update a column to NULL gets in, and you have NULL's, then yes!
As of D7, there is a PDO wrapper, so this issue can be closed.
Comment #13
ohnobinki commented+