MongoDB field value data types [#1658436]

Fields stored in MongoDB can have data types other than string. This is not the case with Drupal's default MySQL field storage; All primitive datatypes are handled by Drupal as string representations. This has caused us considerable trouble while developing an application that relies on custom entities, as we need to keep casting everything returned by Mongo to strings to make comparisons work reliably.

This can be easily demonstrated:
- Install two instances of Drupal
- Enable mongodb_field_storage on one of them
- Add an integer-type field to the Article content type on both
- Create a test article on both, making sure to add a value in the test integer field
- Create a test callback that loads the test article, serializes the node object, and prints it out

See the attachments, and look for "field_my_integer". The MySQL version has a string value in the integer field, just like all other fields that Drupal loads. The MongoDB version, however, has an integer value in the field.

I believe the MongoDB modules should handle all values as strings (cast to the real field type when sending data to Mongo, and casting to string when getting data out of Mongo), to stay consistent with the way Drupal normally works.

Comment	File	Size	Author
	integer_field_mysql.txt	2.08 KB	firebird
	integer_field_mongo.txt	2.08 KB	firebird

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Comment #1

chx CreditAttribution: chx commented 27 June 2012 at 06:58

Storing everything as strings... perhaps. This would break every mongodb install out there. How do you think we should proceed?

Comment #2

firebird CreditAttribution: firebird commented 27 June 2012 at 07:51

I was actually thinking nothing would change in the MongoDB database itself.

The Drupal MongoDB modules would do the type casting so that [--strike that--MongoDB always gets the data types it expects and has been getting so far, and ---] Drupal would always see strings.

(Actually I've thought about this a bit more, and it should be enough if the MongoDB modules just cast everything to strings before giving it to Drupal. The core MySQL storage has numeric fields in the DB, but field values come back as strings. It should be enough if the MongoDB modules did the same.)

From MongoDB's point of view nothing would change, and Drupal shouldn't care about the data types anyway. It doesn't now that there are integers and strings happily mixed, so it shouldn't care that they all start looking like strings. That's what Drupal core does anyway.

I don't think that would break anything. It would also be easy to fix, since we could make changes gradually by adding a type cast here and another one there whenever a "broken" place is found.

Comment #3

fgm

French

Paris, France

CreditAttribution: fgm commented 27 June 2012 at 07:52

I suspect it could break some logic like $gt selectors and make interoperability with any other mongodb-related code more difficult. If you really want to compare with MySQL, although Drupal mostly uses a "strings everywhere" approach, in the end many data are actually stored in the DB differently, notably as numbers.

I don't think we should focus on that least common denominator in the storage controller, whether that be the current entity+fields or the maybe-future unified storage, but on the contrary use whatever the implementation supports, since once the data has been fetched, it can still be used as strings within PHP if needed, but the opposite is not true.

Comment #4

chx CreditAttribution: chx commented 27 June 2012 at 16:58

OK, I get it you don't want to change the storage format, you want us to cast stuff on reading? That'd have a little performance penalty, I guess. What about I provide a variable / $conf override for you?

Comment #5

firebird CreditAttribution: firebird commented 28 June 2012 at 08:34

A configuration option for casting everything to strings on reads would be great.

Comment #6

chx CreditAttribution: chx commented 29 June 2012 at 01:42

I filed https://jira.mongodb.org/browse/PHP-404 but I will write the userspace recursion, sure.

Comment #7

majorrobot CreditAttribution: majorrobot commented 11 September 2015 at 21:58

Category:	Bug report	» Feature request
Issue summary:	View changes

Looks like chx filed a feature request for the PHP-Mongo driver, and the request was turned down. That would leave all the work on the field storage module.

An argument could be made that if you're creating custom entities, you may have to use a little extra code to get only strings from Mongo. Since this request is 3 years old, I'm tempted to set this to "won't fix," but would be curious to hear if this feature were still wanted by anyone?