Non-numeric object ids and revision ids [#368082]

One of the motivations behind the Field API is to support attaching fields to objects that do not themselves exist in the Drupal database (e.g. comments on any flickr photo). For performance, the id and revision id columns in field_data tables (as provided by the default field storage engine) are ints. However, not all remote objects to which we will want to attach fields have numeric ids.

My suggestion is to enhance field_attach_extract_ids():

* We add a boolean value to the info returned from hook_fieldable_info(): 'numeric fields'.
* If 'numeric fields' is TRUE, extract_ids() behaves as it does now.
* If 'numeric fields' is FALSE, extract_ids() translates the string ids to numeric ids using a lookaside table with serial id and revision id keys, and returns the numeric ids.
* We then write a different function, perhaps field_attach_extract_raw_ids(), that returns the untranslated values in case they are needed.

This enables all the rest of the field attach and field storage code to assume numeric ids while also letting manually constructed queries use the raw string ids in a normalized fashion.

Comments

Comment #1

bjaspan CreditAttribution: bjaspan commented 2 February 2009 at 18:47

The id translation lookaside tables are not field_config metadata, and they are not field_data data-storage tables (whose namespace is owned by the separate field storage module anyway). They should be in their own space, perhaps field_id_[objectytpe]: field_id_flickr, field_id_asin, etc. field_schema() would know which field id tables to created based on the Field Info API.

Comment #2

yched CreditAttribution: yched commented 5 February 2009 at 10:21

The 'native id to local numeric id' query, if it lives in field_attach_extract_ids(), should probably be cached because field_attach_extract_ids() can get called quite a lot in a page workflow.

Comment #3

bjaspan CreditAttribution: bjaspan commented 5 February 2009 at 12:51

Cached in a static variable or in a Drupal variable? I assume you mean the former; there would be too many to cache in a db variable or serialized cache array.

Your comment also makes me realize that we should provide a load-multiple ability to retrieve a group of local numeric ids in a single query, priming the extract_ids() static cache along the way.

Comment #4

yched CreditAttribution: yched commented 5 February 2009 at 15:36

sure, I was thinking of a static cache. Although, always the same problem : Are there cases where this static cache is going to bite us with stale data ?

Mainly thinking out loud :
we don't really need to bother with non numeric ids until we actually try to load / write field data, do we ?
One way to see this could be : The object has an id, it just happens that this id is not numeric, and that for performance reasons the field_sql_storage and PBS modules use numeric ids. That's strictly their problem, field api doesn't care or even have to know that.

We could state that the interface for being a 'field storage' module states "you will be asked to load and write field values on objects that can have non numeric ids. Deal with it.". Then the field_id_[objtype] tables you mention in #1 are managed by field_sql_storage.module (or whatever the 'primary sql storage module' ends up being), and they would be queried just-in-time, before the main SELECT or INSERT/UPDATE queries, to get the actual numeric id. For SELECT, maybe this could even be added as a JOIN in the main query instead of a separate preliminary query.

AFAICT that's enough to deal with local field attached to remote objects. Other than that, if local, numeric ids for the object need to be available also for non-field related stuff, then it's out of our scope and it's the job of the module that exposes those objects as fieldable to provide and manage those local ids.

Comment #5

yched CreditAttribution: yched commented 5 February 2009 at 15:44

forgot to state this clearly : the idea behind the above is that I think field_attach_extract_ids() is not the right place to handle the translation of non-numeric ids. It's a helper, quick function that just does quick and handy array manipulations to save the caller's carpal syndrome, and that is called many times during a page request; It shouldn't involve 'costly' operations like a db query, esp when most of the time the caller won't actually need the info fetched from the db.