Use cache_bootstrap for entity and field info

Does this apply to _field_info_collate_fields() and _field_info_collate_types() as well ?

and _field_info_collate_fields() ? ;-)

But where do we draw the line? If it is called on almost every request, it might harm the requests it is not called on?

The name cache_boostrap suggests that it should be called as part of the bootstrap process?

Dries, it seems you committed the field.info.inc part of this along with http://drupal.org/cvs?commit=288558

Er, which means HEAD is broken because the patch had

+    cache_clear_all('field_info_fields', 'cache_field_bootstrap');

(instead of 'cache_bootstrap'), and cache_field_bootstrap table doesn't exist.

Rolled back. Thanks!

Thks :-). back to 'normal', then.

I'll add that it only makes sense to cache field meta data (set of available [field|widget|formatter] types, set of actual field and instance definitions) in a different table than the one used for field values cache.

There is indeed a balance to be made. The typical scenario is that you will have N webfrontends which will put cache_bootstrap in shared memory (apc/xcache variables, System V shm, black magic, whatever). Now, shared memory suffers from some problems: size as one segment is, by default is 32MB (on Linux and FreeBSD). Of course, you might have more than one segment but for some reason apc.shm_segments defaults to 1 so I have a bad suspicion that there are some (serious) disadvantage to several segments. As a working hypothesis, let's accept that 32MB is what we can work with. I will ask about the disadvantages of raising apc.shm_segments.

In that 32MB you need to stuff your op cache and cache_bootstrap. In other words, given the code size of Drupal, you better think twice when stuffing a complicated structure in cache_bootstrap. It's a great idea and very very useful but be careful. Menu was already raised and I would object that severely, for example. How big is this entity info your try to stuff in?

Note that APC supports an MMAP mode where apc.shm_segments are disregarded and the shmmax is not in play at all so at least with APC the size is not a problem . However, explaining someone that in order to use cache_bootstrap efficiently, she might need to recompile APC with --enable-apc-mmap and change configuration to be able to raise the memory accessible to APC, that's fun.

Then that's good. It was worth a few followups to show we thought of this.

Field types info amounts to 13k with core field types and widgets. This should reasonably reach at most 50k-100k with contrib field types.
Field definitions size growth is a little harder to predict. On my setup (9 fields, 15 instances), this amounts to 19k. At this rate, 100 fields and 200 instances would roughly raise to something like 250k.

+1 for me!

Attached is a new patch which applies. Alot of the previous touched files do not need any patch anymore because a function has been added to common.inc:6374 to clear the cache.

/**
 * Resets the cached information about entity types.
 */
function entity_info_cache_clear() {
  drupal_static_reset('entity_get_info');
  cache_clear_all('entity_info', 'cache_bootstrap');
}

rdf.test still has

cache_clear_all('entity_info', 'cache');

Other than that, this can bet set back to RTBC, as per #21.

Hmmz that got in with a recent commit I guess... or just missed it. New patch attached.

raymond@raymond-laptop:/srv/drupal/drupal$ grep -r "'entity_info'" *
includes/common.inc:    $drupal_static_fast['entity_info'] = &drupal_static(__FUNCTION__);
includes/common.inc:  $entity_info = &$drupal_static_fast['entity_info'];
includes/common.inc:    if ($cache = cache_get('entity_info', 'cache_bootstrap')) {
includes/common.inc:      $entity_info = module_invoke_all('entity_info');
includes/common.inc:      drupal_alter('entity_info', $entity_info);
includes/common.inc:      cache_set('entity_info', $entity_info, 'cache_bootstrap');
includes/common.inc:  cache_clear_all('entity_info', 'cache_bootstrap');

raymond@raymond-laptop:/srv/drupal/drupal$ grep -r "'field_info_fields'" *
modules/field/field.info.inc:    cache_clear_all('field_info_fields', 'cache_bootstrap');
modules/field/field.info.inc:    if ($cached = cache_get('field_info_fields', 'cache_bootstrap')) {
modules/field/field.info.inc:      cache_set('field_info_fields', $definitions, 'cache_bootstrap');

Thx !

The change catch mentioned can be found in http://drupal.org/node/503550. I still would like to have a look into this. cache_field is not likely to be cached in APC on every webnode. So it could still be a good change to move 'field_info_types' to a different bin then 'cache_field'.