Let's create a variable caching API

This issue is an escalation of the original issue #254491: Standardize static caching, in which several people are simply trying to standardize the proliferation of pre-defined globals, and the others want to go all out and re-write static variables into an API that allows caching amongst any number of storage engines and caching methods. We'll leave the original issue to be solved in the more simplistic of manners, and instead start up another one to hash out the details of what's already been referred to as "Status: architecture (design needs more thought)". In the following paragraphs, I shall both condense and summarize what has been said in the previous thread. There's more than a few ways to approach the task at hand, we have nothing near a consensus, but there is a vein. Oh, and disclaimer; none of the following opinions are my own.

To summarize the need regarding a static caching API, consider this. Using static caching as a "performance enhancement" in any way that makes it possible for functions calling a public API to tell that static caching is enabled is a bug as severe as "the API function does not work at all." This does not apply to functions that intentionally build up internal state like drupal_set_header(), but observe the following:

* $node = node_load($nid)
* modify $node
* node_save($node)
* $new_node = node_load($nid)

Currently, the second node_load() returns the original $node. That's severely wrong; it should return the modified node.

How appropriate is this proposed static caching API? Dries said, "Because Drupal is so flexible, it's difficult to assess how feasible implementing a new caching API would be without breaking existing modules. I guess it requires some more thought and discussion. It might be nothing but a crazy idea." A lot of static variable caches should be cached longer, i.e. across multiple page requests (in order to increase performance). The caching in node_load() and node_save() could potentially be cached for an extended period of time. We don't necessarily want to cache this information in the database, but in presence of memcache.module, we might want to cache it a bit longer. In absence of memcached availablity, we could fall back to a static variable.

Or, there's cacherouter.module, which takes the approach of offering a public class for storing, retrieving, and flushing global variables from one or more storage engines, i.e., not necessarily the database tables which power Drupal itself. This would be a good starting place for an API that works with current Drupal methods because it's demonstrably functional in regards to cache_get() and cache_set().

Another benefit to implementing a centralized, standardized caching API? We can limit the amount of memory used by the static cache, if so desired. Cool. A major requirement we need to declare for Drupal coding standards is that using static caching as a "performance enhancement" in any way that makes it possible for any caller of any public API function to tell that static caching is enabled is a bug as severe as "the API function does not work at all." We need a centralized method of handling static variables.

So let's get busy.

First, there's the need to work out a naming convention. The var names have to be globally unique, which is a real PITA. We could append or prefix each with the function name, but why not just require that as a separate parameter to the method, which then gives us an extra dimension for the reset? Or, we could use 'origin_function__variable_name' so that it will become harder to create conflicting names, you must really be trying hard in order to break the namespace.

Next, caching should be a simple 'get data' and 'set data', which can be done with or without OOP. The backend for a cache_get/set/clear() would be specified at 'table' level, possibly with a special table name of MEMORY, or CACHE_TABLE_MEMORY. Different drivers could be included and users could specify in their settings file or in a cache info hook what table to use with what driver. Drivers that can be shipped with core could be: 'db', 'fsystem', 'static variable' and maybe even 'memcached' or 'apc'. Refer to cacherouter.module for implementation examples. Also, have a look at the innards of context.module. It provides a static cache api as well ... Look especially at context.module itself, and not at its helper modules.

What are the end-goal benefits? One general Cache API which can be tweaked for each website. People could use EARLY_PAGE_CACHE with fsystem caching for cache_pages, use database caching for views and use memcached caching with an expiration of 5 minutes for node_load. It all could lead to some nice core performance gains, yeah!