Central Cache API

So thinking out this it probably makes sense (in the loosest definition) to provide some kind of central way for other modules to say:

Hey I've got some data that you should store a 'best before date' of.

The module can tell me when the data is not fresh anymore, and the caching plugin can provide a easy way for users to select from a number of different things that could invalidate the data that they are using to construct their view from, the external modules could implement hooks that return forms for setting options on the 'types' of data they expose.

For example, the 'node' module could expose a hook that defines that users can look for changes on nodes. It would supply an option form, allowing users to restrict the types that they'd like to monitor.
Or the votingapi module could define a set of options for the different voting result tags that it supplies.

Seems like there are an awful lot of modules all trying to solve the same issues here. Notably, the expire module is trying to cover all the bases with a very advanced cache clearing system.

I think until that lands, we can add value by making it really easy to cover most bases when it comes to caching.

This basically means, we'll keep caching on a per content type basis, and not offer any further granularity, but will allow the user to decide if they want to consider comments and things like voting api votes as things that should clear the cache.

Here's my current thinking:

There are two distinct sides to the caching coin, one is keeping track of what might change the contents of the view, and another is segmenting viewers so they only see content they are supposed to. The way views handles the expiry time stuff quite handily deals with the first case, and the way that the cache keys are built up goes some way to deal with the second. For the cache keys, I think a simple module_invoke_all would help a lot here, because modules like OG are going to need to segment the data based on what groups the user is in etc.

For the cache expiry, here's what I'm thinking:

On some event, say the posting of a comment, we ask module what keys they'd like to set for an event. The comment module (our hook implementations on its behalf) would set a 'comment' key.
Then, crucially we offer up these keys for altering in a drupal_alter, other modules, such as the node module, or OG can decide to extend the keys or modify them, so node module would duplicate the single 'comment' key for the node type to give 2 keys of comment-{node_type} plus the original global 'comment' key. OG can then come along and add its keys, depending on the group that the node was in, duplicating the existing keys but appending its own OG key. We then store these keys against the timestamp for the action in database.

Then, we allow modules to place bits of form on our plugin form, so they can let users choose what the caching on the view should look for.

When executing a view, we just look for the active options a user chose and concatenate them to give a string that matches somewhere in our DB table, we retrieve the expiry time, and away we go.

Example:

1. Comment is posted on a 'blog' post in the 'First group' and 'Second group' groups, with nid 11 and 12 respectively.
2. hook_comment calls Views content cache with a new event, like so:
   

   event($comment, 'comment', 'insert', array(array(array('module' => 'comment', 'key' => 'changed'))));
   

3. We allow other modules to supply keys for this action. None do.
4. We allow the other modules to alter the keys for the action.
   1. Node module duplicates the keys, to make the $keys array look like:
      

      array(
        array(
          array(
            'module' => 'comment',
            'key' => 'changed',
          ),
        ),
        array(
          array(
            'module' => 'comment',
            'key' => 'changed',
          ),
          array(
            'module' => 'node',
            'key' => 'blog',
          ),
        ),
      );
      

   2. Then OG gets its chance, giving:
      

      array(
        array(
          array(
            'module' => 'comment',
            'key' => 'changed',
          ),
        ),
        array(
          array(
            'module' => 'comment',
            'key' => 'changed',
          ),
          array(
            'module' => 'node',
            'key' => 'blog',
          ),
        ),
        array(
          array(
            'module' => 'comment',
            'key' => 'changed',
          ),
          array(
            'module' => 'og',
            'key' => '11',
          ),
        ),
        array(
          array(
            'module' => 'comment',
            'key' => 'changed',
          ),
          array(
            'module' => 'og',
            'key' => '12',
          ),
        ),
        array(
          array(
            'module' => 'comment',
            'key' => 'changed',
          ),
          array(
            'module' => 'node',
            'key' => 'blog',
          ),
          array(
            'module' => 'og',
            'key' => '11',
          ),
        ),
      array(
          array(
            'module' => 'comment',
            'key' => 'changed',
          ),
          array(
            'module' => 'node',
            'key' => 'blog',
          ),
          array(
            'module' => 'og',
            'key' => '12',
          ),
        ),
      );
      

5. We then sort and store the keys as:
   

   array(
     'comment:changed' => $timestamp,
     'comment:changed|node:blog' => $timestamp,
     'comment:changed|og:11' => $timestamp,
     'comment:changed|og:12' => $timestamp,
     'comment:changed|node:blog' => $timestamp,
     'comment:changed|node:blog|og:11' => $timestamp,
     'comment:changed|node:blog|og:12' => $timestamp,
   );
   

Now we can do lookups to find the last time any comment changed, the last time any comment changed on a blog post, or the last time a comment changed on a blog post in the 'Second group' really quite easily.

Sounds complex, but isn't actually too bad.

I don't think what we're talking about here is actually complicated in terms of code, so when I start implementing some basic testing and sanity checking would be very, very useful. Also, documentation is always a bigger task than you'd think, so help there would be very much appreciated.

This sort of functionality (replacing tokens in cached output) is actually in views already, though I've not seen any working examples.

@yhahn, I knew that there should have been a better way of storing all that duplicate data, and making the DB do the heavy lifting in this makes me happy. Seems like a good refinement of the process to me.

This is now being worked on on github, I will copy it over to the 2.x branch in d.o CVS.

http://github.com/yhahn/views_content_cache

This is now in CVS.