Alternative cache stampede protection

I've adapted this patch, modifying it so it protects all caches generically, not just the page cache. (Note: the attached patch includes this patch, as otherwise the page cache doesn't flush properly.)

I'll start with a picture illustrating the general goal. In this diagram, Object N is "dirty", which is to say it has either expired due to a ttl, or because cache_clear_all() was called (either explicitly for that object, or on the whole table). It takes time to rebuild an object -- on a busy website you may get tens of hundreds of requests all rebuilding the cached object at the same time -- a cache stampede. It can be further compounded by the fact that you may have to rebuilds hundreds of cached objects to rebuild a single page. With this patch, only one request rebuilds the cached objects -- the rest serve the old "stale" cached copy until the new copy is rebuilt:

  Timer  Action   
  0ms:
   |----> Object N request #1, obtain lock, rebuild object
  1ms:
   |----> Object N request #2, failed to obtain lock, serve old object from cache
   |                      
   |----> Object N request #n, failed to obtain lock, serve old object from cache
   |                      
  x ms:
   |      Object request #1 finished building new object, cache updated
   |----> Object request #n+1, cache is up-to-date, nothing to rebuild, serve up-to-date object from cache

In order to provide this functionality, memcache.inc tracks ttls, not memcache itself. Unfortunately this introduces an extra memcache get for every cache_get, thus increasing memcache traffic. To avoid problems with cache coherency, on a page requiring multiple objects, once a process has successfully obtained a lock for the first object, it assumes a lock for all subsequent dirty objects it needs to build a given page. Likewise, If it fails to get a lock on the first object, it assumes a failed lock on all the rest of the items too.

When using this patch, it is suggested that you set up a cache_lock bin that does not ever get flushed. Locks automatically expire after 5 seconds, to assure that an aborted session won't cause any problems.

Patch sponsored by NowPublic.