Make simpletest concurrency work for PostgreSQL

So to some extent, running simpletests in concurrency on the commandline with PostgreSQL is possible - it does do it. And its quite important to do so because it speeds up a very slow PostgreSQL test suite. But Every now and then this happens:

Error message
PDOException: SQLSTATE[23505]: Unique violation: 7 ERROR:  duplicate key value violates unique constraint "cache_bootstrap_pkey": INSERT INTO cache_bootstrap (serialized, created, expire, headers, data, cid) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5); Array
(
)
in cache_set() (line 140 of /home/josh/projects/drupal7/includes/cache.inc).

It would seem a race condition is generating a unique constraint error where two processes are trying to set the bootstrap cache. This sounds like a job for the lock API!

One possibility would be to fix each cache_set call to either lock or catch a database exception like so:

diff --git a/includes/bootstrap.inc b/includes/bootstrap.inc
index aee7d58..440f1d5 100644
--- a/includes/bootstrap.inc
+++ b/includes/bootstrap.inc
@@ -722,15 +722,27 @@ function drupal_get_filename($type, $name, $filename = NULL) {
  * file.
  */
 function variable_initialize($conf = array()) {
+
   // NOTE: caching the variables improves performance by 20% when serving cached pages.
-  if ($cached = cache_get('variables', 'cache_bootstrap')) {
-    $variables = $cached->data;
+  if (!$cached = cache_get('variables', 'cache_bootstrap')) {
+    // If we can't retrive the cache, then we should try to acquire a lock to
+    // set the cache because another process maybe already building it.
+    // If we can't acquire lock, we should wait for it to become free again
+    // then attempt to retrive the cache again.
+    if (lock_may_be_available('variables:cache_bootstrap') && lock_aquire('variables:cache_bootstrap')) {
+      $variables = array_map('unserialize', db_query('SELECT name, value FROM {variable}')->fetchAllKeyed());
+      cache_set('variables', $variables, 'cache_bootstrap'); 
+      lock_release('variables:cache_bootstrap');
+    }
+    else {
+      lock_wait('variables:cache_bootstrap');
+      $cached = cache_get('variables', 'cache_bootstrap');
+    }
   }
-  else {
-    $variables = array_map('unserialize', db_query('SELECT name, value FROM {variable}')->fetchAllKeyed());
-    cache_set('variables', $variables, 'cache_bootstrap');
+  if ($cache) {
+    $variables = $cached->data;
   }
-
+  
   foreach ($conf as $name => $value) {
     $variables[$name] = $value;
   }
@@ -2106,7 +2118,17 @@ function _registry_check_code($type, $name = NULL) {
   // changes to the lookup cache for this request.
   if ($type == REGISTRY_WRITE_LOOKUP_CACHE) {
     if ($cache_update_needed) {
-      cache_set('lookup_cache', $lookup_cache, 'cache_bootstrap');
+      // It is possible that cache_set can fail in a race condition. However,
+      // since, at this point, the lookup cache has already been generated, 
+      // there is little point to aquiring a lock. We are better off to catch
+      // the failure if any and not complain about it. Another process took
+      // care of the cache before us.
+      try {
+        cache_set('lookup_cache', $lookup_cache, 'cache_bootstrap');
+      }
+      catch (Exception $e) {
+        watchdog('cache_set', 'Cache set failed likely because of race conditions', $e->getMessage());
+      }
     }
     return;
   }

Another option could be to add a lock table method to the DB layer that cache_set and other keyed based tables could then lock that table before inserting ensuring unique constraints aren't violated.

Still beyond me why MySQL doesn't see these issues? MyISAM testing maybe?