There is a race condition under certain scenarios in the memcache-lock.inc file. It seems dmemcache_object() calls MemCacheDrupal->get() which calls lock_acquire(). You will get a fatal error as the lock has not yet been included.

Fatal error: Call to undefined function lock_acquire() in /home/vagrant/devel/drupal-7/sites/all/modules/contrib/memcache/memcache.inc on line 90

Call Stack:
0.0004 691960 1. {main}() /usr/share/pear/drush/drush.php:0
0.0184 4837200 2. drush_main() /usr/share/pear/drush/drush.php:16
0.0778 10776240 3. _drush_bootstrap_and_dispatch() /usr/share/pear/drush/drush.php:61
0.0974 10781576 4. drush_bootstrap_to_phase() /usr/share/pear/drush/drush.php:81
0.0974 10781576 5. drush_bootstrap_max() /usr/share/pear/drush/includes/bootstrap.inc:291
0.1449 14002840 6. drush_bootstrap() /usr/share/pear/drush/includes/bootstrap.inc:345
0.1453 14004000 7. _drush_bootstrap_drupal_full() /usr/share/pear/drush/includes/bootstrap.inc:185
0.1453 14045072 8. drupal_bootstrap() /usr/share/pear/drush/includes/bootstrap.inc:930
0.1453 14045032 9. _drupal_bootstrap_variables() /home/vagrant/devel/drupal-7/includes/bootstrap.inc:2194
0.1454 14050920 10. require_once('/home/vagrant/devel/drupal-7/sites/all/modules/contrib/memcache/memcache-lock.inc') /home/vagrant/devel/drupal-7/includes/bootstrap.inc:2413
0.1454 14051256 11. dmemcache_object() /home/vagrant/devel/drupal-7/sites/all/modules/contrib/memcache/memcache-lock.inc:16
0.1454 14051480 12. class_exists() /home/vagrant/devel/drupal-7/sites/all/modules/contrib/memcache/dmemcache.inc:292
0.1454 14051936 13. drupal_autoload_class() /home/vagrant/devel/drupal-7/sites/all/modules/contrib/memcache/dmemcache.inc:0
0.1454 14052016 14. _registry_check_code() /home/vagrant/devel/drupal-7/includes/bootstrap.inc:3026
0.1454 14052344 15. cache_get() /home/vagrant/devel/drupal-7/includes/bootstrap.inc:3053
0.1457 14058624 16. MemCacheDrupal->get() /home/vagrant/devel/drupal-7/includes/cache.inc:56

Comments

socialnicheguru’s picture

I am seeing this too. so using the lock through memcache is unworkable right now

JStarcher’s picture

Actually good to hear that I'm not the only one.

GIve this patch a try and let us know- it's working great for me.

JStarcher’s picture

Status: Active » Needs review
JStarcher’s picture

Version: 7.x-1.0 » 7.x-1.x-dev
socialnicheguru’s picture

moving the right direction.

But there is another error

PHP Fatal error: Call to undefined function lock_acquire() in modules/all/memcache/memcache.inc on line 96

The lines are

  // On cache misses, attempt to avoid stampedes when the
    // memcache_stampede_protection variable is enabled.
    if (!$cache) {
      if (variable_get('memcache_stampede_protection', FALSE) && !lock_acquire(\
"memcache_$cid:$this->bin", variable_get('memcache_stampede_semaphore', 15))) {
	
JStarcher’s picture

Looking at this again I see that really there needs to be a better way to check if memcache is working rather than calling dmecache_object() since dmemcache_object() creates a lock on misses. I also looked at simply trying to set a variable with dmemcache_set() and checking the output but that can ask for a lock as well.

jeremy’s picture

StatusFileSize
new2.19 KB

I was able to duplicate this by enabling page_cache_without_database and hitting a page that's not currently in the cache. The attached patch fixes for me, please test.

jeremy’s picture

Status: Needs review » Fixed

Actually, we need to load it properly through the bootstrap or our lock is going to get lost when we reinit during bootstrap. Fix committed:
http://drupalcode.org/project/memcache.git/commitdiff/1279c66

helmo’s picture

Status: Fixed » Needs review
StatusFileSize
new678 bytes

Sorry to re-open this, but a typo is introduced... $this->lockInit instead of $this->lockInit()

This patch fixes it.

markpavlitski’s picture

Status: Needs review » Reviewed & tested by the community

Applies cleanly and fixes the issue.

jeremy’s picture

Status: Reviewed & tested by the community » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

fabianx’s picture

Title: Race condition in locking framework » [Stampede protection broken] Race condition in locking framework
Priority: Normal » Critical
Status: Closed (fixed) » Needs work

This effectively breaks all sites using stampede protection. A BIG do not upgrade or disable stampede protection is in order:

The reason is:

if ($this->lockInit() && ...

is always FALSE, because lockInit() does not return TRUE;

The consequence is that any expired cache item WILL ALWAYS BE RETURNED, until memcache is restarted, which is effectively a never expiring cache.

There is no cache_clear_all() or anything that will fix that condition - ever, because everything after the expired item is in else blocks and never executed.

diff --git a/memcache.inc b/memcache.inc
--- a/memcache.inc
+++ b/memcache.inc
@@ -463,6 +463,9 @@ class MemCacheDrupal implements DrupalCacheInterface {
 
   /**
    * Helper function to load locking framework if not already loaded.
+   *
+   * @return bool
+   *   Whether the locking system was initialized successfully.
    */
   public function lockInit() {
     // On a cache miss when page_cache_without_database is enabled, we can end
@@ -471,6 +474,8 @@ class MemCacheDrupal implements DrupalCacheInterface {
     if (!function_exists('lock_acquire')) {
       drupal_bootstrap(DRUPAL_BOOTSTRAP_VARIABLES, FALSE);
     }
+
+    return TRUE;
   }
 
   /**

is the proper fix. This makes it possible to keep this->lockInit() within it if () call, which is important to only load it on demand for the second scenario where it is called.

plach’s picture

Status: Needs work » Needs review
StatusFileSize
new694 bytes

This implements #13.

plach’s picture

StatusFileSize
new755 bytes

Better PHP doc

fabianx’s picture

Status: Needs review » Reviewed & tested by the community

RTBC - works locally for me

  • catch committed 5b0cc4e on 7.x-1.x
    Issue #2099893 by plach, Fabianx: Fixed [Stampede protection broken]...
plach’s picture

Status: Reviewed & tested by the community » Fixed

I guess this is fixed :)

fabianx’s picture

Title: [Stampede protection broken] Race condition in locking framework » Race condition in locking framework
Priority: Critical » Normal

Back to normal

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.