Backport tests from: Cache clear fails when using wildcards - strange behavior with wildcard_valid()

I didn't have the time for actual bug-hunting yet, but I've created a new test class to use the menu to verify the bug.
How to use the new test:

• Apply patch.
• Make sure you've enabled memcache in your settings.php
• Run the new memcache test: Real world cache tests
• Test fails
• Disable memcache in your settings.php
• Run the new memcache test: Real world cache tests
• Test passes

Added summary according to the Issue summary standards.
Modified title to be more general.

Bumping to critical, this should block a 1.0 release.

I can't believe how long I now debugged to fix this single line...
Root cause of the whole issue was that MemCacheDrupal::wildcards() didn't use dmemcache_key() on this particular line but for lookup dmemcache_get_multi() is used which internally uses dmemcache_key().
That way the whole counting became inconsistent.

I've tried to another approach to create tests to cover that but I wasn't able to figure out how to really test that, nevertheless I've attached a patch with what I tried.
The tests I've added in #1 pass now.

Oh of course this needs review now :)

Before I forget it: Another change I thought of is to replace the increment by a set.
Currently:

        $this->memcache->increment(dmemcache_key($key, $this->bin));
         $wildcards[$this->bin][$key]++;

Idea:

        $wildcards[$this->bin][$key]++;
        $this->memcache->set(dmemcache_key($key, $this->bin), $wildcards[$this->bin][$key]);

Advantage of this would be that even if initially something went wrong the value is later properly set. Because if you use increment it wont create a item if it doesn't already exist.

Changed summary to fit the new summary initiative standards

Using set() instead of increment() sounds interesting, especially because we could then use the dmemcache_set() helper function, so that the encoding of the $key is again consistently encapsulated by dmemcache_*() functions.

Will look into the tests if I find the time.

Ok, you were actually quite close with your tests, all I had to do was adding a second round of set/get/clear/get because the initial wildcard is written using dmemcache_set() which correctly encoded $key.

The attached patch now consistently uses dmemcache_set() to write the wildcard. This might be bit slower but is way more failure tolerant (e.g. memcache restarts) and consistent.

The attached patch passed all tests for me locally.

Awesome! Thanks for your effort.
dmemcache_set() sounds very good to me. Code will be more consistent and maybe somewhat self-healing within the same request :D

This all makes sense, I've committed/pushed to 7.x-1.x.

We should backport the tests, and possibly the set/get change to 6.x (the increment usage is correct there though since it uses dmemcache_key(), so dropping priority for the backport).

Re-titling.

I am still encountering the same issue, both with memcache-7.x-1.0 as 7.x-1.x-dev (2012-Jan-30) releases. I have the following configuration in settings.php (I replaced the IP with ***):

$conf['memcache_servers'] = array('***:11211' => 'default');
$conf['cache_backends'] = array('sites/all/modules/memcache/memcache.inc');
$conf['cache_default_class'] = 'MemCacheDrupal';
$conf['cache_class_cache_form'] = 'DrupalDatabaseCache';

When I add the following modifying input formats works again like it should:

$conf['cache_class_cache'] = 'DrupalDatabaseCache';

But of course we also want to use Memcache for the general cache bin.

@Cyberwolf, 'same issue' isn't enough information, and the fact we added simpletests here suggests you're most likely hitting a different bug. I'm going to move this back to 6.x for the test backport, however please open a new bug, against 7.x, with clear steps to reproduce (ideally starting with a clean install of Drupal 7 + memcache 7.x-1.0 and nothing else), thanks!

I will specify more clearly what I mean with 'the same issue' then: I am unable to modify input formats with memcache-7.x-1.0 or 7.x-1.x-dev (2012-Jan-30) releases with the following configuration in settings.php:

$conf['memcache_servers'] = array('***:11211' => 'default');
$conf['cache_backends'] = array('sites/all/modules/memcache/memcache.inc');
$conf['cache_default_class'] = 'MemCacheDrupal';
$conf['cache_class_cache_form'] = 'DrupalDatabaseCache';

It's exactly the same behavior as described in the original report by weri.

Will retry on a completely clean install and let you know if it works now or not.

Also make sure that you're not using a broken memcache extension or server, see http://drupal.org/node/1131470

That probably explains everything, we are indeed using PECL memcache 3.0.4 which apparently has a bug with wildcard flushes. I'll ask our sysadmins to update to a newer version. Thanks @Berdir for pointing me in the right direction!

The memcache extension was upgraded and now it works correctly here with the Drupal memcache module 7.x-1.x-dev.

Updated issue summary.