cache_set should not accept number of seconds as a valid expires-value

Jeremy - August 3, 2009 - 21:05
Project:Memcache API and Integration
Version:5.x-1.9
Component:Code
Category:bug report
Priority:normal
Assigned:Unassigned
Status:needs review
Description

It has been reported that upgrading a website to Memcache 6.x-1.3 breaks Mollom captcha's. I tracked this down to this commit.

In order to provide cache stampede protection, we no longer pass memcache an expire time. Instead, we set the expire time in the cache object, and then we check this when retrieving cached content with dmemcache_get(). This function assumes that $cache->expire will be a timestamp, but it can also be expressed as a number of seconds. The attach patch checks if $cache-expire is expressed in seconds and if so converts it to a timestamp so that the cache stampede protection works correctly.

Sponsored by ParentsClick / Lifetime Digital.

AttachmentSize
memcache.patch1.43 KB

#1

Jeremy - August 3, 2009 - 21:09

BTW: In most cases we don't notice this bug because if a cached item doesn't exist, we simply regenerate it. Mollom is using the cache a little differently, and if the cached item doesn't exist it can't validate the session_id and the Captcha validation fails.

#2

chx - August 3, 2009 - 22:31

Well, this looks good however storing any data in cache that can't be regenerated is a bad idea. We learned our lesson with update status and form cache really should not be a cache either. Still: this patch is good.

#3

Jeremy - August 4, 2009 - 00:40

This is the first time I've looked at the Mollom code, but I believe they're doing things right here. The content stored in cache_mollom can and is regenerated as necessary -- but it does require that the cached session can live in the cache long enough for someone to submit a comment, enter their login information, whatever. It was a memcache bug that was causing this content to be "flushed" instantly, breaking Mollom. When this happens, Mollom generates a new session and tries again, as it should using a cache_* table. With the patch attached above, things work as expected -- if the cache is flushed (ie memcache is restarted) Captcha validation will fail and the user will just think they misread the annoying cryptic letters that are Captchas and try again, then it will work...

#4

Jeremy - August 4, 2009 - 18:55

I believe this is also causing this bug:
#533524: High MySQL queries for 6.x-1.3 vs. 6.x-1.2

#5

Jeremy - August 4, 2009 - 19:04

The previous patch fixed items that had an expiration defined in seconds, but it left CACHE_TEMPORARY still broken. The attached patch fixes both.

AttachmentSize
memcache.patch 1.22 KB

#6

Jeremy - August 4, 2009 - 20:16
Status:active» reviewed & tested by the community

Without my patch I can duplicate the Mollom bug, and I can duplicate the page cache issue in that the page cache is rebuilt far too frequently. With the above patch, Mollom works and the page cache no longer is flushed artificially. I will commit this and roll a 6.x-1.4-rc1 to encourage further testing.

#7

Jeremy - August 4, 2009 - 20:21
Status:reviewed & tested by the community» fixed

Fix applied:
http://drupal.org/cvs?commit=246944

#8

Damien Tournoud - August 13, 2009 - 09:58
Status:fixed» active

Reopening.

cache_set() does not take a number of second as the $expire parameter, but only a timestamp. Let's not introduce hacks in memcache for my own programming errors.

#10

mikl - August 13, 2009 - 12:02
Title:Memcache 6.x-1.3 breaks Mollom, cache expiration broken when expressed in seconds» cache_set should not accept number of seconds as a valid expires-value
Version:6.x-1.3» 6.x-1.4

As Damien noted, this behavior is inconsistent with Drupal's own cache_set().

#548078: Mollom should call cache_set() with a timestamp, not a number of seconds has been committed to CVS, so I don't think there's any reason to keep this extra feature around :)

#11

Jeremy - August 13, 2009 - 13:08
Status:active» fixed

Memcache supports either method of specifying the cache lifetime. The current memcache module code is still 100% compatible with Drupal core, while also exposing an extra memcache feature. I don't see any problem with this.

(The only reason this looks like a "hack" is because we've added cache stampede protection where we are handling cache expiration rather than memcache... but that "hack" can be quite a scalability boost.)

#12

System Message - August 27, 2009 - 13:10
Status:fixed» closed

Automatically closed -- issue fixed for 2 weeks with no activity.

#13

yhager - September 23, 2009 - 11:32
Version:6.x-1.4» 5.x-1.9
Status:closed» needs review

Attached a backport of this patch to 5.x version, as this issue was introduced there too.

AttachmentSize
yh.patch 1.03 KB

#14

mikl - September 24, 2009 - 10:13

#11: It is not 100% compatible with Drupal core. It extends the API and makes it possible to make code that works correctly with your specific cache_set implementation, but not necessarily the one in Drupal core. Granted, it is a minor issue, but why support something that Drupal core doesn’t, if that only serves to make the resulting code non-portable?

#15

Jeremy - September 24, 2009 - 13:32

By that argument why do we have any contrib modules, as they clearly provide functionality that's not offered by Drupal core.

You can keep your code 100% compatible with Drupal core if that is your requirement. But you should also be able to tap into memcache features not available to core if that's what you need.

#16

mikl - October 18, 2009 - 19:52

#15: We're not talking about new functionality here. In this case, a documented part of Drupal's API changes behaviour when your module is installed. If other modules changed the behaviour of the common API methods we all used, the Drupal ecosystem would be total chaos.

 
 

Drupal is a registered trademark of Dries Buytaert.