memcache.inc does not handle the scenario where the memcached server is not reachable. (error handling is important)
So if memcached is unreachable, web page loads never complete. This will result in a huge backlog of requests which may exceed the queue limit of the web (or cache) server.
Solution is to fallback to database path, until it has been more than X seconds for the next memcache request, then try again.
Patch included to resolve this issue. I'm a sysadmin as my primary role so my solution may not be 100% "correct" php or drupal in style.
Another problem I confronted... I cannot get 'session' => "database" to work (login denied error messages)... I dont have memcache-session.inc enabled... but I do wonder how it works with memcached enabled but not when it is disabled (which does work if memcache.inc is not used at all)?!?!
Comment | File | Size | Author |
---|---|---|---|
#1 | memcache-database_fallback-1423862.patch | 4.71 KB | dsobon |
Comments
Comment #1
dsobon CreditAttribution: dsobon commentedComment #2
rjbrown99 CreditAttribution: rjbrown99 commentedPatch here, changing status.
Comment #3
scottatdrake CreditAttribution: scottatdrake commentedI believe I experienced this today with 7.x-1.0. Memcache went down, taking my sites with it. I'm surprised nobody else has chimed in here. Seems like a serious issue, no?
I'll look into rolling a D7 version of this patch.
Comment #4
ron_s CreditAttribution: ron_s commentedThis sounds very similar to what we've been encountering. We run a large-scale site with Memcache 6.x-1.10 and typically it behaves very well. However once every month or two, we see a situation where a random page is attempting to be loaded, and one of our load balanced servers stops responding. We receive the following error message:
Type: memcache
User: Anonymous
Message: Exception caught in dmemcache_object: Memcache::pconnect() [memcache.pconnect]: Can't connect to localhost:11211, Unknown error (0)
Severity: warning
Many requests back up in the queue and eventually the server no longer responds.
It seems like there also could be some overlap with the work done as part of this patch?: http://drupal.org/node/1184678
Comment #5
Jeremy CreditAttribution: Jeremy at Tag1 Consulting commentedThere is a memcache.db.inc that offers what you're trying to do here -- however, the functionality is deprecated and not recommended.
In 6.x-1.11-rc1 and beyond we now have much better failure tolerance -- if Memcached goes away, web pages will continue to load normally (though slower, as nothing will be cached).
See:
#1184678: Enhance failure tolerance