Memcache API and Integration
Please upgrade to the newest PECL package (2.2.1) (or a higher 2.2.x version) to avoid a bug with deletes not happening. NB: Unlike previous versions which had no dependencies, 2.2.1 requires PHP 4.3.11 or newer. It is possible to get later versions running on older PHP installs, but its not recommended. Also note that the 3.x versions of the PECL package have not been tested with this module and are reported not to work.
This project consists of several parts:
- An API for using Memcached and the PECL Memcache library with Drupal.
- "Drop-in" replacement libraries for session handling. One that uses memcache exclusively (session-memcache.inc) and one that uses memcache if available and fails over to the database if memcache becomes unavailable.(session-memcache-db.inc)
- "Drop-in" replacement library for caching (works with cache_get and cache_set).
- A module that provides administrative overview of Drupal's interaction with Memcached and logic to invalidate "dirty" cache items.
- A set of tests that can be run to test your memcache setup.
This project is co-maintained by Robert Douglass, Nicholas Thompson, Simon Roberts, Steve Rude, and John VanDyk.
Installation
These are the broad steps you need to take in order to use this software. Order is important.
- Install the memcached binaries on your server. See How to install Memcache on Debian Etch or How to install Memcache on OSX
- Install the PECL memcache extension for PHP.
- In php.ini set memcache.hash_strategy="consistent".
- Put your site into offline mode.
- Download and install the memcache module.
- If you have previously been running the memcache module, run update.php.
- Apply the DRUPAL-5-cache-serialize.patch that comes with the module to your Drupal installation.
- Start at least one instance of memcached on your server.
- Edit settings.php to configure the servers, clusters and bins that memcache is supposed to use.
- Edit settings.php to include either memcache.inc or memcache.db.inc. For example, $conf['cache_inc'] ='sites/all/modules/memcache/memcache.db.inc';
- Bring your site back online.
For instructions on 1 and 2 above, please see the INSTALLATION.txt file that comes with the memcache module download.
Either the memcache.inc or the memcache.db.inc file is intended to be used instead of cache.inc, utilizing Drupal's pluggable cache system. The .db.inc variant saves all data to the database as well, so the site will still have the performance benefits of cache even if you take your memcache offline. The site should not ever break due to memcache not being available... it is only a question of whether caching is still available or not. The memcache.inc file doesn't save any data to the database and thus has the biggest potential for increasing your site's performance. If you use this file it is important to have enough memory allocated to memcache to store everything (including the page cache), otherwise the cache misses will negate the benefit of the cache hits.
Update $conf in settings.php to tell Drupal which cache_inc file to use:
<?php
$conf = array(
// The path to wherever memcache.inc is. The easiest is to simply point it
// to the copy in your module's directory.
'cache_inc' => './sites/all/modules/memcache/memcache.inc',
// or
// 'cache_inc' => './sites/all/modules/memcache/memcache.db.inc',
);
?>Servers
If you want the simple version, you can start one default memcache instance on your web server like this: memcached -m 24 -p 11211 -d If that is enough to meet your needs, there is no more configuration needed. If you want to utilize this module's sophisticated clustering feature and spread your cache over several machines, or if your cache is found on a machine other than your web server, read on.
The available memcached servers are specified in $conf in settings.php. If you do not specify any servers, memcache.inc assumes that you have a memcached instance running on localhost:11211. If this is true, and it is the only memcached instance you wish to use, no further configuration is required.
If you have more than one memcached instance running, you need to add two arrays to $conf; memcache_servers and memcache_bins. The arrays follow this pattern:
'memcache_servers' => array(host1:port => cluster, host2:port => cluster, hostN:port => cluster)
'memcache_bins' => array(bin1 => cluster, bin2 => cluster, binN => cluster)The bin/cluster/server model can be described as follows:
- Servers are memcached instances identified by host:port.
- Bins are groups of data that get cached together and map 1:1 to the $table param in cache_set(). Examples from Drupal core are cache_filter, cache_menu. The default is 'cache'.
- Clusters are groups of servers that act as a memory pool.
- Many bins can be assigned to a cluster.
- The default cluster is 'default'.
Here is a simple setup that has two memcached instances, both running on localhost. The 11212 instance belongs to the 'pages' cluster and the table cache_page is mapped to the 'pages' cluster. Thus everything that gets cached, with the exception of the page cache (cache_page), will be put into 'default', or the 11211 instance. The page cache will be in 11212.
<?php
$conf = array(
'memcache_servers' => array('localhost:11211' => 'default',
'localhost:11212' => 'pages'),
'memcache_bins' => array('cache_page' => 'pages'),
);
?>Here is an example configuration that has two clusters, 'default' and 'cluster2'. Five memcached instances are divided up between the two clusters. 'cache_filter' and 'cache_menu' bins goe to 'cluster2'. All other bins go to 'default'.
<?php
$conf = array(
'cache_inc' => './sites/all/modules/memcache/memcache.inc',
'memcache_servers' => array('10.1.1.1:11211' => 'default',
'10.1.1.1:11212' => 'default',
'10.1.1.2:11211' => 'default',
'10.1.1.3:11211' => 'cluster2',
'10.1.1.4:11211' => 'cluster2'),
'memcache_bins' => array('cache' => 'default',
'cache_filter' => 'cluster2',
'cache_menu' => 'cluster2'),
);
?>Prefixing
If you want to have multiple Drupal installations share memcached instances, you need to include a unique prefix for each Drupal installation in the $conf array of settings.php:
<?php
$conf = array(
'memcache_key_prefix' => 'something_unique',
);
?>Patches
The memcache/patches/DRUPAL-5-x-cache-serialize.patch must be applied to your Drupal installation for this software to work (pick the one that matches your Drupal version). The patch depends on a column that needs to get added to all of the existing cache tables for your site. This column has been introduced in the DRUPAL-6 development branch so this patch is future-safe if you ever upgrade to DRUPAL-6. To actually add the column to your database, you need to either install the memcache.module, or, if it is already installed and you are updating to this version, run update.php. Either installing the module or running update.php will add the needed column, Uninstalling the module will remove the column.
If there are not patches for all of the contributed modules you are running, you need to manually patch them. This is basically a process of finding all cache_get and cache_set instances and removing the serialize and unserialize calls involved with them. See existing patches for examples.
In addition, if you have modules that implement their own cache tables using the cache API, they need to also implement following hook:
<?php
function hook_devel_caches() {
return array('cache_table_1', 'cache_table_2', 'cache_table_n');
}
?>Troubleshooting
| Problem | Solution |
|---|---|
| Warning: require_once(a) [function.require-once]: failed to open stream: No such file or directory in /includes/bootstrap.inc on line 853 | This error occurs after you apply the DRUPAL-5-x-cache-serialize.patch because the code in the patch now expects the cached variables to be unserialized but they are still serialized in the cache. Clear the cache table: mysql> TRUNCATE cache; |
| Fatal error: Cannot use string offset as an array in includes/menu.inc on line 452 | Similar to the error above, this occurs after applying the DRUPAL-5-cache-serialize.patch due to the conflict between the existing cached menu and what the patched code is expecting. Clear cache_menu: mysql> TRUNCATE cache_menu; |
| Error: Failed to set key: Failed to set key: cache_page-...... | Upgrade your PECL library to PECL package (2.2.1) (or higher). |
| Warning: Zlib compression at the php.ini level conflicts with Memcache. | See http://drupal.org/node/273824 |
Memcache Admin
A module offering a UI for memcache is included and provides stats. In the future it might provide a way to clear the cache, and an interface to organize servers/bins/clusters.
