Last updated November 12, 2012. Created by peter bowey on June 8, 2011.
Edited by FanisTsiros, Phizes, HongPong, shamio. Log in to edit this page.
This information enhances the documentation for the Drupal Memcache module. Currently only the PECL Memcache extension can be used for sockets. PECL Memcached support is the subject of #1690130: Memcache fails to connect when using Unix sockets and PECL Memcached. The Drupal configuration on this page is for Drupal 6, while the other information below is Drupal version agnostic. For Drupal 7 sites on Ubuntu 12.04 you may want to see this post.
Quick install of the Memcache daemon for the modern versions of some common Linux Distributions:
# Debian / Ubuntu:
sudo apt-get install memcached
# Fedora / CentOS / RHEL
sudo yum install memcachedIf you have PEAR installed and up to date, you can install the PECL Memcache extension as follows:
# For the stable 2.2.x branch
sudo pecl install memcache
# For the beta, though functional, 3.0.x branch
sudo pecl install memcache-betaPECL Memcache may also be installed with your distributions package manager, but it is usually out of date.
Memcache - Guidelines: see items 1) to 3)
1) A typical Drupal / Pressflow 6.2x use of memcache
The relevant parts of settings.php from a Drupal/Pressflow site utilizing the Memcache module.
<?php
/**
* Memcache:
*/
$conf += array(
'memcache_extension' => 'memcache',
'show_memcache_statistics' => 0,
'memcache_persistent' => TRUE,
'memcache_stampede_protection' => TRUE,
'memcache_stampede_semaphore' => 15,
'memcache_stampede_wait_time' => 5,
'memcache_stampede_wait_limit' => 3,
'memcache_key_prefix' => basename(realpath(conf_path())),
);
// We don't use chained memcached caching for sites cron, php-cli and install!
if (isset($_SERVER['HTTP_USER_AGENT']) && !preg_match("/(?:cron\.php|install\.php)/", $_SERVER['REQUEST_URI'])) {
if (isset($_SERVER['HTTP_HOST'])) {
$conf += array(
'cache_inc' => './sites/all/modules/memcache/memcache.inc',
'session_inc' => './sites/all/modules/memcache/memcache-session.inc',
'lock_inc' => './sites/all/modules/memcache/memcache-lock.inc',
'memcache_servers' => array(
'unix:///tmp/memcached.socket0:0' => 'default',
'unix:///tmp/memcached.socket1:0' => 'block',
'unix:///tmp/memcached.socket2:0' => 'content',
'unix:///tmp/memcached.socket3:0' => 'filter',
'unix:///tmp/memcached.socket4:0' => 'form',
'unix:///tmp/memcached.socket5:0' => 'menu',
'unix:///tmp/memcached.socket6:0' => 'page',
'unix:///tmp/memcached.socket7:0' => 'update',
'unix:///tmp/memcached.socket8:0' => 'views',
'unix:///tmp/memcached.socket9:0' => 'session',
'unix:///tmp/memcached.socket10:0' => 'users'
),
'memcache_bins' => array(
'cache' => 'default',
'cache_block' => 'block',
'cache_content' => 'content',
'cache_filter' => 'filter',
'cache_form' => 'form',
'cache_menu' => 'menu',
'cache_page' => 'page',
'cache_update' => 'update',
'cache_views' => 'views',
'session' => 'session',
'users' => 'users'
),
);
}
}
?>2) Sample memcache php.ini settings:
This can either be placed into the relevant php.ini file or in a file named memcache.ini in the relevant conf.d directory.
[memcache]
; Data will be transferred in chunks of this size
memcache.chunk_size = 32768
memcache.hash_strategy = consistent
memcache.default_port = 0
session.save_handler = memcache
session.save_path = "unix:///tmp/memcached.socket11:0?persistent=1&weight=1&timeout=1&retry_interval=15"3) Sample Linux memcached daemon configuration
On some Linux distributions you can copy and rename /etc/memcache.conf to memcache_myserver.conf to create configurations for each Memcache instance which will be handled by the existing /etc/init.d/memcached script, further details on this are provided in the /etc/init.d/memcache script itself. Alternatively you can replace your /etc/init.d/memcached with the script below (Remember to backup your original init script first):
#! /bin/sh
#
# chkconfig: - 55 45
# description: The memcached-multi daemon is a network memory cache service.
# processname: memcached-multi
# config: /etc/sysconfig/memcached
# pidfile: /var/run/memcached/memcached.*.pid
# Standard LSB functions
#. /lib/lsb/init-functions
# Source function library.
. /etc/init.d/functions
PORT=11211
UDP=0
SOCKET=/tmp/memcached.socket
VAR=0
USER=memcached
MAXCONN=300
CACHESIZE=64
OPTIONS=""
if [ -f /etc/sysconfig/memcached ];then
. /etc/sysconfig/memcached
fi
# Check that networking is up.
. /etc/sysconfig/network
if [ "$NETWORKING" = "no" ]
then
exit 0
fi
RETVAL=0
prog="memcached"
start_instance() {
echo -n $"Starting $prog ($1): "
# daemon --pidfile /var/run/memcached/memcached.$1.pid memcached -d -p $PORT -u $USER -m $2 -c $MAXCONN -P /var/run/memcached/memcached.$1.pid $OPTIONS
daemon --pidfile /var/run/memcached/memcached.$1.pid memcached -d -s $3 -a 766 -L -t 8 -u $USER -m $2 -c $MAXCONN -P /var/run/memcached/memcached.$1.pid $OPTIONS
RETVAL=$?
echo
[ $RETVAL -eq 0 ] && touch /var/lock/subsys/memcached.$1
}
stop_instance() {
echo -n $"Stopping $prog ($1): "
killproc -p /var/run/memcached/memcached.$1.pid /usr/bin/memcached
RETVAL=$?
echo
if [ $RETVAL -eq 0 ] ; then
rm -f /var/lock/subsys/memcached.$1
rm -f /var/run/memcached.$1.pid
fi
}
start() {
# insure that /var/run/memcached has proper permissions
if [ "`stat -c %U /var/run/memcached`" != "$USER" ]; then
chown $USER /var/run/memcached
fi
# we start 12 socket streams for memcached
start_instance default 64 /tmp/memcached.socket0;
start_instance block 64 /tmp/memcached.socket1;
start_instance content 64 /tmp/memcached.socket2;
start_instance filter 64 /tmp/memcached.socket3;
start_instance form 64 /tmp/memcached.socket4;
start_instance menu 64 /tmp/memcached.socket5;
start_instance page 64 /tmp/memcached.socket6;
start_instance update 64 /tmp/memcached.socket7;
start_instance views 64 /tmp/memcached.socket8;
start_instance session 64 /tmp/memcached.socket9;
start_instance users 64 /tmp/memcached.socket10;
start_instance pbold 64 /tmp/memcached.socket11;
}
stop () {
stop_instance default;
stop_instance block;
stop_instance content;
stop_instance filter;
stop_instance form;
stop_instance menu;
stop_instance page;
stop_instance update;
stop_instance views;
stop_instance session;
stop_instance users;
stop_instance pbold;
}
restart () {
stop
start
}
# See how we were called.
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status memcached
;;
restart|reload|force-reload)
restart
;;
*)
echo $"Usage: $0 {start|stop|status|restart|reload|force-reload}"
exit 1
esac
exit $?Additional Notes:
The PHP Memcache PECL can also be compiled fully 'static' into the php-5.3.x binary-by moving the [memcache - PECL] source into the php /ext folder and rebuilding the PHP 'buildconf' - then compile PHP 5.3.x with --enable-memcache (it is *not* essential to use PECL if you are after the best speed - and understand *static* PHP compile builds).
PHP 5.3.x Static Compile guidelines:
In the PHP source root folder, issue the following (Linux);
rm configure
PHP_AUTOCONF=autoconf-2.13 PHP_AUTOHEADER=autoheader-2.13 ./buildconf --forceThen compile PHP manually (or use rpmbuild or similar). If you usually work directly with Fedora RPM sources you can build PHP with:
rpmbuild -bb php.spec >> php.checklist 2>&1
Comments
Will it work for Drupal 7
If i follow the same step , will it work for Druapl 7 ?
~Jayesh
Same question here. Looking
Same question here. Looking to use sockets
Memcached through sockets on Linux with Drupal 7.x
Hi,
I recently got memcached 1.4.5 running with Memcache v3.0.6 (pecl php module). Thought I'd share the setup. Note: it's got some details that need tweaking (see the comments here and there), but this setup works on Debian Squeeze (Linxu 2.6.32-5-openvz-amd64).
php.ini (or /etc/php5/conf.d/memcached.ini):
[memcache]; Data will be transferred in chunks of this size
memcache.chunk_size = 32768
memcache.hash_strategy = consistent
memcache.default_port = 0
session.save_handler = memcache
session.save_path = "unix:///tmp/memcached.socket11?persistent=1&weight=1&timeout=1&retry_interval=15"
Note: no port appended to the session.save_path here.
settings.php
<?php
/**
* Memcache:
*/
$conf += array(
'memcache_extension' => 'Memcache',
'show_memcache_statistics' => 0,
'memcache_persistent' => TRUE,
'memcache_stampede_protection' => TRUE,
'memcache_stampede_semaphore' => 15,
'memcache_stampede_wait_time' => 5,
'memcache_stampede_wait_limit' => 3,
'memcache_key_prefix' => basename(realpath(conf_path())),
);
// We don't use chained memcached caching for sites cron, php-cli and install!
// memcache-session.inc has not successfully been ported to D7 yet!
if (isset($_SERVER['HTTP_USER_AGENT']) && !preg_match("/(?:cron\.php|install\.php)/", $_SERVER['REQUEST_URI'])) {
if (isset($_SERVER['HTTP_HOST'])) {
$conf += array(
'cache_inc' => './sites/all/modules/memcache/memcache.inc',
// 'session_inc' => './sites/all/modules/memcache/memcache-session.inc',
'lock_inc' => './sites/all/modules/memcache/memcache-lock.inc',
'memcache_servers' => array(
'unix:///tmp/memcached.socket0' => 'default',
'unix:///tmp/memcached.socket1' => 'block',
'unix:///tmp/memcached.socket2' => 'content',
'unix:///tmp/memcached.socket3' => 'field',
'unix:///tmp/memcached.socket4' => 'filter',
'unix:///tmp/memcached.socket5' => 'form',
'unix:///tmp/memcached.socket6' => 'menu',
'unix:///tmp/memcached.socket7' => 'page',
'unix:///tmp/memcached.socket8' => 'update',
'unix:///tmp/memcached.socket9' => 'views',
'unix:///tmp/memcached.socket10' => 'session',
'unix:///tmp/memcached.socket11' => 'users',
),
'memcache_bins' => array(
'cache' => 'default',
'cache_block' => 'block',
'cache_content' => 'content',
'cache_field' => 'field',
'cache_filter' => 'filter',
'cache_form' => 'form',
'cache_menu' => 'menu',
'cache_page' => 'page',
'cache_update' => 'update',
'cache_views' => 'views',
'session' => 'session',
'users' => 'users'
),
);
}
}
include_once('./includes/cache.inc');
// FIXME! We have to literally include memcache.inc already at this point
// otherwise D7 won't start (why? probably because php session.save_handler is memcached, but not sure)
include_once('./sites/all/modules/memcache/memcache.inc');
$conf['cache_backends'][] = 'sites/all/modules/apc/drupal_apc_cache.inc';
$conf['cache_backends'][] = 'sites/all/modules/varnish/varnish.cache.inc';
$conf['cache_backends'][] = 'sites/all/modules/memcache/memcache.inc';
/*
* cache prefix is "cache_class", default appointment is set by "cache_default_class"
* See source cache.inc (<a href="http://api.drupal.org/api/views/includes%21cache.inc/7" title="http://api.drupal.org/api/views/includes%21cache.inc/7" rel="nofollow">http://api.drupal.org/api/views/includes%21cache.inc/7</a>)
*
* The cache bin to store the data in. Valid core values are 'cache_block',
* 'cache_bootstrap', 'cache_field', 'cache_filter', 'cache_form',
* 'cache_menu', 'cache_page', 'cache_update' or 'cache' for the default
* cache.
*
* So valid examples: "cache_class_cache_block", "cache_class_cache"
*/
$conf['cache_class_cache'] = 'DrupalAPCCache'; //stuff in the 'cache'-bin goes to APC
$conf['cache_class_cache_bootstrap'] = 'DrupalAPCCache'; //bootstrap stuff from APC
$conf['cache_class_cache_form'] = 'DrupalDatabaseCache'; //Don't store forms in any cache mechanism (why?)
$conf['cache_class_cache_page'] = 'VarnishCache'; //Cache full pages to Varnish
$conf['cache_default_class'] = 'MemCacheDrupal'; //Any other cache bin goes to memcached
?>
Note: as you can see the setup works with a combination of APC, Memcached and VarnishCache. The default cache class is MemCacheDrupal though. Also note that 'session_inc' (don't confuse with php session, this is the drupal Sessions handler) is commented out as it isn't ported to D7 yet (see: http://drupal.org/node/1410218).
Memcached startup script in /etc/init.d/
#! /bin/sh
#
# file: /etc/init.d/memcached
#
# chkconfig: - 55 45
# description: The memcached-multi daemon is a network memory cache service. This script starts
# memcached specifically for Drupal through Unix sockets (13 socket streams)
# processname: memcached-multi
# pidfile: /var/run/memcached/memcached.*.pid
#
# adapted from http://drupal.org/node/1181968
# by Coert van Gemeren (coert.vangemeren@dub.uu.nl) on 2012-03-30
PORT=11211
UDP=0
SOCKET=/tmp/memcached.socket
VAR=0
USER=nobody
MAXCONN=300
CACHESIZE=64
MEMCACHED=/usr/bin/memcached
RETVAL=0
prog="memcached"
start_instance() {
if [ ! -e /var/run/memcached/memcached.$1.pid ] ; then
echo "Starting $prog ($1)"
OPTIONS="-d -s $3 -a 766 -L -t 8 -u $USER -m $2 -c $MAXCONN -P /var/run/memcached/memcached.$1.pid"
start-stop-daemon --start --quiet --pidfile /var/run/memcached/memcached.$1.pid --make-pidfile --exec $MEMCACHED -- $OPTIONS
RETVAL=$?
[ $RETVAL -eq 0 ] && touch /var/lock/memcached/$1
else
echo "$prog ($1) already running: /var/run/memcached/memcached.$1.pid"
fi
}
stop_instance() {
if [ -e /var/run/memcached/memcached.$1.pid ] ; then
PID=$(ps -ef|grep "/var/run/memcached/memcached.$1.pid"|grep -v "grep"|awk '{print $2}')
# FIXME! Not working yet
# PID=$(cat /var/run/memcached/memcached.$1.pid)
echo "Stopping $prog ($1)"
# FIXME! Not working yet
# start-stop-daemon --stop --oknodo --pidfile /var/run/memcached/memcached.$1.pid --signal KILL --exec $MEMCACHED
kill -9 $PID
RETVAL=$?
if [ $RETVAL -eq 0 ] ; then
rm -f /var/lock/memcached/$1
rm -f /var/run/memcached/memcached.$1.pid
fi
else
echo "$prog ($1) not running: /var/run/memcached/memcached.$1.pid"
fi
}
start() {
# insure that /var/run/memcached has proper permissions
if [ "`stat -c %U /var/run/memcached`" != "$USER" ]; then
chown $USER /var/run/memcached
fi
# we start 13 socket streams for memcached
start_instance default 64 /tmp/memcached.socket0;
start_instance block 64 /tmp/memcached.socket1;
start_instance content 64 /tmp/memcached.socket2;
start_instance field 64 /tmp/memcached.socket3; #Field is new in D7, it is not yet supported in the memcache module reports page
start_instance filter 64 /tmp/memcached.socket4;
start_instance form 64 /tmp/memcached.socket5;
start_instance menu 64 /tmp/memcached.socket6;
start_instance page 64 /tmp/memcached.socket7;
start_instance update 64 /tmp/memcached.socket8;
start_instance views 64 /tmp/memcached.socket9;
start_instance session 64 /tmp/memcached.socket10;
start_instance users 64 /tmp/memcached.socket11;
start_instance pbold 64 /tmp/memcached.socket12;
}
stop () {
stop_instance default;
stop_instance block;
stop_instance content;
stop_instance field;
stop_instance filter;
stop_instance form;
stop_instance menu;
stop_instance page;
stop_instance update;
stop_instance views;
stop_instance session;
stop_instance users;
stop_instance pbold;
}
restart () {
stop
start
}
# See how we were called.
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status memcached
;;
restart|reload|force-reload)
restart
;;
*)
echo $"Usage: $0 {start|stop|status|restart|reload|force-reload}"
exit 1
esac
exit $?
To get the Drupal 7.x memcache module to report Memcached statistics (admin/reports/memcache) I've had to change memcache_admin/memcache_admin.module a little, because memcache will return its bins with a concatenated port ":0", which isn't the way the bins are stored in /tmp. Therefore to get a readout we need to append ":0" if the bin is a unix socket:
mcache_admin/memcache_admin.module261a262,263
> if(strstr($server, 'unix'))
> $server .= ':0';
So this is all still a bit rough on the edges, but I can happily report memcached does it's job though sockets on my D7 installation. I'm not quite sure about the devision of labour between APC, Memcached and Varnish yet, this probably needs tweaking, but I hope this does someone any good.
Ubuntu 12 version of Memcached startup script
Hi there
Is it possible to get a ubuntu version of Memcached startup script, i tried the above with replacing . /lib/lsb/init-functions and . /etc/network/ but where i get stuck is when i try and restart the service I get a :
tat: cannot stat `/var/run/memcached': No such file or directory
chown: invalid user: `memcached'
$Starting memcached (default): /etc/init.d/memcached: 42: /etc/init.d/memcached: daemon: not found
for all d bins created, please advise
thank you
there's always light at the end of the tunnel
Ubuntu 10.04 LTS
Hi together,
I used the posted init.d memcached script on an Ubuntu 10.04 LTS. Works fine when you create the following directories before:
Greetz from Tutzing / Germany
IT-Cru - Das IT-Gewächs
Webdevelopment and IT-Services
Dividing server memory into 12 separate caches?
You are starting 12 separate
memcached-processes dedicating each one to caching a separate piece. This divides the server's memory into 12 separate pieces. Are you sure, that's a good idea?Generally such resource-dedication is considered harmful — it is very easy to end up with a situation, where one cache has too much, while another — too little memory.
What, in your experience, is the downside to using a single (large)
memcachedinstance for all caches? I'm primarily interested in Drupal-7 integration, but any experience with Drupal-6 would be good to learn too. Thank you!Also, the documentation for memcache-module advises against using volatile cache for "
cache_form" — apparently, the data there can not be easily recreated. It is, probably, worth adjusting both your real config and this tutorial to ensure, thecache_formdata is still stored in the database.The choice to separate the
The choice to separate the bins into separate processes is inspired mainly to get some insight into which bins actually use a lot and which use a little memory. In our setup the default-bin uses most memory, followed by menu and filter. But you are right, the ratio of mem usage between default-bin and 'the rest' is quite off. The default-bin uses almost all memory (with a hit rate of 94% at 128MB), and the lowest used bin is block (with a hit-rate of 73% at not even 1MB).
I'm not aware that this separation is considered harmful. Do you have any links that can give some reasoning behind this? I don't really see any other benefit than insight into bin-usage for separating out the memcached instances. If there's good reasons not to do the separation I would consider putting them back together, but even though the instances have quite bit of memory at their disposal, they're hardly using it as resident memory... (except the default bin).
I don't have any experience with D6 and memcached unfortunately, so I can't say anything about this. Thanks for the read on the cache_form variable, that's good to know.
I'm not aware that this
Simple logic: by dedicating resources to different tasks yourself, you reduce efficiency — because some of the tasks will be starved of resources, while others will have too much. In your own example, whereas the default bin in your own setup would've used more memory, it can not, because you've dedicated additional chunks to bins, that don't fully utilize it.
If, instead, you were to use the same memory in a single chunk, you'd get a better utilization — heck, the size of that single chunk could be (quite a bit) smaller than the combined size of the 12 chunks you use currently.
Dedicating resources makes sense only, if they are distinct to begin with — such as when you are dealing with different physical hard-drives, for example, or when the OS (for whatever reason) would not give a process a single 768Mb chunk of memory, but does not mind 12 different processes each with a 64Mb chunk.
Ability to collect stats is nothing to sneeze at either, but you should be able to do it from a single chunk too.
session.save_handler in example wrong?
So, according to http://drupal.org/node/57150#comment-108379 it looks like setting the session.save_handler to anything other than "user" and session.save_path to '' is incorrect for Drupal.
When the save_handler is "user" ./includes/bootstrap.inc includes the memcache-session.inc file (if you've configured that part right) and that in turn handles saving Drupal sessions with memcache.
When I configured something like this I could see that there was data being written and read in both 11213 and 11214 using session.save_handler = 'user':
<?php'memcache_servers' => array(
'127.0.0.1:11212' => 'default',
'127.0.0.1:11213' => 'session',
'127.0.0.1:11214' => 'user',
),
'memcache_bins' => array(
'cache' => 'default',
'session' => 'session',
'users' => 'user'
),
?>
That doesn't seem to be the case with Drupal 7
As far as I can tell, it doesn't change
session.save_handleranywhere in its code, and neither does the Memcache module.Joakim Stai
Session handling hasn't been
Session handling hasn't been ported to D7 completely yet according to the status of this issue http://drupal.org/node/656838
The instructions on this page, like step 2, and some of the comments in this thread say to change the session save handler to memcache. I'm saying maybe that's not a good idea because it bypasses the pluggable Drupal implementation of sessions.
Strict enabled error
After editing the memcached config on Ubuntu 12.04, memcached refused to start, throwing the error:
Can't use string ("") as a subroutine ref while "strict refs" in use at /usr/share/memcached/scripts/start-memcached line 74, <$etchandle> line 121.It seems that the config file doesn't agree with memcached amd64 1.4.13-0ubuntu2