Is apc.shm_size and kernel.shmmax related?
h3000 - March 27, 2009 - 08:39
| Project: | Cache Router |
| Version: | 6.x-1.0-beta8 |
| Component: | Miscellaneous |
| Category: | support request |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | active |
Jump to:
Description
Hi all,
Server is currently experiencing freeze ups once or twice a day. I don't see any errors in the logs, and mysql process list shows only null queries. Is this related to APC? Is there a relationship between the apc.shm_size setting and kernel.shmmax?
The server is on a standard ubuntu setup with shmmax setting unchanged at 32mb. apc.shm_size is currently set at 256 - if I set it at 32mb, the server just freezes. apc.php shows a lot of fragmentation when apc.shm_size is set at 256.
Do I need to change the shmmax setting? I have seen some discussion about changing shmmax size in standard Linux releases being too small.
Thanks
#1
I believe that they are related, I cannot find the source that I found that info in though.. Makes sense if you think about it, apc cannot have more shared mem than is available...
I run a debian (lenny) box and had to change the shm_size in /etc/default/tmpfs in order to increase apc's shm size to a reasonably effective number.
For some reason debian ships with a small amount of shared memory.
#2
As a test,, I've reduced the SHM_SIZE on my server back down to 32M while keeping apc.shm_size at 128M, and *still* apc seems to be using the larger amount (acc. to apc.php). Which would indicate that it is not using system shared memory at all.
EDIT: I have done some searching through the source code of apc.
It appears that apc has the ability to mmap memory, or to use sysv-style shared memory. This option is decided when apc is first installed (it is in the m4 macro that generates the configure file).
AC_MSG_CHECKING(whether we should use mmap)AC_ARG_ENABLE(apc-mmap,
[ --disable-apc-mmap
Disable mmap support and use IPC shm instead],
[
PHP_APC_MMAP=$enableval
AC_MSG_RESULT($enableval)
], [
PHP_APC_MMAP=yes
AC_MSG_RESULT(yes)
])
From the technotes:
It mmaps a temp file andthen unlinks that file right after the mmap to provide automatic shared memory
cleanup in case the process dies.
From the code (apc_sma.c - the mem allocator functions)
#if APC_MMAPsma_segments[i] = apc_mmap(mmap_file_mask, sma_segsize);
...
#else
sma_segments[i] = apc_shm_attach(apc_shm_create(NULL, i, sma_segsize));
#endif
apc_mmap()andapc_shm_create()are the cache memory allocators for apc, and are supposed to/will allocate up to apc.shm_size * amp_shm_segments "memory".Looking at my site's apc.php, you find the following:
Shared Memory 1 Segment(s) with 128.0 MBytes(mmap memory, pthread mutex locking)
My site appears to be mmapping instead of using shm, which is why apc can use more than SHM_SIZE of memory.
It is better IMO to use shared memory, because it is *actual RAM*, whereas the mmapped memory is just a temp file on the disk.
NOTE: I just found these articles:
http://www.imminentweb.com/technologies/tune-apc-improve-php-performance
http://groups.google.com/group/eaccelerator/browse_thread/thread/003df07...
They validate the above.
Why 2 hours of googling didnt find these, and I had to grep through a few thousand lines of C, I have no idea ;)
#3
To deal with your issues:
- What do you mean by NULL queries?
- Is there a relationship between apc.shm_size and SHM_SIZE? Well, maybe ;) (see above)
- Fragmentation: I would try increasing the amount of shm, if you can (that is if you are using mmapped memory - see apc.php)
I did some googling, found :
apparently the issue is a too high rate ofinsertions by multiple processes concurrently. If every process uses the
same APC key to store, they conflict and only one of them wins and the
other leave a "hole" in the memory space. This should not take the
server down though. He says he's already working on improving this, and
this should hit CVS soon (ie, before the end of the month). He also
suggests that you try the current CVS version of APC to see if that
already solves the taking down of the server. Could you perhaps try
that?
Do you have a high quantity of simultaneous connections?
Have you tried updating to the latest version of apc from cvs?
- There seem to be several ways to change the shm size:
1) (debian-based) Edit /etc/default/tmpfs, define or set the variable SHM_SIZE.
2) You can set /proc/sys/kernel/shmmax directly, but that won't survive a reboot.
3) You can define/set kernel.shmmax in /etc/sysctl.conf
Once you reboot, you can check it's value at:
cat /proc/sys/kernel/shmmax
#4
Subscribing. Would be good to work out how to configure APC properly. I think it even deserves a book page in the Drupal documentation. =)
#5
OK, more info:
The PECL package for APC, as of v 3.0.11 (8-16-2006) defaults to mmap's memory instead fo sysV-style shared memory:
Link
*Made --enable-apc-mmap the default compile optionAll of the distros that I checked (Debian, Ubuntu, Fedora) use PECL and the --enable-mmap .configure option when installing APC via their package manager.
Refs:
Fedora 11 php-pecl-apc rpm source:
45 %configure --enable-apc-mmap --with-apxs=%{_sbindir}/apxs --with-php-config=%{_bindir}/php-configDebian lenny php-apc deb source:
./configure --with-php-config=$(PHPCONFIG) --prefix=/usr \--enable-apc --enable-apc-mmap --with-apxs=/usr/bin/apxs2)
So, if APC is installed using PECL, or via apt-get or yum, it will use mmap'd memory instead of the system's shared mem (and therefore kernel shmmax is not relevant).
#6
Ok, so basically you have to compile from source if you want to use shared memory?
#7
This does not mean that you cannot use shared memory, it just means that the type of shared memory that will be used by APC is not actual kernel shared memory/"shm" (an actual chunk of RAM mounted as a tmpfs filesystem); rather APC will create a temporary file that will masquerade as memory ("mmap").
So to summarize, APC's default configuration is to cache everything to a file that is pretending to be RAM. If you compile it from source without the '--enable-mmap' ./configure option, it will use actual RAM, and it will be up to you to make sure that your operating system has provided that RAM (as kernel shm).
I hope that is clear.