Integration with cache_browser module
markus_petrux - February 23, 2009 - 14:11
| Project: | Cache Router |
| Version: | 6.x-1.x-dev |
| Component: | Code |
| Category: | feature request |
| Priority: | normal |
| Assigned: | slantview |
| Status: | needs work |
Description
Hi,
I'm the author of the cache_browser module and I'm wondering if it would be possible to integrate with cache router.
For this, we would need a couple of things that cache router would have to provide:
1) A method to get a list of cache tables managed by cache router (name and description). Probably from $conf array?
2) A method to get a list of cache entries managed by each table.
Probably the second thing is harder to do?
#1
Hi,
I really like this idea. It would be especially useful once I get the statistics information available. I don't think it would be too difficult to add some functionality that allows you to get a list of cache entries at all. I could add a method to each cache engine.
I'm gonna put this in patch (code needs work) as we need a patch, if someone does it before me, great, otherwise I'll add it to my list of things to do.
Steve
#2
Sweet!
If that helps, here's a snippet of code I have extracted from a program I use to monitor memcache entries on our forums.
<?php
$mmc_servers = array(
'127.0.0.1' => '11111',
);
$max_entries_per_request = 1000;
$mmc_items = array();
$mmc_items_count = $mmc_items_size = 0;
foreach ($mmc_servers as $host => $port ) {
$mmc = new Memcache;
if ($mmc->connect($host, $port)) {
$slabs = $mmc->getStats('slabs');
foreach ($slabs as $slab_id => $slab_info) {
$slab_dump = $mmc->getStats('cachedump', $slab_id, $max_entries_per_request);
foreach ($slab_dump as $item_key => $item_info) {
if (!isset($mmc_items[$item_key])) {
$mmc_items[$item_key] = array(
'host' => $host,
'slab' => $slab_id,
'lru' => $item_info[1],
'size' => $item_info[0],
);
$mmc_items_size += $mmc_items[$item_key]['size'];
$mmc_items_count++;
}
}
}
$mmc->close();
}
unset($mmc);
}
ksort($mmc_items, SORT_STRING);
var_dump($mmc_items);
?>
I wrote that a long time ago, and still works on our site :) ...though, we're not running the latest memcache/memcached version, so maybe there's an easier way to get this information.
#3
It's really interesting - enumerate cache entries in memcache, it can be very useful to clear which is main problem of cacherouter
@markus_petrux Do you have some performance statistics about this - how long it takes to enumerate 1k , 10k entries?
#4
Unfortunately, nope. I use a similar code in our phpBB forums that I modified to cache just a few queries. We have 52 different entries in 3 separate memcached hosts (we have 4 apaches querying 3 memcached hosts via 1 Gbps ethernet). I runs quite fast, but I cannot tell how exactly.
I have modified the code to run the memcached queries in a loop, 100 times (which then reads 5200 entries from 3 hosts) and it took around 1 second, more or less.
#5
Code to enumerate keys for:
APC
<?php
$info = apc_cache_info('user', FALSE);
//var_dump($info);
print 'Entries: '. count($info['cache_list']) ."\n";
foreach ($info['cache_list'] as $entry) {
//print $entry['info'] .', ';
if (strpos($entry['info'], $mask) === 0) {
//apc_delete($entry['info']);
}
?>
Entry
array(10) {["info"]=>
string(14) "varvarvarvar10"
["ttl"]=>
int(0)
["type"]=>
string(4) "user"
["num_hits"]=>
int(0)
["mtime"]=>
int(1237770418)
["creation_time"]=>
int(1237770418)
["deletion_time"]=>
int(0)
["access_time"]=>
int(1237770418)
["ref_count"]=>
int(0)
["mem_size"]=>
int(118)
}
and for eAccelerator
<?php
$info = eaccelerator_list_keys();
//var_dump($info);
print 'Entries: '. count($info) ."\n";
foreach ($info as $entry) {
//print $entry['name'] .', ';
if (strpos($entry['name'], $mask) === 0) {
eaccelerator_rm($entry['name']);
}
}
?>
Entry
array(4) {["name"]=>
string(30) "pd--cache-imagecache%3Apresets"
["ttl"]=>
int(0)
["created"]=>
int(1237751104)
["size"]=>
int(1194)
}
my tests on 10000 items (scripts attached)
APC
storing... elapsed 0.034 seconds
fetching info... elapsed 0.057 seconds
Entries: 10000
delete... elapsed 0.085 seconds
cleaning all... elapsed 0.001 seconds
storing... elapsed 0.033 seconds
fetching info... elapsed 0.065 seconds
Entries: 10000
delete... elapsed 0.063 seconds
cleaning all... elapsed 0.001 seconds
eAccelerator
storing... elapsed 0.391 seconds
fetching info... elapsed 0.026 seconds
Entries: 10007
delete... elapsed 0.231 seconds
storing... elapsed 0.401 seconds
fetching info... elapsed 0.026 seconds
Entries: 10007
delete... elapsed 0.230 seconds
#6
Forget to say -
strpos()highly depends on size of key#7
No updates in a while... Anyone still working on this?