Let's consider using Redis hashes to store cache entries instead of multiple keys. From the Redis documentation:
Small hashes are encoded in a very small space, so you should try representing your data using hashes every time it is possible. For instance if you have objects representing users in a web application, instead of using different keys for name, surname, email, password, use a single hash with all the required fields.
Source: http://redis.io/topics/memory-optimization
Memory optimization depends on the hash-max-zipmap-entries and hash-max-zipmap-value settings. We may be able to recommend some values for optimal memory optimization with a "standard" Drupal install. However, these values will vary depending on how the cache is used. More advanced users will be able to analyze the cache to determine these settings based on their needs.
Some other benefits:
Using hashes allows us to store more key-value pairs in a single key. For example,
$hash = array('data' => $data, 'serialized' => $serialized, 'created' => $created, 'expire' => $expire, 'headers' => $headers);
$redis->hmset($key, $hash);
The hash can be retrieved with a single command:
$cache = (object) $redis->hgetall($key);
These snippets were taken from my D6 implementation. I'll try making a patch against your module for D7.
| Comment | File | Size | Author |
|---|---|---|---|
| #3 | hash_storage-1425062-3.patch | 5.1 KB | jtsnow |
Comments
Comment #1
pounardYes this is indeed a good idea. I just commited and released msonnabaum's patch which actually serialize an object in one key instead of using a hash, but I would definitely test the hash implementation since it sounds like a good compromised between his code -really more performant- and my original -a bit more complex but more memory friendly- code. It worth to test, and profile, and if that gives the same or better results than his patch, then it's a go.
Comment #2
pounardStill didn't forgot this issue, next one on my issue list :)
Comment #3
jtsnow commentedHere is a patch. I've tested with both Predis and PhpRedis. When using Predis, I couldn't see any noticeable difference in performance. However, when using PhpRedis, there was a big drop in performance when using hashes compared with serialized strings.
I'll see if I can improve the performance when using PhpRedis.
Comment #4
pounardYour PhpRedis code seems buggy I see some SET calls we should only see HSET I guess?I misread the code, sorry. Sounds good I will try it by myself as soon as I can.I'd like msonnabaum to do some benchmark of it in order to compare with serialized object (actual code) storage.
Comment #5
pounard@jtsnow, you patch is good, I will commit it in tonight.
Your PhpRedis implementation had a bug, a GET call was being done instead of a HGETALL which actually made the single cache entry fetch always fail, this is why it was slower for you.
I fixed it, also removed some useless string casts, I prefer to let Redis handle itself the datatypes, it will be more efficient this way and cause less network trafic. This is pretty much all I done to your code.
I did some measures on my box, and the hash storage is actually faster than the serialization based single key one when using PhpRedis, maybe this depends on the use cases, Redis version etc etc... Anyway, thank you very much for this, this was really helpful!
Comment #6
pounardOkay, pushed and tagged, new release comes out in a few minutes!
Comment #8
1mundus commentedHi pounard,
does this work as soon as we enable the module or we have to add "hash-max-ziplist-entries" and "hash-max-ziplist-value" in redis.conf?
Comment #9
1mundus commentedComment #11
pounardThis module won't cover Redis fine tuning (not yet at least), but it works without you needing to do anything special to Redis. So I guess the answer is that it works as soon as you enable the module. Feel free to reopen if you have any other questions about this issue.