Hi,

I'm maintainer of Memcache Storage module. This module is alternative to Memcache module, but its works faster and more stable (see difference).

I'd love to move Memcache Storage into 7.x-2.x branch of Memcache module and continue development here.
What do you think about it?

Comments

podarok’s picture

Status: Active » Reviewed & tested by the community

nice to see collaboration here
+1

andypost’s picture

+1 to 7.x-2.x-dev branch

Jeremy’s picture

Can you provide a little insight into the background of the Memcache Storage code base? Is it entirely from scratch, or a fork, or a combination?

Also, can you provide more insight into your performance tests -- what was the process for setting up the test, and what tools were used? The more detail you can provide, the better.

Spleshka’s picture

It's entirely from scratch.

I tried to apply some features from Memcache module, use memcache best practise, and used my expirience with memcached on my high-load projects.

About performance test, see code I used to compare speed :

$bins = array('cache', 'cache_path', 'cache_filter', 'cache_bootstrap', 'cache_page');
$data = array_fill(0, 100, rand());

timer_start('cache_set');
for ($i = 0; $i < 2000; $i++) {
  foreach ($bins as $bin) {
    cache_set($bin . '_' . $i, $data, $bin);
  }
}
print '10 000 sets: ' . timer_read('cache_set') . ' ms<br>';

timer_start('cache_get');
for ($i = 0; $i < 2000; $i++) {
  foreach ($bins as $bin) {
    cache_get($bin . '_' . $i, $bin);
  }
}
print '10 000 gets: ' . timer_read('cache_get') . ' ms<br>';

timer_start('cache_clear_all');
for ($i = 0; $i < 2000; $i++) {
  foreach ($bins as $bin) {
    cache_clear_all($bin . '_' . $i, $bin);
  }
}
print '10 000 deletes: ' . timer_read('cache_clear_all') . ' ms<br>';

First I set Memcache Storage as cache backend and run this code, then set Memcache and repeat this test again.

Jeremy’s picture

I will try and make time to spin up a regression-testing image I built some time ago to debug and performance test a number of issues in the memcache module. I'm pretty busy at the moment though, and can't promise anything.

I'm not in support of putting a brand new and relatively untested code base into the 7.x-2.x branch at this time; I'd much rather see effort go into bug fixing and otherwise improving the current code base (I'm aware we've done a really poor job of keeping up with the issue queue for a crazy long time). Regardless, I'll still try and find time to run a complete regression test on your code base and see if that changes my mind.

Spleshka’s picture

Ok, thanks Jeremy, I'll be looking forward your results. Please, feel free to ask me for any help you need.

quicksketch’s picture

Hi @Jeremy and @Spleshka. It's great to see collaboration here. The memcache module has been sorely lacking any development work for years now. The README/INSTALL.txt files don't even include D7 instructions yet and #1536222: Update README.txt for Drupal 7, remove INSTALLATION.txt has been sitting dormant for almost a year. The attitude towards Memcache 7.x-1.x has always been "it sorta works but is probably broken", and so I wouldn't mind a new branch entirely to remove that stigma.

Jeremy’s picture

@quicksketch collaboration would be working together to clean up this module. Throwing it all away and starting over is in no way collaboration, and is better done in a new project, as has already been done.

quicksketch’s picture

Yeah, fair enough. Even if it's not collaboration on the existing project, I'd advocate that the best module get the memcache namespace. I haven't validated that Memcache Storage is superior, but if it turns out to be the case I'd like to see it take the namespace. The memcache project has been neglected for so long my assumption is that anything has to be better than the current situation.

DamienMcKenna’s picture

+1 to making positive progress.

Spleshka’s picture

@Jeremy, any progress with review/tests of Memcache Storage?

markpavlitski’s picture

@Spleshka Here are the results I get from running your test script. This is just with a basic default configuration, so it would be interesting to see what settings you use.

memcache_storage:
10 000 sets: 974.13 ms
10 000 gets: 901.09 ms
10 000 deletes: 597 ms
Page execution time was 2574.65 ms

memcache (PECL Memcached):
10 000 sets: 545.75 ms
10 000 gets: 770.11 ms
10 000 deletes: 280.89 ms
Page execution time was 1714.31 ms

memcache (PECL Memcache):
10 000 sets: 1334.58 ms
10 000 gets: 975.19 ms
10 000 deletes: 737.08 ms
Page execution time was 3257.84 ms

It looks like the memcache module using the PECL Memcached extension comes out on top, though the memcache_storage module is faster when comparing like for like with the memcache extension. Do you have any plans to implement the PECL Memcached extension as well?

Spleshka’s picture

@markpavlitski,

Thanks for the test results! I never run this script for PECL memcached extension, but now I will definitely do it. And if PECL memcached will be faster I will implement this extension too, sure.

About module settings - I can't reproduce them now, but I changed this two settings to get my results:

$conf['memcache_storage_compress_data'] = MEMCACHE_COMPRESSED;
$conf['memcache_storage_compress_threshold'] = array('threshold' => 10000, 'min_savings' => 0.1);
Jeremy’s picture

Title: Co-maintainer » Request to replace module
Category: support » task
Status: Reviewed & tested by the community » Postponed (maintainer needs more info)

Updating title and Status -- as noted before, I'm only interested in improvements and bugfixes, not wholesale replacement.

ygerasimov’s picture

Jeremy, can you please add link to memcache_storage module on the project page of memcache module? #1940554: Memcache module alternative

andypost’s picture

I think better way to allow Memcache storage to grow a bit - to get more eyes on its code.
There's no test coverage on new module so they should be written to get a chance to replace.
Let's leave postponed while working on #1974254: Memcache Storage Roadmap

Spleshka’s picture

@markpavlitski,

Added PECL Memcached support to Memcache Storage then run some tests using the same script:

Memcache Storage (PECL Memcache)
10 000 sets: 1232.38 ms
10 000 gets: 1317.51 ms
10 000 deletes: 819.4 ms

Memcache Storage (PECL Memcache) + UNIX SOCKET
10 000 sets: 1079.81 ms
10 000 gets: 1083.23 ms
10 000 deletes: 670.9 ms

Memcache Storage (PECL Memcached)
10 000 sets: 381.98 ms
10 000 gets: 1010.02 ms
10 000 deletes: 145.95 ms

Memcache Storage (PECL Memcached) + UNIX SOCKET
10 000 sets: 352.63 ms
10 000 gets: 908.82 ms
10 000 deletes: 142.94 ms

Memcache API (PECL Memcache)
10 000 sets: 1745.43 ms
10 000 gets: 1543.33 ms
10 000 deletes: 890.73 ms

Memcache API (PECL Memcache) + UNIX SOCKET
10 000 sets: 1547.99 ms
10 000 gets: 1348.77 ms
10 000 deletes: 712.43 ms

Memcache API (PECL Memcached)
10 000 sets: 446.28 ms
10 000 gets: 1067.4 ms
10 000 deletes: 170.53 ms

Memcache API (PECL Memcached) + UNIX SOCKET
**Not working**

MySQL (Percona) + fun :)
10 000 sets: 33322.87 ms
10 000 gets: 2457.66 ms
10 000 deletes: 22653.61 ms

For PECL Memcached I used this settings:

$conf['memcache_options'] = array(
  Memcached::OPT_TCP_NODELAY => TRUE,
  Memcached::OPT_BUFFER_WRITES => TRUE,
  Memcached::OPT_NO_BLOCK => TRUE
);

For PECL Memcache was used default settings.

As you may see, Memcache Storage + PECL Memcached + Unix sockets are best choise for now.

Spleshka’s picture

Today Memcache Storage got stable release. Can we continue discussion about merging it into 7.x-2.x branch of Memcache API?

Jeremy’s picture

As noted before, I'm only interested in improvements and bugfixes, not wholesale replacement.

deanochips’s picture

maybe working together on a release for Drupal 8,
ready for when its released might be a solution.....

Spleshka’s picture

I will provide release for D8 after 1st of July, but for Memcache Storage.

Spleshka’s picture

@Jeremy,

What if I migrate Memcache Storage to D8 and then we merge it into 8.x of Memcache API? Or you want to make your own migration for Memcache API?

Jeremy’s picture

Status: Postponed (maintainer needs more info) » Fixed

If you want to help make incremental improvements to the existing Memcache API, that's fantastic. I'm not interested in a wholesale replacement in any branch.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

meba’s picture