receiving a PHP notice "Connection reset by peer .... dmemcache.inc on line 104"

I'm getting a different error, same line. Not sure if it's related. I am on 6.x-1.x-dev as of 2010-May-26, with Pressflow 6.16.77, Fedora 11 x86_64, stock memcached-1.2.8, php-pecl-memcache-3.0.4, php 5.2.13.

[Tue Jul 06 15:22:39 2010] [error] [client 1.2.3.4] PHP Notice:  MemcachePool::get() [<a href='memcachepool.get'>memcachepool.get</a>]: Failed to uncompress data in /mypath/drupal/sites/all/modules/memcache/dmemcache.inc on line 107, referer: http://mysite.com/path/15975

I have multiple buckets, and I'm getting the same message for different buckets. It has not reset connections that I am aware of and the site seems to be working, just a lot of notices.

My next move is going to be to upgrade memcached to 1.4.5 and move to the php-pecl-memcached-1.0 package.

EDIT: I'm going to start with just migrating to php-pecl-memcached-1.0 and see how that works. I'll report back.

I upgraded to php-pecl-memcached-1.0 and so far after 1/2 day of work on the site have not seen that error come back even one time. I made no other changes or upgrades, so I'm going to suggest that anyone running into this try upgrading to php-pecl-memcached from php-pecl-memcache.

I fear we're hijacking this issue but I had the same problem as rjbrown99 and it was solved by switching to php-pecl-memcached (php5-memcached package in Ubuntu).

Thanks #8, sounds like a very good find. I popped in the patch tonight with $conf['memcache_persistent'] = TRUE; in settings.php. I'll let you know how it goes.

+      // Indicate wether to conncet to memcache

typo for connect.

Patch looks good otherwise.

Benchmarks:

Current dev:

STAT total_connections 11858
STAT cmd_get 120468
STAT cmd_set 57259
STAT get_hits 63100
STAT get_misses 57368
STAT incr_hits 0
STAT bytes_read 67173183
STAT bytes_written 167311928
STAT evictions 6378
STAT total_items 57259

Hit rate: 52.3700%
Miss rate: 47.6200%

HTTP return codes:  20x(11845) 30x(1310) 40x(2)  50x(0)
 200: 11845
 302: 1310

Pages per second: 20.81

Patch no longer applied, so attaching re-roll, will include benchmark results with the patch (persistent connections and not) next followup.

With the patch, I'm testing with a single memcache server, so it should be exactly the same with the default setting, however it's not:

Current dev:

STAT total_connections 12067
STAT cmd_get 122032
STAT cmd_set 57907
STAT get_hits 64014
STAT get_misses 58018
STAT incr_hits 0
STAT bytes_read 67896561
STAT bytes_written 169612824
STAT evictions 6302
STAT total_items 57907

Hit rate: 52.4500%
Miss rate: 47.5400%

HTTP return codes:  20x(12054) 30x(1328) 40x(2)  50x(0)
 200: 12054
 302: 1328

Pages per second: 21.17

Patch and default setting (not persistent connections) - not what I expected:

/root/jmeter
Runtime error (func=(main), adr=9): Divide by zero
Runtime error (func=(main), adr=9): Divide by zero
STAT total_connections 11
STAT cmd_get 0
STAT cmd_set 0
STAT get_hits 0
STAT get_misses 0
STAT incr_hits 0
STAT bytes_read 6
STAT bytes_written 0
STAT evictions 0
STAT total_items 0

Hit rate: %
Miss rate: %

HTTP return codes:  20x(7901) 30x(827) 40x(1)  50x(0)
 200: 7901
 302: 827

Pages per second: 13.81

Patch and persistent connections:

STAT total_connections 41
STAT cmd_get 121671
STAT cmd_set 57754
STAT get_hits 63807
STAT get_misses 57864
STAT incr_hits 0
STAT bytes_read 67837672
STAT bytes_written 169387084
STAT evictions 6183
STAT total_items 57754

Hit rate: 52.4400%
Miss rate: 47.5500%

HTTP return codes:  20x(12014) 30x(1327) 40x(2)  50x(0)
 200: 12014
 302: 1327

Pages per second: 21.07

So there's a bug in the non-persistent code flow somewhere, I think this patch might do it.

That didn't fix it, I still get this:

/root/jmeter
Runtime error (func=(main), adr=9): Divide by zero
Runtime error (func=(main), adr=9): Divide by zero
STAT total_connections 11
STAT cmd_get 0
STAT cmd_set 0
STAT get_hits 0
STAT get_misses 0
STAT incr_hits 0
STAT bytes_read 6
STAT bytes_written 0
STAT evictions 0
STAT total_items 0

Hit rate: %
Miss rate: %

HTTP return codes:  20x(7871) 30x(824) 40x(1)  50x(0)
 200: 7871
 302: 824

Pages per second: 13.73

Updating latest tidied up version of the patch, but looks like this needs work.

Alright not sure what was going on there, but this one works fine:

STAT total_connections 12022
STAT cmd_get 121899
STAT cmd_set 57544
STAT get_hits 64242
STAT get_misses 57657
STAT incr_hits 0
STAT bytes_read 67763625
STAT bytes_written 169482096
STAT evictions 3186
STAT total_items 57544

Hit rate: 52.7000%
Miss rate: 47.2900%

HTTP return codes:  20x(12009) 30x(1333) 40x(2)  50x(0)
 200: 12009
 302: 1333

Pages per second: 21.08

Back to CNR.

The performance here with and without persistent connections is about the same, it's possible that a different load test plan would show a difference, but the fact there's no obvious change either way (short of my weird results earlier) makes me think we could commit this happily enough.

It may be we'd see a bigger performance change with multiple connections and/or longer load tests - since 5 connections per request * 1000 requests you'd save 4955 connections with persistent connections on.

That's a good point, adding just that one change.

I'm going to mark this RTBC, but I'm offline the next 3-4 days so will leave it there for Jeremy to decide whether to commit.

Agreed that this patch is fixes a big when using the Memcache PECL extension, as addServer() defaults to TRUE for persistent connection [1], while we are using connect and not pconnect for the initial connection. Researching persistent connections an re-running load tests, I agree with the choice to keeping persistent connections disabled by default (as it can result in memcache running out of available connections the memcache server isn't tuned on busy websites).

The patch is wrong when using the Memcached PECL extension, however. For the Memcached extension, the addServer() parameters change [2] so with this patch we end up adjusting the 'weight' parameter instead of enabling persistent connections. Further research online suggests that support for persistent connections is buggy in the Memcached extension, and reading these docs and the comments below it's a little confusing [3] which has led some to erroneously report memory leaks [4].

[1] http://php.net/manual/en/memcache.addserver.php
[2] http://php.net/manual/en/memcached.addserver.php
[3] http://www.php.net/manual/en/memcached.construct.php
[4] http://pecl.php.net/bugs/bug.php?id=16740

I'm thinking it may be best to simply document that this is a Memcache PECL extension only option for now, and to revisit adding support for persistent connections with the Memcached PECL extension at a later time. This allows us to fix the bug that was initially reported in this issue without getting hung up on adding a new feature.

I updated the patch to work with both PECL extensions, and documented the fact that we currently only support persistent connections with the Memcache PECL extension. Committed, thanks for tracking this bug down!
http://drupalcode.org/project/memcache.git/commitdiff/3eaa8f8

The new memcache_admin statistics screen was very helpful in confirming that we were properly enabling/disabling persistent connections (as the total # of connections only increases when persistent connections are disabled).

This has been pulled into 7.x