Hi,
we have an issue on our environment. Following you can find the architecture details: we have 4 web front-end Centos 6.2 x86_64 hosts with this software installation:

drupal 7
php-5.3.3
apache 2.2
memcached-1.4.4-3
php-pecl-memcache-3.0.5-3

The Apache's document root (/var/www/html/) is shared (NFS) across the 4 front-ends and all the servers have three memcached istances with this configuration:

memcached -d -p 11213 -u memcached -m 2048 -c 5120 -P /var/run/memcached/memcached-3.pid
memcached -d -p 11211 -u memcached -m 2048 -c 5120 -P /var/run/memcached/memcached-1.pid
memcached -d -p 11212 -u memcached -m 2048 -c 5120 -P /var/run/memcached/memcached-2.pid

The drupal configuration for memcache (settings.php) is shared (is it correct?):

$conf['cache_backends'][] = 'sites/all/modules/memcache/memcache.inc';
$conf['cache_default_class'] = 'MemCacheDrupal';
$conf['cache_class_form'] = 'DrupalDatabaseCache';

$conf['memcache_servers'] = array(
'host2:11211' => 'default',
'host3:11211' => 'default',but
'host4:11211' => 'default',

'host3:11212' => 'page',
'host4:11212' => 'page',
'host5:11212' => 'page',

'host2:11213' => 'menu',
'host4:11213' => 'menu',
'host5:11213' => 'menu',

'host5:11211' => 'users',
'host2:11212' => 'users',
'host3:11213' => 'users',
);
$conf['memcache_bins'] = array(
'cache' => 'menu',
'cache_menu' => 'menu',
'cache_bootstrap' => 'menu',
'cache_path' => 'menu',
'cache_page' => 'page',
'cache_block' => 'page',
'session' => 'users',
'users' => 'users',
);

Sometimes we have in /var/log/messager this error:
User error: Failed to connect to memcache server: host3:11212 in dmemcache_object() (line 415 of /var/www/html/sites/all/modules/memcache/dmemcache.inc)

and this error is also visible on the browser.

We noticed we have many time_wait (about 8000) on every server. Is it normal ?

We have already set the tcp_tw_reuse=1 parameter but we read that is not good in a load balancer environment the follow parameters tcp_tw_recycle=1

Any suggestions? Do you think it could be a better way to improve the configuration?

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

girishmuraly’s picture

Component: Documentation » Code
Category: task » bug

Due to the fact that the 'User errors' show up on the screen to logged out users as well, I think this should be a bug report.

paullomax’s picture

We just got that error show up on our live site, visible to users, which got cached by Varnish.

We use AWS ElastiCache, which must have a temp connection problem or lag.

Is there a time out setting that can be changed somewhere?

It really should fail more gracefully.

ludo1960’s picture

Same problem here, anyone any ideas?

girishmuraly’s picture

Here's a patch to hide the embarrassing error message from non-administrators.

With respect to the original timeout issues in #0, the settings seem ok to me. Is this only periodical? Were you able to resolve it so far?

marcelovani’s picture

I have removed another message.
This patch replaces trigger_error_admin_only-1470324-4.patch.

fayedrupal’s picture

This problem cause by telnet firewall configuration

check your iptables --list
check the rules
the changes of ip -s -d will cause this problem

markpavlitski’s picture

Title: Memcache NFS settings.php - User error: Failed to connect to memcache server: host3:11212 in dmemcache_object() (line 415 » Missing PECL extension error is displayed to all users
Priority: Major » Normal
Status: Active » Needs review
FileSize
718 bytes

#1830496: Better logging and handling of connect errors has partially fixed this issue.

This patch converts the missing PECL extension message to use the register_shutdown_handler() technique, and ensures the error is only logged once per request.

marcelovani’s picture

Status: Needs review » Fixed
markpavlitski’s picture

Status: Fixed » Needs review

The patch in #1830496 only partially fixes the issue.

An error is still displayed to all users using drupal_set_message() if neither PECL extension is installed.

The patch in #7 is required to fully fix this issue.

Jeremy’s picture

Status: Needs review » Closed (works as designed)

This is by design -- you can't use memcache w/o the extension, and it's an error to enable it w/o the extension. What's a legitimate reason for masking this error state? Re-open and explain if this is in fact causing a problem.

markpavlitski’s picture

Status: Closed (works as designed) » Needs review

@Jeremy I don't believe this patch would mask the issue, instead I think it should be included for the following reasons. If you disagree feel free to close the ticket again.

  1. drupal_set_message() should be used to display messages to end users, not to display site-wide errors. Watchdog is the best fit for this, and is used everywhere else in the memcache module. If the developer has display log messages enabled they will still see these message on screen, if not we should respect their settings. Either way end users shouldn't see this error.
  2. drupal_set_message() calls drupal_page_is_cacheable(FALSE), which disables page caching. Since this message is displayed indiscriminately, this could result in a significant performance penalty for all requests (including anonymous). Although it would be the developers fault for incorrectly configuring their system, we shouldn't introduce further performance problems on top.

The patch also has a secondary effect of reducing the errors shown to one per page rather than one per memcache request. If you prefer I can roll a patch just for that in a separate ticket.

Jeremy’s picture

Status: Needs review » Needs work

My feeling is: if you're enabling memcache without installing a PECL extension, you're doing it wrong in a very basic and problematic way. All cache attempts are already failing. If you've configured your environment like this, you need all the help you can get -- one would hope you're just testing the module in a dev or staging environment.

My concern with moving this to the log is that it may go unnoticed, and people won't be aware they've accidentally disabled all caching. At this early step I think we should be giving as much help as we can.

That said, I'd commit a clean patch for only displaying the error once per page load. (And I'm still thinking about this, I could be persuaded if people are hitting this in an unexpected way.)

markpavlitski’s picture

Here's the patch just to limit errors to 1-per page.

I agree that the developer would need to set it up fundamentally wrong to get to this situation, and I'd fully expect any cache bins using memcache to fail miserably, however it may not be the only cache in place.

A common use-case would be to use varnish module for page caching, and as-is this would also break varnish.

In the case you mentioned where it's a dev/staging site, chances are that errors will be set to display on screen anyway, since it's the default. And most people check the error logs as a matter of course when deploying a new site to production so would pick it up there.

Since the site will be slow either way I think not confusing site users is better than ensuring the developer sees it; alternatively, we could just wrap a $user->uid == 1 check around drupal_set_message()?

Having said that, I don't think this is a critical issue for the 7.x-1.1 release so am happy for this to be delayed for now to focus on the release.

Jeremy’s picture

Status: Needs work » Fixed

This was fixed as part of the cleanup done in #1172754: Add a warning on module_enable() if the module.

(FWIW, note that drupal_set_message has a "repeat" parameter that prevents the same message from showing up more than once.)

markpavlitski’s picture

@jeremy true, but the repeat parameter uses in_array() to search for string matches, which seemed like a bad idea given it was running dozens of times per page view.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.