Hello

I'm trying to get Varnish up & running.
A plain default Varnish install (+adjusting settings of the DAEMON_OPTS) and I already notice a huge performance gain.

I still have 2 questions:

1) From Fourkitchens (and a lot of other sites) I saw an 'optimized' default.vcl to tweak performance with Drupal sites even more. However, when I try to use the (attached) default.vcl, Varnish won't (re)start. Can somebody share his/her optimal default.vcl that works with Varnish 3.x?

2) In D6, you could choose the caching mode ('external' for use with Varnish), in D7 this option isn't available. Any specific guidelines about this? Do I just need to turn on caching in Drupal? (when I use Varnish, the Drupal X-Cache is still visible in the headers now)

Regards.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

amanaplan’s picture

Status: Active » Needs review

I recommend the practices found in the article below (which is complete with a VCL file and health check file):
http://www.lullabot.com/articles/varnish-multiple-web-servers-drupal

We use most of those practices in our environments.

In addition, you'll need to to add (or consider adding by making sure to read the comments) the following to your settings.php file:


/**
 * Enable this setting to determine the correct IP address of the remote
 * client by examining information stored in the X-Forwarded-For headers.
 * X-Forwarded-For headers are a standard mechanism for identifying client
 * systems connecting through a reverse proxy server, such as Squid or
 * Pound. Reverse proxy servers are often used to enhance the performance
 * of heavily visited sites and may also provide other site caching,
 * security or encryption benefits. If this Drupal installation operates
 * behind a reverse proxy, this setting should be enabled so that correct
 * IP address information is captured in Drupal's session management,
 * logging, statistics and access management systems; if you are unsure
 * about this setting, do not have a reverse proxy, or Drupal operates in
 * a shared hosting environment, this setting should remain commented out.
 */
$conf['reverse_proxy'] = true;

/*
 * When Varnish is used, "page_cache_invoke_hooks" must be set as "false". While static files
 * (JS, CSS, images, etc.) all show the proper cache lifetime, the nodes themselves
 * will send back "max-age=0", causing a reverse proxy cache miss on eminently-cacheable content.
 * Without the last variable, anonymous users will get nodes/pages with the max-age=0
 * and the proxy won’t cache it. Setting it to false allows the node to be sent with the max-age set
 * to the cache lifetime set in the admin UI.
 *
 * @see http://drupal.org/node/804864
 */
$conf['page_cache_invoke_hooks'] = false;
$conf['cache'] = 1;
$conf['cache_lifetime'] = 0;
$conf['page_cache_maximum_age'] = 21600;

/**
 * Set this value if your proxy server sends the client IP in a header other
 * than X-Forwarded-For.
 *
 * The "X-Forwarded-For" header is a comma+space separated list of IP addresses,
 * only the last one (the left-most) will be used.
 */
$conf['reverse_proxy_header'] = 'HTTP_X_FORWARDED_FOR';

/**
 * reverse_proxy accepts an array of IP addresses.
 *
 * Each element of this array is the IP address of any of your reverse
 * proxies. Filling this array Drupal will trust the information stored
 * in the X-Forwarded-For headers only if Remote IP address is one of
 * these, that is the request reaches the web server from one of your
 * reverse proxies. Otherwise, the client could directly connect to
 * your web server spoofing the X-Forwarded-For headers.
 */
$conf['reverse_proxy_addresses'] = array('127.0.0.1');

/**
 * Page caching:
 *
 * By default, Drupal sends a "Vary: Cookie" HTTP header for anonymous page
 * views. This tells a HTTP proxy that it may return a page from its local
 * cache without contacting the web server, if the user sends the same Cookie
 * header as the user who originally requested the cached page. Without "Vary:
 * Cookie", authenticated users would also be served the anonymous page from
 * the cache. If the site has mostly anonymous users except a few known
 * editors/administrators, the Vary header can be omitted. This allows for
 * better caching in HTTP proxies (including reverse proxies), i.e. even if
 * clients send different cookies, they still get content served from the cache
 * if aggressive caching is enabled and the minimum cache time is non-zero.
 * However, authenticated users should access the site directly (i.e. not use an
 * HTTP proxy, and bypass the reverse proxy if one is used) in order to avoid
 * getting cached pages from the proxy.
 */
$conf['omit_vary_cookie'] = true;

Then navigate to the Performance page at ?q=admin/config/development/performance and enable the page cache (click Cache pages for anonymous users) and set a non-zero time for "Expiration of cached pages".

flatcircle’s picture

Thanks for the info but this doesn't work.

When I copy/paste the .vcl file from http://www.lullabot.com/articles/varnish-multiple-web-servers-drupal. I can't start the Varnish service. (I can start Varnish with v2.0.x)

Like I said, it seems like the syntax used in the .VCL file is not compatible for use with Varnish 3.0.

flatcircle’s picture

I narrowed it down.

Seems like:

 sub vcl_hash {
   if (req.http.Cookie) {
     set req.hash += req.http.Cookie;
   }
 }

doens't work (anymore) with Varnish 3.0.

Any ideas?

fabsor’s picture

use

 sub vcl_hash {
   if (req.http.Cookie) {
     vcl_hash(req.http.Cookie);
     return (hash);
   }
 }
flatcircle’s picture

Below is the content of my default.vcl:

I can start Varnish (3.0) but it seems that Varnish isn't caching.

1) In Firebug: I only see one Varnish timestamp instead of two (so varnish isn't serving from cache)
(f.e. X-Varnish 453348501)

2) When I check with 'varnishlog -b' and refresh my Drupal pages, I see that Varnish still makes requests to the web backend instead of serving from cache.

Any ideas what's missing in my default.vcl to get Varnish 3.0 up & running with Drupal 7?

backend default {
  .host = "127.0.0.1";
  .port = "8080";
  .connect_timeout = 600s;
  .first_byte_timeout = 600s;
  .between_bytes_timeout = 600s;
}

 sub vcl_recv {
     if (req.request != "GET" &&
       req.request != "HEAD" &&
       req.request != "PUT" &&
       req.request != "POST" &&
       req.request != "TRACE" &&
       req.request != "OPTIONS" &&
       req.request != "DELETE") {
         /* Non-RFC2616 or CONNECT which is weird. */
         return (pipe);
     }

     if (req.request != "GET" && req.request != "HEAD") {
         /* We only deal with GET and HEAD by default */
         return (pass);
     }

     // Remove has_js and Google Analytics cookies.
     set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(__[a-z]+|has_js)=[^;]*", "");

     if (req.http.Authorization || req.http.Cookie) {
         /* Not cacheable by default */
         return (pass);
     }

    // Remove a ";" prefix, if present.
    set req.http.Cookie = regsub(req.http.Cookie, "^;\s*", "");
    // Remove empty cookies.
    if (req.http.Cookie ~ "^\s*$") {
    unset req.http.Cookie;
    }

   // Skip the Varnish cache for install, update, and cron
   if (req.url ~ "install\.php|update\.php|cron\.php") {
    return (pass);
   }

   // Normalize the Accept-Encoding header
   // as per: http://varnish-cache.org/wiki/FAQ/Compression
   if (req.http.Accept-Encoding) {
    if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
      # No point in compressing these
      remove req.http.Accept-Encoding;
    }
    elsif (req.http.Accept-Encoding ~ "gzip") {
      set req.http.Accept-Encoding = "gzip";
    }
    else {
      # Unknown or deflate algorithm
      remove req.http.Accept-Encoding;
    }
  }

  // Let's have a little grace
  set req.grace = 30s;


     return (lookup);
 }
 
 // Strip any cookies before an image/js/css is inserted into cache.
 sub vcl_fetch {
   if (req.url ~ "\.(png|gif|jpg|swf|css|js)$") {
     unset beresp.http.set-cookie;
   }
 }

  sub vcl_error {
     set obj.http.Content-Type = "text/html; charset=utf-8";
     set obj.http.Retry-After = "5";
     synthetic {"
 <?xml version="1.0" encoding="utf-8"?>
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
  "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
 <html>
   <head>
     <title>"} + obj.status + " " + obj.response + {"</title>
   </head>
   <body>
     <h1>Error "} + obj.status + " " + obj.response + {"</h1>
     <p>"} + obj.response + {"</p>
     <h3>Guru Meditation:</h3>
     <p>XID: "} + req.xid + {"</p>
     <hr>
     <p>Varnish cache server</p>
   </body>
 </html>
 "};
     return (deliver);
 }
steinmb’s picture

This is interesting, getting close to installing Varnish 3 on a Drupal 7 site so, subscribing :)

flatcircle’s picture

Small update.
Seems that with Drupal 7, you also need to include the cookie 'Drupal.toolbar.collapsed' as well.

Otherwise, Varnish won't serve from cache anymore when you've logged in and out of your Drupal site.

set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(__[a-z]+|has_js)=[^;]*", "");

has to be

set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(__[a-z]+|has_js|Drupal.toolbar.collapsed)=[^;]*", "");
_-.’s picture

re #4, i believe that should be,

---	vcl_hash(req.http.Cookie);
+++	hash_data(req.http.Cookie);
Tor Arne Thune’s picture

Also, elsif should be else if, no?

John_B’s picture

I upgraded to Varnish 3 on Centos 5 64bit, using the repositories here http://repo.varnish-cache.org/redhat/varnish-3.0/el5/noarch/, and yum update varnish.

I could not start with the old VCL and now use a new one based on the above thread, taking into account posts #4, #7 and #8, #9 (I think elsif also works though).

It was not caching.

I changed

--- if (req.http.Authorization || req.http.Cookie) {
+++ if (req.http.Authorization) {
/* Not cacheable by default */
return (pass);
}

It is caching great. However, I am not expert enough to know the consequences. I have checked I can still log in to Drupal 6 and Drupal 7 sites on the server, and can edit content.

!!!!! EDIT !!!!!! later I found the sites arriving without css.... still looking for an answer.

waverate’s picture

FileSize
3.29 KB

#10 didn't make a difference for me. I am using:

 sub vcl_deliver {
   if (obj.hits > 0) {
     set resp.http.X-Cache = "HIT";
   } else {
     set resp.http.X-Cache = "MISS";
   }
 }

to display HIT or MISS in the HTTP Header. My pages are still showing up as MISS.

Also, what is the Varnish 3.0 equivalent of beresp.cacheable. I cannot find it in the Varnish Ref. I am using it to try and diagnose why there was a hit or miss (link).

Attached is my .vcl.

Tor Arne Thune’s picture

Replace set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(__[a-z]+|has_js)=[^;]*", "");
with set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(__[a-z]+|has_js|Drupal.toolbar.collapsed|Drupal.tableDrag.showWeight)=[^;]*", "");

This will remove the cookies for logged-out users as well as anonymous users (that have never logged in). That might increase your hits if you have been testing on your own logged-out user (a user that have logged into the site at least once).

John_B’s picture

#12 does not help.I am still getting no hits (having undone the change in #10).

Running varnish with this config has also stripped the css again out of my Wordpress Twentyeleven theme on the same server, which I only recovered after database restore. I cannot see a problem in the database, but there must be one because the css was not loading before a second db restore.

bizarre. I regret upgrading Varnish.... I can get hits with mod in #10, but it too risky for the databases. I have to take it down for now

waverate’s picture

FileSize
3.43 KB

Change at #12 did not help me:

HTTP/1.1 302 Found
Server: Apache
X-Powered-By: PHP/5.3.6
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Last-Modified: Sat, 16 Jul 2011 03:45:49 +0000
Cache-Control: no-cache, must-revalidate, post-check=0, pre-check=0
Etag: "1310787949"
Set-Cookie: SESSffd1b2f837e05a7a248a8010ca1be175=deleted; expires=Fri, 16-Jul-2010 03:45:48 GMT; path=/; domain=.server-3.example.com; httponly
Location: http://server-3.example.com/
Content-Type: text/html; charset=UTF-8
Content-Length: 0
Accept-Ranges: bytes
Date: Sat, 16 Jul 2011 03:45:49 GMT
X-Varnish: 2066837699
Age: 0
Via: 1.1 varnish
Connection: keep-alive
X-Cache: MISS

This is my default.vcl. Using Varnish 3.0, Memcache 1.4.5 and APC 3.1.9.

Tor Arne Thune’s picture

#14: It looks like you have more things in the cookie of your requests. This is a request from a logged-out user?

John_B’s picture

The mod in #10 is a VERY bad idea (my idea). Causes sites to break. But it still gives me a great hit rate on Drupal 6, Drupal 7, and Wordpress, instead of zero hits on all these. This config is letting something though which causes a pass 100% of the time, though the same kind of setup worked for Varnish 2.

The stylesheet problem I had is maybe something to watch for: with this configuration browser requests seem to generating pages with optmized / minified style sheets even where they no longer exist. For now I have turned of off css optimization.

John_B’s picture

Ok I have an answer, tested on Drupal 6 and Drupal 7....working so far...

After the line set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(__[a-z]+|has_js)=[^;]*", "");
(or the amended version of it in #7 or #12)

you need to remove the apparently empty cookie with this

+++if (req.http.Cookie == "") {
+++ remove req.http.Cookie;
+++ }

(taken from https://www.varnish-cache.org/trac/wiki/VCLExampleRemovingSomeCookies)

I have removed the following lines as they do not seem to be adding anything (is that right?:
---// Remove empty cookies.
--- if (req.http.Cookie ~ "^\s*$") {
--- unset req.http.Cookie;
--- }

With my Wordpress site I am getting PHPSESSID when logged out. But that is separate issue, nothing to do with Drupal. The Drupal sites are caching fine.

John_B’s picture

However, I am still having issues with vanishing stylsheets and vanishing images, sometimes resolved by restarting varnish. Maybe this version of Varnish is buggy. For now I will try running Varnish with no caching, while I or someone else works out a good config.

John_B’s picture

FileSize
3.02 KB

I am rebuilding the file from scratch trying a piece at a time to get cache hits (without the sudden failure to find image and css files I was experiencing, which at one point disabled every site on the server).

In the old version the
#remove Google Analytics cookies
part is causing some problems, and the
#remove a ";" prefix
was preventing cache hits. I do not yet know why.

For me that attached file is basically working on Drupal 6 and 7 sites. I look forward to advice / contributions :)

flatcircle’s picture

Maybe the following template(s) come in handy to build a perfect D7-Varnish3-VCL.

https://github.com/mattiasgeniar/varnish-3.0-configuration-templates

flatcircle’s picture

Maybe

if (req.http.Authorization || req.http.Cookie) {
  # Not cacheable by default
  return (pass);
}

Still needs some fixing...

Based on the templates from https://github.com/mattiasgeniar/varnish-3.0-configuration-templates

The .VCL below is almost working. (Varnish 3 + Drupal 7.2).
Varnish is caching perfectly but I can't login with this config.

backend default {
  .host = "127.0.0.1";
  .port = "80";
  .connect_timeout = 600s;
  .first_byte_timeout = 600s;
  .between_bytes_timeout = 600s;
}

sub vcl_recv {

# A configuration file specific for Drupal 7

# Either the admin pages or the login
if (req.url ~ "/admin/?") {
        # Don't cache, pass to backend
        return (pass);
}

# Remove the "has_js" cookie
set req.http.Cookie = regsuball(req.http.Cookie, "has_js=[^;]+(; )?", "");

# Remove the "Drupal.toolbar.collapsed" cookie
set req.http.Cookie = regsuball(req.http.Cookie, "Drupal.toolbar.collapsed=[^;]+(; )?", "");

# Remove any Google Analytics based cookies
set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", "");

# Remove the Quant Capital cookies (added by some plugin, all __qca)
set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(; )?", "");

# Are there cookies left with only spaces or that are empty?
if (req.http.cookie ~ "^ *$") {
        unset req.http.cookie;
}

# Static content unique to the theme can be cached (so no user uploaded images)
if (req.url ~ "^/themes/" && req.url ~ "\.(css|js|png|gif|jp(e)?g)") {
        unset req.http.cookie;
}

# Normalize Accept-Encoding header (straight from the manual: https://www.varnish-cache.org/docs/3.0/tutorial/vary.html)
if (req.http.Accept-Encoding) {
        if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
                # No point in compressing these
                remove req.http.Accept-Encoding;
        } elsif (req.http.Accept-Encoding ~ "gzip") {
                set req.http.Accept-Encoding = "gzip";
        } elsif (req.http.Accept-Encoding ~ "deflate") {
                set req.http.Accept-Encoding = "deflate";
        } else {
                # unkown algorithm
                remove req.http.Accept-Encoding;
        }
}

# Don't cache the install, update or cron files in Drupal
if (req.url ~ "install\.php|update\.php|cron\.php") {
	return (pass);
}

# Uncomment this to trigger the vcl_error() subroutine, which will HTML output you some variables (HTTP 700 = pretty debug)
#error 700;

# Anything else left?
if (!req.http.cookie) {
        unset req.http.cookie;
}


if (req.http.Authorization || req.http.Cookie) {
  # Not cacheable by default
  return (pass);
}

# Try a cache-lookup
return (lookup);

}


sub vcl_fetch {

# For static content related to the theme, strip all backend cookies
if (req.url ~ "^/themes/" && req.url ~ "\.(css|js|png|gif|jp(e?)g)") {
	unset beresp.http.cookie;
}

# A TTL of 30 minutes
	set beresp.ttl = 1800s;
}

sub vcl_error {
    
	set obj.http.Content-Type = "text/html; charset=utf-8";
    set obj.http.Retry-After = "5";
    synthetic {"
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
 "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
  <head>
    <title>"} + obj.status + " " + obj.response + {"</title>
  </head>
  <body>
    <h1>Error "} + obj.status + " " + obj.response + {"</h1>
    <p>"} + obj.response + {"</p>
    <h3>Guru Meditation:</h3>
    <p>XID: "} + req.xid + {"</p>
    <hr>
    <p>Varnish cache server</p>
  </body>
</html>
"};

}

I guess the above configuration still needs tweaking for additional performance.
I notice a huge drop in performance (+/-20%) when using Varnish3 compared to Varnish 2.1x or even 2.0.x.

Lullabot-guys...help us out ;-)

waverate’s picture

FileSize
1.25 KB

I was working on trying to fix #14 and it looks like I had a few issues. The most glaring was that under Site Configuration > Development > Performance, I was not caching pages for anonymous users. Selecting that changed the Cache-Control from

Cache-Control: no-cache, must-revalidate, post-check=0, pre-check=0

to

Cache-Control: public, max-age=21600

in the HTTP header. I can also compress Javascript pages with no issues. I am unable to aggregrate and compress the CSS; so I am working on that now.

The nice thing is that it looks like by configuring grace, one can even turn off the webserver and still server pages. That is a great.

I have attached my new default.vcl which is from ideas from:

Configuring Varnish for High-Availability with Multiple Web Servers" by Lullabot.

I do like their suggestion:

It's important to note that Varnish includes a large set of defaults that are always automatically appended to the rules that you have specified.

Thus a number of lines from my previous default.vcl (#14) were removed.

This is working for me with Varnish 3.0 and Drupal 7.4.

waverate’s picture

I was able to use the default.vcl from #22 with Compress cached pages and Aggregate and compress CSS files. I needed to clear all caches and clear the browser cookies for it to work.

So, at this time, it looks like Varnish is caching nicely requests from anonymous users.

Next to tackle is the passing of traffic to Apache that should not be cached and optimize Varnish caching.

John_B’s picture

Thanks guys.

I downgraded to Varnish 2. Varnish 3 adds nothing for a simple single server installation as far as I can see, and I saw some unpredicatable behaviour (sometimes allowing log in, sometimes not with the same config) which makes me wonder whether it is buggy.

Probably for anyone posting here, the downgrade is a simple process, but in case anyone is struggling I summarised how I did the downgrade on my blog (which is mostly on an subject nothing to do with IT) : http://classicsblog.net/web-technology/downgrade-varnish-3-to-2 .

dtwork’s picture

Following the thread here, I managed to get Varnish 3 + Drupal 7 up & behaving. Mostly (Still can't figure out why after Logout in my browser, I still SEE the logged in page)

ANyway, I've found two forms of vcl_hash. One using 'ref.http.Cookie'

sub vcl_hash {
	if (req.http.Cookie) {
		hash_data(req.http.Cookie);
		return (hash);
	}
	... ( etc ) ...
	return (hash);
}

And another using 'reg.http.host'

sub vcl_hash {
	hash_data(req.url);
	if (req.http.host) {
		hash_data(req.http.host);
	} else {
		hash_data(server.ip);
	}
	... ( etc ) ...
	return (hash);
}

I have no idea which is more correct or just preferred, or whether some combination of the two should be used.

I also saw this snippet

# If the client supports compression, keep that in a different cache
if (req.http.Accept-Encoding) {
	hash_data(req.http.Accept-Encoding);
}

added where the '... (etc) ...' are, above.

Are any of these helpful? Required?

Dan

flatcircle’s picture

I switched back to Varnish 2.1.5 (which also has better performance compared to Varnish 3).

For the moment I don't risk running Varnish 3 and Drupal-sites on a production server.
(There are still too many glitches.)

I guess it's only a matter of time to use Varnish 3.x with Drupal 7.x with a fast & perfectly working configuration. But because I'm not a Varnish-guru myself to write this config-file myself, I'll use Varnish 2.1.5.

luthien’s picture

what vcl file should I use with varnish 3 and Drupal 6 (not pressflow)? can anyone post it here please? thanks.

dtwork’s picture

@ luthien

https://www.varnish-cache.org/trac/wiki/VarnishAndDrupal

"Using Varnish with Drupal prior to version 7 will require the use of Pressflow, which backports Drupal 7's lazy session creation and other important performance enhancement. Without this, Drupal will create a PHP session cookie for all users, meaning Varnish is reduced to caching static files only."

Iiuc, that's referring to Varnish v2, at least. I'd be surprised it it's either supported or recommended to use 'vanilla' Drupal6 with Varnish.

If you're happy using Drupal6, the Pressflow alternative really is a simple drop-in replacement ...

bgilhome’s picture

subscribe

dropbydrop’s picture

New ubuntu 11.10 has varnish 3 as default, so there is no option but to adapt to it.

halcyonCorsair’s picture

I've posted a good basic Drupal 7, Varnish 3 configuration that I'm using to github here:

https://github.com/halcyonCorsair/varnish-for-drupal

capynet’s picture

halcyonCorsair, Worked very well

halcyonCorsair’s picture

Hopefully a few improvements on the way soon as I get time. :)

Mark Theunissen’s picture

halcyonCorsair:

You are matching the url against the list of file extensions twice, which is unnecessary and could cause problems in future. Try limit to one place in which this is done.

In this line:

set req.http.Cookie = regsuball(req.http.Cookie, ";(S{1,2}ESS[a-z0-9]+|NO_CACHE)=", "; \1=");

You've modified the Lullabot version to match SESS or SSESS cookies, why is that?

Then:

// Remove has_js and Google Analytics cookies.
# Remove cookies. has_js, google analytics, drupal related, google ads, piwik, cloudflare-uid
set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(__[a-z]+|has_js)=[^;]*", "");
set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(_pk_(ses|id)[\.a-z0-9]*)=[^;]*", "");
set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(__[a-z]+|Drupal.toolbar.collapsed|Drupal.tableDrag.showWeight|__gads|__cfduid)=[^;]*", "");

These are unnecessary. By the time the code gets here, all other cookies besides SESS and NO_CACHE are gone. To help understand the cookie stripping code, I've commented what it's doing in the 'vanilla' version below:

    # 1. Append a semi-colon to the front of the cookie string. 
    # 2. Remove all spaces that appear after semi-colons.
    # 3. Match the cookies we want to keep, adding the space we removed 
    #    previously, back. (\1) is first matching group in the regsuball.
    # 4. Remove all other cookies, identifying them by the fact that they have
    #    no space after the preceding semi-colon.
    # 5. Remove all spaces and semi-colons from the beginning and end of the 
    #    cookie string. 
    set req.http.Cookie = ";" + req.http.Cookie;
    set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");    
    set req.http.Cookie = regsuball(req.http.Cookie, ";(SESS[a-z0-9]+|NO_CACHE)=", "; \1=");
    set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
    set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");
halcyonCorsair’s picture

Regarding:
set req.http.Cookie = regsuball(req.http.Cookie, ";(S{1,2}ESS[a-z0-9]+|NO_CACHE)=", "; \1=");
It keeps SSESS secure session cookies.

I've pushed some updates to my config. Let me know if anyone finds any issues. :)

nathan.zhu’s picture

Hi, halcyonCorsair

Really thanks provide this varnish configuration for drupal7.

I get problem after use varnish3.x with your vcl files, it not cache anything for my website.

HTTP/1.1 200 OK
Server: Apache/2.2.14 (Ubuntu)
X-Powered-By: PHP/5.3.2-1ubuntu4.14
X-Drupal-Cache: MISS
Last-Modified: Thu, 23 Feb 2012 06:21:36 +0000
Cache-Control: no-cache, must-revalidate, post-check=0, pre-check=0
ETag: "1329978096"
Content-Language: en
X-Generator: Drupal 7 (http://drupal.org)
Vary: Accept-Encoding
Content-Type: text/html; charset=utf-8
Accept-Ranges: bytes
Date: Thu, 23 Feb 2012 06:21:36 GMT
X-Varnish: 98815207
Age: 0
Via: 1.1 varnish
Connection: keep-alive
X-Varnish-Cache: MISS

X-Varnish: only have starttime
X-Varnish-Cache:Miss
I don't know why it cache miss, any sugession about this?
where is wrong for my settings?

nathan.zhu’s picture

it works now, the problem is on drupal settings.php file,
this is my settings.php.

# Varnish
$conf['cache_backends'] = array('sites/all/modules/contribute/varnish/varnish.cache.inc');
$conf['cache_class_cache_page'] = 'VarnishCache';
$conf['page_cache_invoke_hooks'] = False;
$conf['reverse_proxy'] = True;
$conf['cache'] = 1;
$conf['cache_lifetime'] = 0;
$conf['page_cache_maximum_age'] = 21600;
$conf['reverse_proxy_header'] = 'HTTP_X_FORWARDED_FOR';
$conf['reverse_proxy_addresses'] = array('127.0.0.1');
$conf['omit_vary_cookie'] = True;

also have question about vanish secret.
when i add "-S /etc/varnish/secret \" in varnish configure file, like below

DAEMON_OPTS="-a :80 \
             -T localhost:6082 \
             -f /etc/varnish/default.vcl \
            -S /etc/varnish/secret \
             -s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,1G"
             #-s malloc,256m"

drupal module varnish can't get access for varnish system, show can't connect to 127.0.0.1:6082

why? have to setting secret for drupal or others?

halcyonCorsair’s picture

@nathan.zhu

When you install varnish, it should generate a secret file with an authentication key, eg.
845530f0-3521-40db-857f-efe544d5afec

Then you enter that into the configuration for the varnish module so it can authenticate against your varnish server (and prevent unauthorized console access).

atlea’s picture

@halcyonCorsair: Thank you for putting together the vcl. It was a great help when migrating to Varnish 3.0.

But; there is an error in your cookie stripping logic. You have:

    set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");
    set req.http.Cookie = regsuball(req.http.Cookie, ";(S{1,2}ESS[a-z0-9]+|NO_CACHE)=", "; \1=");
    set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
    set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");

where it should be:

    set req.http.Cookie = ";" + req.http.Cookie;
    set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");
    set req.http.Cookie = regsuball(req.http.Cookie, ";(S{1,2}ESS[a-z0-9]+|NO_CACHE)=", "; \1=");
    set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
    set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");

The first line; "set req.http.Cookie = ";" + req.http.Cookie;" is missing. Please refer to the whitelisting example on https://www.varnish-cache.org/trac/wiki/VCLExampleRemovingSomeCookies.

Without this extra line, the first cookie will never be removed.

Atle
http://newreach.no/

halcyonCorsair’s picture

@Atle:

Thanks! Not sure how that slipped through, I'll update shortly.

nathan.zhu’s picture

Hi, halcyonCorsair

thanks for help on secert issue, it connect to varnish with -S secert option.

I facing another strange problem, on varnish cache missing.

this is my test website with d7.12 and varnish 3, get header through firebug.

Accept-Ranges	bytes
Age	0
Cache-Control	public, max-age=21600
Connection	keep-alive
Content-Encoding	gzip
Content-Language	en
Content-Length	3970
Content-Type	text/html; charset=utf-8
Date	Mon, 27 Feb 2012 06:31:22 GMT
Etag	"1330324282-1"
Last-Modified	Mon, 27 Feb 2012 06:31:22 +0000
Server	Apache/2.2.20 (Ubuntu)
Vary	Accept-Encoding
Via	1.1 varnish
X-Drupal-Cache	MISS
X-Generator	Drupal 7 (http://drupal.org)
X-Powered-By	PHP/5.3.6-13ubuntu3.6
X-Varnish	443141558
X-Varnish-Cache	MISS

the Age also is 0, and show X-Varnish-Cache MISS, i'm confuse about it,
do you have any sugession about this? thanks.

atlea’s picture

@nathan.zhu: Header looks good. See #39 for possible solution.

John_B’s picture

Watch out for using a browser where you have logged in and logged out, and which may still be sending auth cookies, and so having requests passed to backend.

halcyonCorsair’s picture

@atlea, @nathan.zhu:
Change from #39 is now committed to my repository.

tmi’s picture

Hi,

many thanks for putting together the configuration halyconCorsair and thanks to the rest for commenting on this topic.

I finally succeeded in getting my site up and running with APC and Varnish. Using the files provided by you (halyconCorsair) I also got the Varnish working.

It seems to be fast but I was not able to figure out how I can check if Varnish is actually operating as designed or not. Is there any chance of checking if content is being delivered from Varnish or not?

My understanding so far was to use varnishlog -b which should show only few if not none entries when I request a page that was accesses previously. In addition I assumed that the X-Drupal-Cache should indicate HIT for a cache hit or MISS for a cache miss.

Unfortunately I always see requests in varnishlog -b even if I repeatedly access the same page (as anonymous user) and also see the X-Drupal-Cache on all HTTP streams in Firebug as MISS.

My questions are now pretty simple and I also hope the answers are:
a) Is the way I "debug" Varnish correct with the varnishlog -b and the X-Drupal-Cache header field?
b) If so, what can I check that could be wrong in my setup when I used the above files plus the varnish module and appropriate drupal settings?

Thanks,
>tom

atlea’s picture

@tmi: Sounds like you are debugin this the right way. Drupal with Varnish module installed is now relying on external cache, so X-Drupal-Cache=MISS is, though confusing, correct. Perhaps the vcl should change this to EXTERNAL? :)

BUT; you should now see a header for X-Varnish-Cache, indicating either a HIT or MISS. This could be MISS on first page load, then should change to HIT for the next.

Here's the important parts in the header:
X-Drupal-Cache: MISS (set to external)
X-Varnish-Cache: MISS or HIT (MISS if cache is cold/expired)
Age: > 0 (Set by Varnish/external cache. Greater than zero = HIT, 0 = MISS)
Cache-Control: public, max-age=nnn (not 0 - this means external cache can look at last modified and set Age, and when Age >= max-age it expires and fetches the content again (MISS))

+ you could look at sane values in Last-modified and date

AND NOW; back yto the X-Varnish-Cache header. If you don't see this, using the default.vcl from halcyonCosair, you probably have not set your ports correctly and the content is still served by Apache.

Assuming you are running Varnish on the same server: You either need to change your Apache port to something other than 80 (eg 8080, and update backends.vcl) and your Varnish port to 80, or set up a prerouting port mapping (dnat, nat, redirect - pick your flavour) from port 80 to 6081 (varnish default port) in iptables. Though it is more common to change the porst, I would recommend port mapping/redirect if you are using an admin tool like plesk/virtualmin/directadmin etc..

I have not tried this, as I run Varnish on a separate server, but this should work:
iptables -D -I PREROUTING -s ! 127.0.0.1 -d YOURIP -p tcp -m tcp --dport 80 -j DNAT --to-destination YOURIP:6081 -t nat

Change YOURIP to your public ip in the iptables rule, and Varnish backend to 127.0.0.1:80 in backends.vcl.

If this works you need to save your iptables. Consult Google for details for your os.

..and good luck. ;)

tmi’s picture

Hi atlea,

many thanks for your answer! If actually mixed up the two header fields. I was looking at the X-Varnish-Cache but wrote X-Drupal-Cache....

I did now try both, setting Apache to 8080 and Varnish to 80 but since I am running on a Plesk based server this ended up unsuccessful...
Now I figured how to do the pre-routing in iptables and the redirection seems to be working. I see the Varnish headers on port 80 although Varnish is on port 6081 and Apache on port 80.

Anyway, I still don't get my sites cached. Even a plain website with just a JPG in it does not get cached in my setup at the moment.

I am using three domains, the main TLD, a dev and a prod subdomain for testing (xxxx.com/ dev.xxxx.com / prod.xxxx.com) and have four aliases enabled in plesk (.de/ .net/ .org).

In order to check that I have the following backend.vcl:

backend dev {

  .host = "dev.xxxx.com";
  .port = "80";
  .connect_timeout = 600s;
  .first_byte_timeout = 600s;
  .between_bytes_timeout = 600s;

}

backend prod {
  .host = "prod.xxxx.com";
  .port = "80";
  .connect_timeout = 600s;
  .first_byte_timeout = 600s;
  .between_bytes_timeout = 600s;
}

backend default {
 .host = "xxxx.com";
  .port = "80";
  .connect_timeout = 600s;
  .first_byte_timeout = 600s;
  .between_bytes_timeout = 600s;
}

And use the following code in the default.vcl to check for the domain that was used to access the sites:

  if ((req.http.host == "xxxx.com") || (req.http.host == "xxxx.de") || (req.http.host == "xxxx.org") || (req.http.host == "xxxx.net")) {
    set req.http.host = "xxxx.com";
    set req.backend = default;
  }
  if ((req.http.host == "dev.xxxx.com") || (req.http.host == "dev.xxxx.de") || (req.http.host == "dev.xxxx.org") || (req.http.host == "dev.xxxx.net")) {
    set req.http.host = "dev.xxxx.com";
    set req.backend = dev;
  }
  if ((req.http.host == "prod.xxxx.com") || (req.http.host == "prod.xxxx.de") || (req.http.host == "prod.xxxx.org") || (req.http.host == "prod.xxxx.net")) {
    set req.http.host = "prod.xxxx.com";
    set req.backend = prod;
  }

The rest of the files is using halyconCorsairs configuration which also looks pretty fine for me.

But I still only get misses on the Caching. I tried to just use a very plain VCL file that worked. There must be either still an issue in my config part above or in halyconCorsairs config together with my part.

Does anyone have an idea?

Thanks!
>tom

halcyonCorsair’s picture

@tmi:

Can you post your browser request & response headers for a page that isn't getting cached?

Also, when a page is returned by varnish, you shouldn't be seeing an X-Drupal-Cache header, eg. a cache page gets something like:

Request Headers

Accept-Encoding: gzip, deflate
Accept-Language: en-gb
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/534.52.7 (KHTML, like Gecko) Version/5.1.2 Safari/534.52.7
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Cache-Control: max-age=0

Response Headers

Response Headers
X-Varnish-Cache: HIT
Content-Language: en
Date: Fri, 02 Mar 2012 09:02:46 GMT
Content-Encoding: gzip
Age: 1067
Connection: close
Content-Length: 10537
Via: 1.1 varnish
Last-Modified: Fri, 02 Mar 2012 08:44:57 +0000
Server: nginx
Etag: "1330677897-1"
X-Frame-Options: sameorigin, sameorigin
X-Varnish: 740819579 740817838
Cache-Control: public, max-age=3600
Vary: Accept-Encoding
Content-Type: text/html; charset=utf-8
Accept-Ranges: bytes
atlea’s picture

@tom: If it works with a plain vlc file, make sure you have the very latest vlc from halyonCorsair git.

Please post your header, as I mentioned max-age in Cache-Control is important. If this is higher than zero Drupal (this module) is set up correctly.

To debug further, try adding this to your vcl_fetch {} subrutine:

  if (beresp.ttl <= 0s) {
    set beresp.http.X-Varnish-Cacheable = "NO:Not cacheable";
 else {
    set beresp.http.X-Varnish-Cacheable = "YES";
  }

and you could always add som temporary debug logging:

- add to start of default.vlc:
import std;

- add before return (pass) e.g. in the cookie check and similar places:
std.log("Some debug message");

Then you restart varnish and start varnishlog -i VCL_Log

Hope you get this sorted! :)

Atle
http://newreach.no/

tmi’s picture

Hi,

thanks for your answers. By using the very good hint with the std.log I apparently found the issue... I am using htaccess basic authentication to protect my domains from external access as long as they are not productive. These codes lines here caused the issue:

  ## Unset Authorization header if it has the correct details...
  #if (req.http.Authorization == "Basic <hash>") {
  # unset req.http.Authorization;
  #}
  if (req.http.Authorization || req.http.Cookie) {
      /* Not cacheable by default */
      return (pass);
  }

When I removed the htaccess from the site, everything worked perfectly.

I did now try with uncommenting the unsetting of the field, but without success. Could somebody explain to me why requests with an authorization header should be passed and not looked up?

I did for now remove this check from the second IF to make things work.

Thanks!

>tom

tmi’s picture

Ok - that's not working either. Apparently something is wrong with the authorization that I don't understand. Could somebody help out here to get this working? That would be great!

Mark Theunissen’s picture

Varnish doesn't cache requests with Auth headers - it's incompatible with this - because otherwise it could cache pages that require auth, and continue serving them even after you change the password on the webserver side. That's just one problem.

halcyonCorsair’s picture

@tmi:

If you are using .htaccess authorisation but only for access control to the site (eg. not for logging in to Drupal), you can have varnish pass through people who have the correct authorization header, eg:

Request Header:

Authorization: Basic bGF4J29tMmJldPRlcmxhdw==

Varnish config:

  # Unset Authorization header if it has the correct details...
  if (req.http.Authorization == "Basic bGF4J29tMmJldPRlcmxhdw==") {
   unset req.http.Authorization;
  }
Mark Theunissen’s picture

That doesn't work - because then you have to disable Auth in Apache ... because you're unsetting the header so Apache won't see it so won't be able to auth the user.

halcyonCorsair’s picture

@mark, @tmi:

Sorry, I should have been clear, the reason it work for me was:
nginx (ssl & htaccess) -> varnish -> drupal

Obviously it won't work if the authorization is requested behind varnish.

ndeschildre’s picture

Since http://www.lullabot.com/articles/varnish-multiple-web-servers-drupal is referenced in this thread, a word of warning on their status.php: it is buggy.

if (!$account->uid == 1) {

is bad: !$x == 1 is not $x != 1

if ($test = tempnam(variable_get('file_directory_path', conf_path() .'/files'), 'status_check_')) {

is bad: variable_get is not initialized at this stage of the boostrap (DRUPAL_BOOTSTRAP_DATABASE), you'll get conf_path() .'/files' in all cases.

Here are 2 working Varnish status file for Drupal, without the bugs: D6: http://pastebin.com/QUFWdQqr D7: http://pastebin.com/yqeUjFwG

SilviuChingaru’s picture

FileSize
8.44 KB
152 bytes
83 bytes

I have the attached config files but I cannot make varnish 3.2 work. Please help. I tested using 2 browsers chrome & firefox clear all cookies and I tried loading a page from one browser and got X-Varnish-Cache:MISS then I tried to load the same page from other browser and the response header was also X-Varnish-Cache:MISS.
Obviously something is not working as it should be but I cannot solve this. I'm not to good on varnish setup...

Headers from first browser (chrome):
Requests Headers

Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3
Accept-Encoding:gzip,deflate,sdch
Accept-Language:ro,en-US;q=0.8,en;q=0.6
Connection:keep-alive
Cookie:has_js=1; __utma=119826830.425794287.1333979910.1333979910.1333979910.1; __utmb=119826830.23.10.1333979910; __utmc=119826830; __utmz=119826830.1333979910.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none)
Host:www.example.com
Referer:http://www.example.com/node/234
User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.151 Safari/535.19

Response Headers

Accept-Ranges:bytes
Age:0
Cache-Control:public, max-age=21600
Connection:keep-alive
Content-Encoding:gzip
Content-Language:ro
Content-Length:5432
Content-Type:text/html; charset=utf-8
Date:Mon, 09 Apr 2012 14:57:23 GMT
Last-Modified:Mon, 09 Apr 2012 14:57:21 +0000
Link:<http://www.example.com>; rel="canonical"
Server:Apache/2.2.15 (CentOS)
Vary:Accept-Encoding,User-Agent
Via:1.1 varnish
X-Drupal-Cache:MISS
X-Generator:www.example.com (http://www.example.com)
X-Powered-By:PHP/5.3.3
X-Varnish:1500648663
X-Varnish-Cache:MISS

Headers from second browser (firefox):
Request Headers

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip, deflate
Accept-Language: en-us,en;q=0.5
Connection: keep-alive
Cookie: has_js=1; __utma=119826830.1144458027.1333979921.1333979921.1333982030.2; __utmc=119826830; __utmz=119826830.1333979921.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utmb=119826830.6.10.1333982030
Host	: www.example.com
If-None-Match	"1333929163-0"
Referer: http://www.example.com/node/234
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:11.0) Gecko/20100101 Firefox/11.0

Response Headers

Accept-Ranges: bytes
Age: 0
Cache-Control: public, max-age=21600
Connection: keep-alive
Content-Encoding: gzip
Content-Language: ro
Content-Length: 5433
Content-Type: text/html; charset=utf-8
Date: Mon, 09 Apr 2012 14:57:15 GMT
Last-Modified: Mon, 09 Apr 2012 14:57:13 +0000
Link: <http://www.example.com>; rel="canonical"
Server: Apache/2.2.15 (CentOS)
Vary: Accept-Encoding,User-Agent
Via: 1.1 varnish
X-Drupal-Cache: MISS
X-Generator: www.example.com (http://www.example.com)
X-Powered-By: PHP/5.3.3
X-Varnish: 1500648662
X-Varnish-Cache: MISS

Please help me to make Varnish 3.2 and Drupal 7.12 work. I have also installed varnish module. Thank you very much.

quicksketch’s picture

Just a note that I updated the Lullabot article today for Varnish 3, since the syntax changed slightly from Varnish 2.

http://lullabot.com/articles/varnish-multiple-web-servers-drupal

SilviuChingaru’s picture

Following this https://www.varnish-cache.org/trac/wiki/VCLExampleHitMissHeader I've got the header:

...
X-Cacheable: NO:Not Cacheable
...

so Varnish determined the object was not cacheable based on object's TTL.

I think the problem is from drupal/apache cacheble header, but I don't know what could cause this 0 ttl for anonymous pages. Pls help...

It seems that when I load a page in a browser and then press Ctrl+F5 in the same browser I get a HIT but if I load the same page in other browser I get a MISS. I can't figure out why the same page is not cached in browsers both with gzip support?!

I found out that the same page in same browser (tested with chrome) but from different computers I get a MISS so certainly is not working as it should.

I also made a test.txt file in the root directory and I got a MISS on every first load in different browser. On second load I got a HIT.

Mark Theunissen’s picture

You've answered your question... it's because the browsers are sending different headers.

Try debug in your VCL. Try manually forcing different headers to be the same and see if that makes a difference.

SilviuChingaru’s picture

The problem was in fact from apache. Following the quide from apache docs located here: http://httpd.apache.org/docs/2.2/mod/mod_deflate.html I have enabled the deflate module but those instructions are incompatible with varnish because the Very was set:
WRONG:

Vary:Accept-Encoding, User-Agent

and should be set according to

Vary:Accept-Encoding

So when I comment out the last line of apache config mod_deflate:

# Header append Vary User-Agent env=!dont-vary

all is working as it should.

I think this should be putted in readme as special care or something. It took me 1 week to figure it out.

SilviuChingaru’s picture

By the way from varnish docs https://www.varnish-cache.org/trac/wiki/VCLExampleNormalizeAcceptEncoding

Note that in varnish 3.0, varnish can handles gzip by itself, so this is not needed.

So the following lines from default.vcl should be removed:

  # Handle compression correctly. Different browsers send different
  # "Accept-Encoding" headers, even though they mostly all support the same
  # compression mechanisms. By consolidating these compression headers into
  # a consistent format, we can reduce the size of the cache and get more hits.=
  # @see: http:// varnish.projects.linpro.no/wiki/FAQ/Compression
  if (req.http.Accept-Encoding) {
    if (req.url ~ "(?i)\.(png|gif|jpeg|jpg|gz|tgz|bz2|tbz|mp3|ogg)$") {
      # No point in compressing these
      remove req.http.Accept-Encoding;
    }
    else if (req.http.Accept-Encoding ~ "gzip") {
      # If the browser supports it, we'll use gzip.
      set req.http.Accept-Encoding = "gzip";
    }
    else if (req.http.Accept-Encoding ~ "deflate") {
      # Next, try deflate if it is supported.
      set req.http.Accept-Encoding = "deflate";
    }
    else {
      # Unknown algorithm. Remove it and send unencoded.
      remove req.http.Accept-Encoding;
    }
  }

Am I right?!?!

SilviuChingaru’s picture

Another issue I have:
Now varnish caches all anonymous pages as it should and I get HIT right, but when a user navigate to pages as anonymous so the browser caches those pages and then the user log in to the site and visit the same pages, no request is sent to the network, and the browser serves the cached page without requesting it to varnish and varnish doesn't have the chance to send it to apache because the user is authenticated.

My headers sent to the browser are:

Response Headers:

Accept-Ranges:bytes
Age:334
Cache-Control:public, max-age=21600
Connection:keep-alive
Content-Encoding:gzip
Content-Language:ro
Content-Length:12380
Content-Type:text/html; charset=utf-8
Date:Tue, 10 Apr 2012 18:10:20 GMT
ETag:"1334081083-0"
Last-Modified:Tue, 10 Apr 2012 18:04:43 +0000
Link:<http://www.example.org/>; rel="canonical"
Server:Apache/2.2.15 (CentOS)
Vary:Accept-Encoding
Via:1.1 varnish
X-Cacheable:YES
X-Drupal-Cache:MISS
X-Generator:www.example.org (http://www.example.org)
X-Powered-By:PHP/5.3.3
X-Varnish:767160618 767160117
X-Varnish-Cache:HIT

I think the problem could be from line:

Cache-Control:public, max-age=21600

that it is sent to the browser. Should I change this line and rewrite it in varnish with no-cache, must-revalidate or something? Thank you for any help.

EDIT: My solution was to rewrite the Cache-Control sent to the browser as follow:

sub vcl_deliver {
  ...
  if (!(req.url ~ "(?i)\.(png|gif|jpeg|jpg|ico|swf|css|js)(\?[a-z0-9]+)?$")) {
    set resp.http.Cache-Control = "no-cache, must-revalidate, post-check=0, pre-check=0";
  }
  ...
}

Is working but I don't know if is the right way.

Now the headers sent to the browser look like this:

Accept-Ranges:bytes
Age:1926
Cache-Control:no-cache, must-revalidate, post-check=0, pre-check=0
Connection:keep-alive
Content-Encoding:gzip
Content-Language:ro
Content-Length:7748
Content-Type:text/html; charset=utf-8
Date:Tue, 10 Apr 2012 18:36:47 GMT
ETag:"1334081080-0"
Last-Modified:Tue, 10 Apr 2012 18:04:40 +0000
Link:<http://www.example.org>; rel="canonical"
Server:Apache/2.2.15 (CentOS)
Vary:Accept-Encoding
Via:1.1 varnish
X-Cacheable:YES
X-Drupal-Cache:MISS
X-Generator:www.example.org (http://www.example.org)
X-Powered-By:PHP/5.3.3
X-Varnish:767161864 767160059
X-Varnish-Cache:HIT
Mark Theunissen’s picture

I think you're right about #62. Looks like that behavior is now part of the default VCL:

https://www.varnish-cache.org/docs/trunk/tutorial/compression.html
https://www.varnish-cache.org/docs/trunk/phk/gzip.html#phk-gzip

Mark Theunissen’s picture

Regarding #63, your Vary is supposed to be:

Vary Cookie, Accept-Encoding

The browser will then realize that you have a cookie (when you login) and will re-request the page.

SilviuChingaru’s picture

And how can I do this? From apache conf file?

Mark Theunissen’s picture

You must be overriding it somehow, because it's default Drupal behavior. So I'm not really sure where that might be...

Lukas von Blarer’s picture

I am installing Varnish on my webserver for multiple D7 sites. would you suggest to use the Lullabot VCL or the one of halcyonCorsair?

Mark Theunissen’s picture

We maintain one on the Four Kitchens wiki, too. It incorporates the best practices here from Lullabot et al.

https://fourkitchens.atlassian.net/wiki/display/TECH/Configure+Varnish+3...

This version is in production - currently powers ICANN.org.

I would suggest reading all of them and picking the parts you need.

Lukas von Blarer’s picture

Thank you, I will look into it!

It would be nice to have some kind of official Drupal base VCL that can be adapted to what ever you need.

gilzero’s picture

Thanks Mark Theunissen for the link, #69

Sub.

nerdcore’s picture

#22 indicates that setting "Cache Pages for Anonymous Users" changed

Cache-Control: no-cache, must-revalidate, post-check=0, pre-check=0

to

Cache-Control: public, max-age=21600

but this has only worked on certain pages for me, i.e. the front page. Other pages are still sending the top of the above two headers. I would very much like to get these additional pages cached in Varnish.

What is causing this header to be set the way it is, and where can I look to change it aside from the Performance -> Cache Pages for Anonymous setting?

nerdcore’s picture

I believe my issues here are related to #1829312: Enabling Global: PHP field in views breaks Varnish

alifspb’s picture

127.0.0.1:6082 with secret key is working on me thanks :)) thanks good after 8 hour :((

MiSc’s picture

Issue summary: View changes
Status: Needs review » Closed (outdated)

Closing issues that had no activity the latest year, please re-open if the issues is still relevant.