Logged in user get cached page instead non-cached

Its a browser cache issue. Throw this function in somewhere in your custom site module and call it from MY_MODULE_node_view(). Swap out the filter logic to whatever is appropriate for your site.

function MY_MODULE_page_no_cache() {
  drupal_add_http_header('cache-control', 'no-cache, no-store, must-revalidate');
}

function MY_MODULE_node_view($node, $view_mode, $langcode) {
  // Don't let the browser think the front page is cacheable.
  if (drupal_is_front_page()) {
    MY_MODULE_page_no_cache();
  }
}

What you've desribed in #3 as what you want to happen is exactly how Four Kitchen's drupal vcl should work. Authenticated users don't get anything but CSS, images & javascript cached by Varnish per that config. It works this way because the varnish vcl strips off all cookies for css, image and javascript. Anonymous users should get everything cached by varnish because they don't have the cookie in the first place.

The page_no_cache function from #2 tells the browser that it needs to re-fetch the requested content from the remote server. This request will get sent to Varnish and it is up to it to determine if it should serve up the request from its cache or pass it along to the back-end. I add this function on the homepage so that once a user becomes "authenticated" and we redirect them to the homepage, the browser knows it needs to call off to varnish again. This time, remember we are authenticated now, we have our SESS cookie and the homepage renders differently. If we didn't call the above mentioned function on the homepage, then users of the site have to hit F5 or refresh the homepage to see all the new content.

But this actually prevents homepage (or any page put in the logic) to be Varnish-ed even for anonymous users. (tested it)-

The page_no_cache function will not cause varnish not to cache anything. If varnish isn't caching for anonymous users, then there's something wrong with your vcl. The most common cause for non-caching is when someone messes with the cookie logic. Its there for a purpose and shouldn't be altered unless you have a good reason.

  # Remove all cookies that Drupal doesn't need to know about. We explicitly 
  # list the ones that Drupal does need, the SESS and NO_CACHE. If, after 
  # running this code we find that either of these two cookies remains, we 
  # will pass as the page cannot be cached.
  if (req.http.Cookie) {
    # 1. Append a semi-colon to the front of the cookie string.
    # 2. Remove all spaces that appear after semi-colons.
    # 3. Match the cookies we want to keep, adding the space we removed 
    #    previously back. (\1) is first matching group in the regsuball.
    # 4. Remove all other cookies, identifying them by the fact that they have
    #    no space after the preceding semi-colon.
    # 5. Remove all spaces and semi-colons from the beginning and end of the 
    #    cookie string. 
    set req.http.Cookie = ";" + req.http.Cookie;
    set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");    
    set req.http.Cookie = regsuball(req.http.Cookie, ";(SESS[a-z0-9]+|NO_CACHE|location)=", "; \1=");
    set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
    set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");

The addition of
set req.http.Cookie = regsuball(req.http.Cookie, ";(SESS[a-z0-9]+|fbs_[a-z0-9]+|NO_CACHE)=", "; \1=");
effectively breaks varnish caching. Remove it from your vcl and caching will start working again. Varnish doesn't work very well with authenticated users. Which is what you are trying to do when you tell it to accept the facebook cookie. You aren't authenticated to drupal, but you are to facebook. However, its the same difference to varnish.

Try setting $conf['omit_vary_cookie'] = FALSE in your settings.php file.

Has anyone fixed this problem? Having exactly the same problem. Tried so many different things, nothing helps.

The problem I'm having right now is that I'm trying to use annonymous caching with varnish.

That's the current problem:

User visits some pages such as Home, News, Forum.
User logs in
User visits these pages again (Home, News, Forum)
And when he visits these pages, they're all cached to the clients local browser. Here's the header:

Request URL:http://terrangaming.com/
Request Method:GET
Status Code:200 OK (from cache)

--

What can I do to prevent this from happening? www.terrangaming.com
I've already made these changes to my settings.php but doesn't seem to help:

$conf['cache_backends'] = array('sites/all/modules/varnish/varnish.cache.inc');
$conf['cache_class_cache_page'] = 'VarnishCache';
$conf['reverse_proxy'] = TRUE;
$conf['page_cache_invoke_hooks'] = FALSE;
$conf['cache'] = 1;
$conf['cache_lifetime'] = 0;
$conf['reverse_proxy_addresses'] = array('127.0.0.1');
$conf['omit_vary_cookie'] = FALSE;

Any tips would be great. Thanks in advance.

ifish, have you tried #1 (further expounded in #4)?

No, the browser still stores it locally

Request URL:http://terrangaming.com/
Request Method:GET
Status Code:200 OK (from cache)

Tried custom module:

function terran_page_no_cache() {
  drupal_add_http_header('cache-control', 'no-cache, no-store, must-revalidate');
}
function terran_node_view($node, $view_mode, $langcode) {
    terran_page_no_cache();
}

Ive just thought about meta tags solution. Does anyone know how caching can be disabled with meta tags in drupal?

I don't see 'no-cache, no-store, must-revalidate' cache-control headers when I access http://terrangaming.com/. Have you flushed cache?

That's odd, just incase I cleared caches and even ran cron for 3 times. It doesn't seem to work. I'm not familiar with drupal API at all, is the PHP code correct?

Finally got it to working, atleast seems so. The fix was this to modify htaccess if someone is interested:

<FilesMatch \.php$>
    FileETag None
    <IfModule mod_headers.c>
        Header unset ETag
        Header set Cache-Control "max-age=0"
        Header set Pragma "no-cache"
        Header set Expires "Wed, 11 Jan 2012 05:00:00 GMT"
    </IfModule>
  </FilesMatch>

Fix:

<IfModule mod_headers.c>      
        Header set Cache-Control "no-cache, no-store, must-revalidate"
        Header set Pragma "no-cache"
        Header set Expires "0"
 </IfModule>

@ifish Wish I could say that fixed my issue as well, but I've run into other issues along with not fixing the original issue... Authenticated pages are still being cached as anonymous and if I reload the browser cache, Varnish seems to be caching the authenticated page for anonymous.

Can't win :-/

On a side note, I decided to drop my cache TTL from 300 seconds down to (what came as the default) 20 seconds and my problem might be gone... Time will tell. (dropped all of the .htaccess settings proposed by @ifish and running the normal .htaccess config)

Can disable browser cache in varnish config, and only allow for certain items to be cached by browser:

sub vcl_fetch{
  set beresp.http.cache-control = "max-age=0";
  # Allow browser caching for images, js, and css.
  if (req.url ~ "(?i)\.(png|gif|jpeg|jpg|ico|swf|css|js)(\?[a-z0-9]+)?$") {
        set beresp.http.cache-control = "max-age=300";
  }
  ... other stuff ...
}

There is a note in settings.php that says:

* Page caching:
 *
 * By default, Drupal sends a "Vary: Cookie" HTTP header for anonymous page
 * views. This tells a HTTP proxy that it may return a page from its local
 * cache without contacting the web server, if the user sends the same Cookie
 * header as the user who originally requested the cached page. Without "Vary:
 * Cookie", authenticated users would also be served the anonymous page from
 * the cache. If the site has mostly anonymous users except a few known
 * editors/administrators, the Vary header can be omitted. This allows for
 * better caching in HTTP proxies (including reverse proxies), i.e. even if
 * clients send different cookies, they still get content served from the cache.
 * However, authenticated users should access the site directly (i.e. not use an
 * HTTP proxy, and bypass the reverse proxy if one is used) in order to avoid
 * getting cached pages from the proxy.
 */

Most guides suggest adding $conf['omit_vary_cookie'] = TRUE; but have you tried with it set to false?

EDIT: OOPS, just re-read and saw comment #9