Pages are not cached / not cacheable / caching won't work - except when cookies are disabled

Boost only works when anonymous users surf the site, so if you are logged in then you would have a DRUPAL_UID cookie which would stop boost working. The wget is interesting though since wget shouldn't be logged in. It sounds rather like you have a module sending a DRUPAL_UID to anonymous users (I use wget on a cron job every day to freshen the cache), use the contact form by clicking on my name as I'd like to see the site and URL just to check a few things.

Thank you
Philip.

I've had a look at the site, in particular the cookie being set. The rewrite rules for boost look for a cookie DRUPAL_UID which is not set by an anonymous user in my testing. The interesting thing is that whichever page you go to first, is cached, click on any link though and the site attempts to establish a session cookie which is not normal drupal behaviour and testing on several personal sites with boost installed, (7.x branch), it's not boost.

I suspect you have another module that is establishing the session and triggering the code segment that you found above. (well done). I'd recommend going through your modules and trying to work out which one it is, I've been using lynx to test as it's very obvious when the session cookie pops up. I've also chosen different entry points into the website and the same thing happens e.g. entering through the contact link and then going elsewhere, the second page always requests a session cookie.

I've ruled out the comment and tracker modules in core. I would initially rule out php.ini and session.auto_start (simple phpinfo should show that up) and then also look for any hooks. You should have

php_flag session.auto_start off

in your root .htaccess file by default anyway. I'm closing this as it appears to not be boost setting this session.

Sorry, stepped away for a bit, but see that you've found the problem. The general overall picture is that we can only work within the scope of Drupal's login mechanism and if someone wants to add a module to do something custom, then they should use a differing cookie so that anonymous users are not assigned a login id, otherwise basically boost cannot work and also it does somewhat break the strict Drupal separation between anonymous and authenticated users which may also be a security issue with assigned roles.

I would submit a bug report to the module authors, pointing the above out.

Drupal is unlikely to set a session cookie (which would be DRUPAL_UID) and this means that you have an errant module that is using Drupal Core sessions, (not allowed) to implement some kind of function or block. You need to track down the module and then take it up with whomever developed the module, they need to set their own cookie specific to their module.

This does not sound like the same problem. It sounds more like a permissions problem if you are getting internal server error. Please open another bug and post more details of the errors you are receiving since this thread is closed.

What would probably also hep is pointing out to the module maintainers that the login cookie is only for drupal's core use and should either be removed or set to a valid value (the id of the user), if they need a session function then they should use their own cookie. Even setting Drupal_UID=0 or -1 is not permitted but affects both the core drupal cache and disables boost.

Boost is only disabled if DRUPAL_UID is set, therefore if you just have a session then there is something incorrect with your configuration. There is a difference between a Drupal Core session (DRUPAL_UID) and a php session (which you have), which is probably not explained very well above. the .htaccess rules only work if DRUPAL_UID is not set to any value (even 0, -1, null or false), the cookie must not exist..

message me privately with the site's location as I would like to see this as it is extremely unlikely that the session cookie is relevant, since DRUPAL_UID is hard coded in to the .htaccess file.

well the good news is that the .htaccess rules are working, the bad news is that for some reason the main boost functions that should only be disable by DRUPAL_UID are being disabled by the PHP session cookie. I can confirm this as deleting the session cookie, hitting reload, creates the cache and then .htaccess serves the page for an hour and the folder permissions must be set correctly as I can visit pages such as http://example.com/cache/normal/exmple.com/personal_.html correctly if I type the url into the browser.

What I believe is happening is that looking at boost.module there is a line

|| !empty($_SESSION['messages']) // do not cache pages with messages

and I strongly suspect that your template may be setting a message (whether it be null or not). On this side of the server this would be impossible to see but you should be able to place a little PHP into a page to display it and then modify the boost.module if statement to ignore the value.

Therein lies the problem there is nothing in boost that should be affected by a Session cookie, except for the "MESSAGE' section

 if (   strpos($_SERVER['SCRIPT_FILENAME'], 'index.php') === FALSE
      || $_SERVER['SERVER_SOFTWARE'] === 'PHP CLI'
      || ($_SERVER['REQUEST_METHOD'] != 'GET' && $_SERVER['REQUEST_METHOD'] != 'HEAD')
      || isset($_GET['nocache'])
      || variable_get('maintenance_mode', 0)
      || defined('MAINTENANCE_MODE')
      || !empty($_SESSION['messages']) // do not cache pages with messages
  ) {
    $_boost['cache_this'] = FALSE;
  }
  else {
    // More advanced checks
    $_boost = boost_transform_url();
    if (empty($_boost['menu_item']['status']) || $_boost['menu_item']['status'] != 200) {
      $_boost['cache_this'] = FALSE;
    }
  }

where you have just removed the MESSAGES sections I would suggest checking the values of $_boost['cache_this'] and $_boost['is_cacheable'] before the block and seeing if you can backtrace the trigger, I would be very interested to see what the trigger is as I can certainly see that the cache is being created and served at least once, there isn't anything in your .htaccess that could cause a skipping of the rules based on the existence of a cookie rather than the specific DRUPAL_UID cookie ?

cache_this is only ever set to FALSE, I don't think that that is the problem though. The code block is the main block in PHP which corresponds to the .htaccess rules so it would be logical to assume that the session cookie was triggering something in there and it would be helpful to know what (except that it appears that the rules are passed).

It is normal to need to refresh the page to get the cached version, first time it hits the page and saves it out, second time .htaccess takes over and pass the html back. In your case it appears that the session cookie needs to be deleted for all subsequent pages to be generated and I can't see from this side what is triggering this behaviour. Taking the front page as an example, if deleted, if it always generated ? I've just generated it myself, then after it is presented, then does going to /personal also generate the page ? (you'll have to remove the pages from the cache) or does the existence of the session cookie stop pages being written out, so one would

delete cached pages
go to index/ home
check if cache has been written out,
confirm that a session cookie has been set
visit /personal to see if a cached page gets written or whether it requires the deletion of the session cookie to be written.

something that might be worth trying it

setting session.auto_start=0

or php_value session.auto_start 0 in the top of .htaccess (it might be php_flag session.auto_start 0 ) it may solve the immediate problem but not the long term one. I'm pretty sure it's one of the page generation functions, so I'd be now looking to put that code above into the function boost_exit and then working backwards to see what triggers the flags to be set to false.

Well the reason that boost uses the drupal_page_is_cacheable function is that if a DRUPAL_UID is set e.g. a non anonymous user is present then boost disables, also if something were to go wrong even on the anonymous side and a message sent by another module, then that page should be flagged a not cacheable so that it didn't hang around and present potentially compromising information to non-anonymous people. I agree something has not been very well though out, since having seen the if clause where $_SESSION['messages'] was set, I did think that it was a little off when really there should be a drupal core mechanism for $_COOKIE['DRUPAL_UID']['messages'] which would then also be linked in to the roles side of thing so that the correct people got the message too.

This theme you have, I would suggest having a look through and seeing if there are any session functions in it before disabling boost or a core function. Also a small concern has popped up, I've looked at your site for a few days now with firebug on and I cannot recall having ever seen a http 401 header which suggest a flaw in firebug because authentication is certainly requested and I was wondering if that triggered the session cookie so it might the worth temporarily turning that off just to check.

I agree with you about that five year old article and of ask myself why it will only be fixed in drupal 8, when in all likelihood boost will not exist in drupal 8 with the change over to the symfony framework and it's inbuilt file based caching, I've yet to see any concrete documentation but my feeling is very much that boost will become obsolete.

print_r($SESSION) would probably help to find out what it's got inside, or with shell access then

grep -i session_  `find sites/all/themes -type f`

if the themes is setting the cookie, that should pull any PHP functions related to session handling

I feel it would be safer to add an extra statement to the core code below

if (!empty($user->uid) || !empty($_SESSION)) {
      drupal_page_is_cacheable(FALSE);
    }

like

if (!empty($user->uid) || (!empty($_SESSION && !empty($_SESSION['time'] && ...)

and hope that it has no negative impact on the theme itself. What concerns me is that boost is specifically designed not cache those private pages where information could be revealed like login.php and I would prefer to rely on the core mechanisms.

Regarding the 401 section, I just wondered what happened if you briefly turned off the Basic Authentication in the .htaccess file, I think this is a last chance hope, but some versions of apache have been known to do odd things with rewrite rules due to a bug that was later fixed but the original bug is still floating around in some distributions even with updates and the boost rules state that the response should be HTTP 200 so I was wondering if the initial HTTP 401 when one enters the password could in theory still be passed to the .htaccess file a second time and therefore skip the rules (I have no evidence of this but find it odd that I have seen no 401 headers when authenticating in firebug).

I've thought very hard about this and my advice is strongly "use another caching system with that template" do not alter any code. I believe that the underlying reason for the session is to make the website appear differently to tablet/ mobile users instead of having a responsive template that adjusts the css automatically. What concerns me is that user A visits the site using a mobile, user B visits the website using a computer, user B gets to see the version of the site that is mobile and requires touch screen navigation because boost has cached it.

I believe that the purchase of the template was not thought through by someone conversant with drupal and boost. The only thing that may be an alternative avenue is to rewrite a large section of the htaccess code to store different caches based on the values stored in the session which would have to be pulled out in PHP and set as a cookie, then boost would need major modification to remove stale cached files in the different cache locations. It is not an easy answer, and I'm sorry.