The Boost module was all working very well, until i installed the new drupal core update to 7.20 ten days ago. Ever since then, the cache files are getting created properly, but the cached pages are not being served. Also the .htaccess file (in the cache > normal > website directory gets recreated each day the first time i go to any page on my site. (not on each cron run, but once per day, first time I visit) The cache gets created on first anonymous page visit, but then the cached page is not served the next time I visit. If i go there from a different browser, the cached page gets re-created (w/ new timestamp), but then is not served the next time I visit.

I've tried changing the .htaccess settings, and re-generated and changed out my BOOST code in the root .htaccess file. All to no avail...

I've flushed cache and disabled/re-enabled boost.

These are the last five lines in the .htaccess file in my cache directory:
SetHandler Drupal_Security_Do_Not_Remove_See_SA_2006_006
Options None
# For security reasons, Option followsymlinks cannot be overridden.
#Options +FollowSymLinks
Options +SymLinksIfOwnerMatch

Any ideas?
Thanks, Jean

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Anonymous’s picture

Which version of boost are you using ?

Options +SymLinksIfOwnerMatch

can be set in the configuration for boost, so a couple of those lines don't seem necessary. If the files are being created in your cache but not being served, then it suggests that either the root .htaccess file is incorrect or that the boost is flushing the pages on every visit, which would be odd but explains a little the recreation of the cache .htaccess file. Have you checked that your normal drupal page caching is disabled ? I updated to 7.20 and 7.21 and have had no problems but I'm running the dev version of boost.

jmatuska’s picture

7.x-1.0-beta2 is the boost version.
normal drupal page caching IS disabled.

where can i set that: "Options +SymLinksIfOwnerMatch" setting in the boost config?
and then, i should remove it from the cache .htaccess file?
what are the other lines that are unnecessary in that .htaccess file (my last 5 lines)

all was working brilliantly up to 10 days ago, so i know my root .htaccess file was just fine. Since it stopped working i have tweaked the boost htaccess code by changing those boost config settings numerous times/options to see if that would help.

what might cause boost to flush the cached pages on every visit (that you said explains the recreation of the htaccess file?)

my root .htaccess file looks like this:

#
# Apache/PHP/Drupal settings:
#

# Protect files and directories from prying eyes.

Order allow,deny

# Don't show directory listings for URLs which map to a directory.
Options -Indexes

# Follow symbolic links in this directory.
# For security reasons, Option followsymlinks cannot be overridden.
#Options +FollowSymLinks
Options +SymLinksIfOwnerMatch

# Make Drupal handle any 404 errors.
ErrorDocument 404 /index.php

# Set the default handler.
DirectoryIndex index.php index.html index.htm

# Override PHP settings that cannot be changed at runtime. See
# sites/default/default.settings.php and drupal_initialize_variables() in
# includes/bootstrap.inc for settings that can be changed at runtime.

# PHP 5, Apache 1 and 2.

php_flag magic_quotes_gpc off
php_flag magic_quotes_sybase off
php_flag register_globals off
php_flag session.auto_start off
php_value mbstring.http_input pass
php_value mbstring.http_output pass
php_flag mbstring.encoding_translation off

# Requires mod_expires to be enabled.

# Enable expirations.
ExpiresActive On

# Cache all files for 2 weeks after access (A).
ExpiresDefault A1209600


# Do not allow PHP scripts to be cached unless they explicitly send cache
# headers themselves. Otherwise all scripts would have to overwrite the
# headers set by mod_expires if they want another caching behavior. This may
# fail if an error occurs early in the bootstrap process, and it may cause
# problems if a non-Drupal PHP file is installed in a subdirectory.
ExpiresActive Off

# Various rewrite rules.

RewriteEngine on

# Block access to "hidden" directories whose names begin with a period. This
# includes directories used by version control systems such as Subversion or
# Git to store control files. Files whose names begin with a period, as well
# as the control files used by CVS, are protected by the FilesMatch directive
# above.
#
# NOTE: This only works when mod_rewrite is loaded. Without mod_rewrite, it is
# not possible to block access to entire directories from .htaccess, because
# is not allowed here.
#
# If you do not have mod_rewrite installed, you should remove these
# directories from your webroot or otherwise protect them from being
# downloaded.
RewriteRule "(^|/)\." - [F]

# If your site can be accessed both with and without the 'www.' prefix, you
# can use one of the following settings to redirect users to your preferred
# URL, either WITH or WITHOUT the 'www.' prefix. Choose ONLY one option:
#
# To redirect all users to access the site WITH the 'www.' prefix,
# (http://example.com/... will be redirected to http://www.example.com/...)
# uncomment the following:
RewriteCond %{HTTP_HOST} !^www\. [NC]
RewriteRule ^ http://www.%{HTTP_HOST}%{REQUEST_URI} [L,R=301]
#
# To redirect all users to access the site WITHOUT the 'www.' prefix,
# (http://www.example.com/... will be redirected to http://example.com/...)
# uncomment the following:
# RewriteCond %{HTTP_HOST} ^www\.(.+)$ [NC]
# RewriteRule ^ http://%1%{REQUEST_URI} [L,R=301]

# Modify the RewriteBase if you are using Drupal in a subdirectory or in a
# VirtualDocumentRoot and the rewrite rules are not working properly.
# For example if your site is at http://example.com/drupal uncomment and
# modify the following line:
# RewriteBase /drupal
#
# If your site is running in a VirtualDocumentRoot at http://example.com/,
# uncomment the following line:
RewriteBase /

### BOOST START ###

# Allow for alt paths to be set via htaccess rules; allows for cached variants (future mobile support)
RewriteRule .* - [E=boostpath:normal]

# Caching for anonymous users
# Skip boost IF not get request OR uri has wrong dir OR cookie is set OR request came from this server
RewriteCond %{REQUEST_METHOD} !^(GET|HEAD)$ [OR]
RewriteCond %{REQUEST_URI} (^/(admin|cache|misc|modules|sites|system|openid|themes|node/add|comment/reply))|(/(edit|user|user/(login|password|register))$) [OR]
RewriteCond %{HTTP_COOKIE} DRUPAL_UID [OR]
RewriteCond %{ENV:REDIRECT_STATUS} 200
RewriteRule .* - [S=3]

# GZIP
RewriteCond %{HTTP:Accept-encoding} !gzip
RewriteRule .* - [S=1]
RewriteCond %{DOCUMENT_ROOT}/cache/%{ENV:boostpath}/%{HTTP_HOST}%{REQUEST_URI}_%{QUERY_STRING}\.html -s
RewriteRule .* cache/%{ENV:boostpath}/%{HTTP_HOST}%{REQUEST_URI}_%{QUERY_STRING}\.html [L,T=text/html,E=no-gzip:1]

# NORMAL
RewriteCond %{DOCUMENT_ROOT}/cache/%{ENV:boostpath}/%{HTTP_HOST}%{REQUEST_URI}_%{QUERY_STRING}\.html -s
RewriteRule .* cache/%{ENV:boostpath}/%{HTTP_HOST}%{REQUEST_URI}_%{QUERY_STRING}\.html [L,T=text/html]

### BOOST END ###

# Pass all requests not referring directly to files in the filesystem to
# index.php. Clean URLs are handled in drupal_environment_initialize().
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !=/favicon.ico
RewriteRule ^ index.php [L]

# Rules to correctly serve gzip compressed CSS and JS files.
# Requires both mod_rewrite and mod_headers to be enabled.

# Serve gzip compressed CSS files if they exist and the client accepts gzip.
RewriteCond %{HTTP:Accept-encoding} gzip
RewriteCond %{REQUEST_FILENAME}\.gz -s
RewriteRule ^(.*)\.css $1\.css\.gz [QSA]

# Serve gzip compressed JS files if they exist and the client accepts gzip.
RewriteCond %{HTTP:Accept-encoding} gzip
RewriteCond %{REQUEST_FILENAME}\.gz -s
RewriteRule ^(.*)\.js $1\.js\.gz [QSA]

# Serve correct content types, and prevent mod_deflate double gzip.
RewriteRule \.css\.gz$ - [T=text/css,E=no-gzip:1]
RewriteRule \.js\.gz$ - [T=text/javascript,E=no-gzip:1]


# Serve correct encoding type.
Header set Content-Encoding gzip
# Force proxies to cache gzipped & non-gzipped css/js files separately.
Header append Vary Accept-Encoding



order allow,deny
allow from all

deny from 61.154.226.252
deny from 184.82.29.169
deny from 205.203.134.197
deny from 208.115.111.
deny from 208.115.113.
deny from 208.138.254.150
deny from 65.94.2.225
deny from 199.21.99.125
deny from 95.108.158.241
deny from 182.118.20.
deny from 182.118.21.
deny from 182.118.22.
deny from 182.118.25.
deny from 101.226.168.
deny from 101.226.166.
deny from 101.226.167.
deny from 101.226.169.
deny from 78.30.200.81
deny from 69.30.238.26
deny from 184.154.48.82

Anonymous’s picture

where can i set that: "Options +SymLinksIfOwnerMatch" setting in the boost config?

Bottom of the page

admin/config/system/boost/htaccess

under /var/www/html/drupal/cache/normal/%{HTTP_HOST}/.htaccess Options

certainly remove it from the root .htaccess file there is a distinct possibility with your hosting settings the cache/normal folder and sib directories could be being ignored/ denied. If your cache/normal .htaccess is being regenerated frequently then it may swap out the options correctly rather than you requiring to remove the entire cache.

jmatuska’s picture

FileSize
77.76 KB

i don't have/see that setting on my htaccess settings page: admin/config/system/boost/htaccess (see screenshot)

also, on that screenshot, be advised, i have tried every combination of settings for those first three sets of choices. the combination that USED to work (10 days ago) was the third choice for Servers URL, 2nd choice for Doc Root and 2nd choice for ETag Settings (i'm not on a server cluster). now, i can find no combo that works to serve my cached pages.

Anonymous’s picture

Oddly the "stable" release is still a beta, I'm using http://drupal.org/node/937552 which is stable and I thought that the SymLinks had been put into the stable branch. I cannot explain the sudden change 7.20 was a security release dealing with a vulnerability in the image resizing and URL's so shouldn't have an effect on boost.

How about you manually alter the .htaccess files in the root and cache to change the FollowSymLinks to if IfOwnerMatch. You should also be able visit the cached pages directly by typing in the URL which should give you an idea as to whether your configuration is/ was working, I would expect a 500 denied error based on your hosting and your current boost version.

jmatuska’s picture

These are the last 5 lines of my cache > htaccess file.
the last line is NOT commented OUT, so that should be what you wanted me to manually set, correct? i believe it's already correct...

SetHandler Drupal_Security_Do_Not_Remove_See_SA_2006_006
Options None
# For security reasons, Option followsymlinks cannot be overridden.
#Options +FollowSymLinks
Options +SymLinksIfOwnerMatch

Also, the same goes in my root htaccess file: (it's not commented out there, either)
# Follow symbolic links in this directory.
# For security reasons, Option followsymlinks cannot be overridden.
#Options +FollowSymLinks
Options +SymLinksIfOwnerMatch

And, yes, i CAN manually type in the URL of my cached pages and they show up just fine.
However, very rarely I do get a 500/internal server error when trying to load different pages on the site. Then a refresh brings the page up. What should those settings be to not get the intermittent 500 error? I've tried the recommended settings and then the 500 error happens very frequently. That's what caused me to use other settings.

Should i be using the dev version of Boost?

Will let you know what happens tomorrow, if my cache htaccess file gets re-created again...
Thanks for your help.

Anonymous’s picture

And, yes, i CAN manually type in the URL of my cached pages and they show up just fine.

That would be expected if the +SymLinksIfOwnerMatch is currently in your cache .htaccess file, what I expect to happen is that the file would be replaced with a +FollowSymLinks on a cache flush and then you should get a 500 error.

I would say from the description of intermittent 500 errors that there is an issue with your symlinks

Should i be using the dev version of Boost?

Yes.
I think that the first thing is to get consistent behaviour with boost handling +SymLinksIfOwnerMatch, the problem is that your web host and their "security" doesn't give a diagnosis as we have no idea of the configuration in the virtual hosts.

Options None

May even give you problems and may require hacking the boost module yet "options none" is a necessity for Drupal and Boost to prevent exploits.

jmatuska’s picture

Version: 7.x-1.0-beta2 » 7.x-1.x-dev
FileSize
142.35 KB

Most pages on site are being served correctly as cached pages, but only for one day, even though I have Max Cache Lifetime set at 3 weeks. The next morning, the pages re-cache as they are visited for the first time.

Steps taken:
I've upgraded to the dev version of boost: 7.x-1.0-beta3+12-dev, and i selected the option at bottom of htaccess config page (attached screenshot), so that now the last THREE lines of my cache .htaccess file look like this:

SetHandler Drupal_Security_Do_Not_Remove_See_SA_2006_006
Options None
Options +SymLinksIfOwnerMatch

Pages are all caching fine, but here are the things that are still wonky:
1. The htacccess file is recreated at each cron run (every 3 hours). At least the TIMESTAMP on the .htaccess file changes every 3 hours to be the same time as the most recent cron run. (Is that supposed to happen? I don't think it was happening before...)
2. It seems the whole site is being served as cached pages for one day. The home page re-caches itself every morning the very first time I visit it, even tho it is set to expire in 3 weeks, just like all other pages on site. (Max Cache Lifetime = 3 weeks). The other pages remained cached a little longer (for me) until someone else visits the page, then those pages are re-cached.

Thoughts?
Thanks again!
Jean

myriadtasks’s picture

Component: Caching logic » Apache integration
FileSize
6.88 KB

Prior to 7 July I was on
Drupal 7.22
Boost 7.x-1.0-beta2

everything was working fine and cache pages were being served correctly.

On 7 July I upgraded a few modules

AdSense 7.x-1.0 to 7.x.1.1
Backup and Migrate 7.x-2.5 to 7.x-2.7
Display Suite 7.x-1.7 to 7.x-1.8
GMap Module 7.x-2.6 to 7.x-2.7
HTTPRL 7.x-1.11 to 7.x-1.12
Location 7.x-3.0-alpha9 to 7.x-3.0-rc2
Taxonomy Manager 7.x-1.0-rc2 to 7.x-1.0

On 8 July I noticed that the anonymous page access had slowed down. An inspection of the page source showed no boost header. Further investigation revealed that every anonymous page visit was recreating the boost cache disregarding the cache expiry which was set to 3 hrs. Accessing the cached page directly via URL worked and it showed the boost headers with correctly set cache expiry.

Prior to the 7 July update the boost pages were being served correctly.

After reading numerous posts on this issue and finding no solution I hoped, that as described in this post, a shift from boost 7.x-1.0-beta2 to boost 7.x-1.0-beta3+12-dev might solve it. But it looks like this has not worked as well.

I have tried various boost configurations with %{HTTP_HOST}, %{SERVER_NAME}, and URL but none of them have helped serve the cache page.

The .htaccess in the root as well as the cached pages seems to be accurate and as described by Philip_Clarke and jmatsuka above. For some reason it looks like .htaccess is unable to redirect to the cached page and sends the anonymous user to index.php everytime which leads boost to re-created the cache ? a copy of my root .htaccess is attached below. Any help in identifying the issue will be much appreciated.

Thanks in advance

Samir

blunder’s picture

I'm having the exact same issue since updating a few modules earlier this month.

The modules I updated that you also updated are as follows:

Backup and Migrate updated to 7.x-2.7
HTTPRL updated to 7.x-1.12

I have a nickel that says it is HTTP Parallel Request Library that did it.

I'm going to see about reverting mine and then seeing what happened.

myriadtasks’s picture

I had no luck with rolling back or disabling the HTTP Parallel Request Library.

blunder’s picture

Nor did I. I gave up on it for now. It's broke. I removed HTTP PRL, Boost and a couple other modules and re-installed them. No luck on getting Boost to serve cache again. It creates them just fine though, which is why I have now disabled it. No reason to waste time creating pages if it's never going to server them. Such a shame because when the module worked it was fantastic.

Anonymous’s picture

Component: Apache integration » Core compatibility
Assigned: Unassigned »

Can we just pause for a minute and go back to the basics? I believe that there is probably more than one issue here.

Debug number 1: the ReWrite rules will not serve pages if there is a cookie called DRUPAL_UID set, so look for it with firebug.

Debug number 2: 500 errors historically were to do with the web server not being able to handle +FollowSymLinks which was solved in dev, if there are still 500 errors then they should appear in the apache logs.

Debug number 3: from working on another open source project I have found that PHP 5.3 sessions are handled slightly differently and it has been confirmed in a couple of other threads on drupal.org that some of the php flags need to commented out. Unfortunately there is such a variety of php.ini and server combinations in play that it is impossible to pin it down within the scope of this project I'd suggest as a starting point, creating a basic


phpinfo();

and comparing the values set in .htaccess from the defaults (which may be ignored or cause crashes if php is not running as mod_php) and removing them to narrow down the scope of debugging required.

skadu’s picture

When working on a separate project with boost, I noticed this same issue (seemingly) and it was related to the IP_Geolocate module. This module was storing the users location in session memory which created a session cookie that caused boost to stop serving cached pages (it actually prevented them from being created entirely after the session cookie was set on the first page visit). To fix this, i installed the Sessions Cache API module and chose to store information about the users location in a cookie instead of inside session memory, the cookie that it stores information in did not cause any problems with boost.

I would check out any location modules you may have installed to see if they are doing anything with sessions. The module that jumps out at me from myriadtasks module list is location. It uses the smart_ip module if it exists to get session information about someones location, I can't take the time right now to dig in to smart_ip to see how it is storing that information, but its certainly something to check out.

Anonymous’s picture

Thank you for your help and a possible solution.

The debugging info about checking whether a DRUPAL_UID cookie is set would confirm that a session starting module is responsible for this bug.

myriadtasks’s picture

Philip Clarke & skadu, thanks a ton for the inputs and the help to solve this issue. Much Appreciated.

Here's what I have so far

1. Checked with Firebug. The only cookies created are the ones by Google Analytics, addthis and the has_js cookie. No DRUPAL_UID Cookie.
2. I do not get any 500 errors. all pages are served correctly, cache is created for each of the pages served every time the page is accessed, but the cached pages are not served for anonymous users.
3. My PHP Version or php.ini values have not changed from the last time that Boost worked. htaccess has changed over the last few upgrades of drupal I think. I am currently on Drupal 7.23 and have tried different versions of .htaccess including the ones that worked with boost earlier.
4. I can confirm that I am not using IP_Geolocate module, and cannot see any other cookie that might be related to sessions.

Kindly let me know if there is any additional information I can provide to help narrow down the possible cause for the cache not being served.

Thanks a ton !

Anonymous’s picture

Has your apache version changed ? as there's been a report that changing from 2.2 to 2.4 conflicts with the global redirect module. Which version ?

Can you post your current .htaccess file ?

myriadtasks’s picture

FileSize
6.88 KB

I am on godaddy 4gH shared hosting and am unable to get the version of Apache from shell as httpd command is disabled. phpinfo does not return the version as well. However, forum posts I came across mention that Godaddy has indeed upgraded its shared hosting from 2.2 to 2.4.

Global redirect module is not installed as I read that it conflicts with Domain Access modules. I use multiple sub-domains that share certain content across all sub-domains.

Maybe this conflicts with something else ?

The current .htaccess is attached.

Thanks !

Update : 11 Sep 2013

The many posts I have been reading on Godaddy forums complain of redirect issues after the Apache upgrade from 2.2 to 2.4 which took place just about the time Boost stopped serving cached pages. It certainly looks like you are right about the Apache upgrade being the issue here. below is an extract from a forum post on godaddy

#### Godaddy Forum Post begins #####

After more extensive research of this issue, I found the upgrade seems to have changed how paths are referenced (though I’m not yet clear if this is regarding REQUEST_URI or all paths for mod_rewrite). In my case, this required changes from:

RewriteCond %{REQUEST_URI} /foo.html$ [NC]
RewriteCond %{QUERY_STRING} $
RewriteRule (.*) /index.php?foo [L,QSA]

to:

RewriteCond %{REQUEST_URI} /bar/foo.html$ [NC]
RewriteCond %{QUERY_STRING} $
RewriteRule (.*) /index.php?foo [L,QSA]

…where bar is the directory in which the site exists relative to my main account. Everything worked fine prior to this for years.

#### Godaddy Forum Post Ends #####

Hope this helps in identifying the issue.

yannickoo’s picture

I had the same problem and I found a solution: Try to set Server's URL or Name to the hardcoded URL and not to a dynamic one. The same for Document Root.

So Server's URL or Name is not more %{HTTP_HOST} but domain.tld and Document Root is not more %{DOCUMENT_ROOT} but /var/www/... (the absolute path).

Good luck!

taote’s picture

I'm getting an extrange behaviour.

I'm testing it on Firefox for Mac, and only if I press cmd+R, the message "Page cached by Boost" is added at the bottom. If after that I go back to the same page, clicking on the menu for example, that message is removed from the source code. Why????

Is that not very weird?

Anonymous’s picture

How long is the page supposed to be cached for ? Using ftp can you see the target page in the cache ?

I've noticed odd things with firefox and safari before, its not that boost is not functioning it's that the browser is just displaying the version in it's original memory cache that it picked up on the first visit to the page. The when you "view source" it displays the cached version, but then hitting CTRL+R (cmd+R) the source window forces a reload and suddenly the comment appears. Using firebug in firefox and clicking on the Net tab and the downwards arrow and ticking disable browser cache helps. In safari using the Develop menu you can disable the caches but it's not perfect when using the web inspector.

tomotomo’s picture

Issue summary: View changes

I see the cached page file get refreshed every single time I reload the page, even if reloading immediately. Never is the cached file being served, so I never see the Boost block nor the Boost html comment at the bottom.

So for me, Boost is not actually doing anything useful, just wasting resources.

Anonymous’s picture

Unfortunately there is not enough information in your comment to provide an answer. Firstly is there a cookie set when logged out called DRUPAL_UID ? this can be found by using firefox or chrome and firebug to see what cookies are set, it is extremely important that there is no cookie at all and some modules have been known to set a 0 or a negative number.

tomotomo’s picture

There is no cookie set because I'm testing in a Chrome Incognito window every time, so that there are no cookies at all.

The cache file (e.g. cache/normal/www.website.com/en/foo_.html) is being generated each time the url is accessed, and that file is never being served instead.

tomotomo’s picture

Ah, it seems to work in Firefox (which I only use for testing). But it doesn't work for Chrome in incognito mode.

BOGUƎ’s picture

A little late to the party, but hardcoding to the file path as suggested by yannickoo did the trick here (1&1 shared hosting). thanks!

bellagio’s picture

deleting my comments.