Boost setup

Boost only works for anonymous users, so the first thing to do and to check is to log out, view a page, refresh it (in case the page was not cached on the first viewing) and then view the source. If boost is working then at the bottom of the page will be a comment saying when the page was cached and when it should expire.

If no comment appears then you have a module that sets a user id, (or your site is https), and boost disabled itself. (Boost turns itself off for things like login pages so that username are not cached is someone enters them incorrectly).

You could easily set your boost lifetime to a month and not have any further use for the setting. Every time you edit a page then boost automatically wipes it out and regenerates it using the crawler, which never crawls the whole site, instead it just deals with pages that are edited, deleted, or related pages with links on them.

Since boost does not deal with logged in users, if the majority of your visitors are logged in, then you would not see any improvements and should looks at authCache or another caching mechanism. Boost only deals with php pages being turned into html, so server resources are split among many things, network bandwidth in which case you should be looking at your gzip settings (and making sure that you are also not wasting cpu cycles by zipping compressed items like images), the amount of things you are trying to send the user in the first place, all of which can be analysed by installing firebug in firefox, or using developer tools in chrome, and selecting the network tab, which will give you a breakdown of what your page is doing. With boost installed, the first page call will be "normal", a second call to a cached page will be much faster as then html is just sent out, so once that is out of the way, you can focus on what else is slowing down the site, your css and js files should be aggregated in Drupal's normal caching mechanism and a long time should be chosen for those.

Whatever your logging mechanism is, it should be analysed carefully to determine the bottleneck, high cpu usage can be database/ PHP related, of server compression. If you have a lot of 404 errors, you could be the victim of dumb bots probing your site for vulnerabilities that don't exist since they run through a list of joomla, wordpress, zen-cart.... etc exploits which are irrelevant but will use up your resources, and you should certainly consider the fast 404 drupal module that sends a static page out rather than hog your db resources.

Do you have anything being created under the folder cache/normal, also I notice visiting your site that there is an almost immediate redirect to /en/ which may be the problem for a rewrite path. There are several threads for multilinugual module configuration with .htaccess examples. I would cut the .htaccess right back and only use html for the time being to debug it.

Also I noticed something.

http://studiesinuk.net/cache/normal/studiesinuk.net/ give a 500 error, you may need to use the SymLinksIfOwnerMatch option which is available in the dev version of boost and is required for some web hosting services.

When I went to the site by just pasting in

studiesinuk.net

to the browser from your .htaccess file, it immediately redirected to

studiesinuk.net/en/

I have noticed a problem however. You should be able to access the cache files directly using the directory structure you mentioned

http://studiesinuk.net/cache/normal/studiesinuk.net/en/_.html

should give the cached index page, but if placed in the browser it is redirected to

http://studiesinuk.net/en/cache/normal/studiesinuk.net/en/_.html

and giving a 404 error which is correct, it looks like the there is something incorrect with the multilingual module and I suggest you look through the threads, I believe the problem was solved for a japanese site if memory serves correctly, it appears boost is functioning correctly but that it is the rewrite rules that need tweaking.

The pages are being generated I can see that much as the comment appears on

http://www.green-clinic.com.php53-2.dfw1-2.websitetestlink.com/cache/nor...

The boost code is very specific for it's location it must be placed after RewriteBase

and before

  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_FILENAME} !-d
  RewriteCond %{REQUEST_URI} !=/favicon.ico
  RewriteRule ^ index.php [L]

which is the standard boost rewrite url code. I cannot see any cookies being set that would override boost, I'd certainly recommend a lifetime much greater than 1 hour.

If you have the Drupal code in a virtual host configuration file, then that can also do the same thing. Boost itself is working but your web server is not sending out the pages so the rewrite rules are being missed for some reason. You may want to do a simple

order allow,deny
deny from all

in your .htaccess file just to check that .htaccess is working. This kind of error normally comes down to one of two things, a module installed that logs a user in even if they are anonymous (and no DRUPAL_UID cookie is set on your site), or the rewrite rules being ignored.

As for the length of time. Boost can be set for a massively long time especially if you have httprl, cache expiry and boost_crawler (bad name for it as it regenerates pages that are updated, inserted or deleted rather than crawls the site) installed, then your anonymous users would build a cache of static files and the only reason ever to delete the cache would be if the site underwent any style changes that you wanted to propagate to older pages.

Your problem is the Rewrite one, and if you are using a cloud server then you almost certainly have a root login and probably need someone a little more experienced to go through your apache configuration. It is rather slow and the main speed loss is from processing the PHP, I can see that looking at it through firebug. If you rename your .htaccess file (must remove the .ht at the beginning) and drupal still works then there is configuration elsewhere that has set up your system, if you are not comfortable with editing files directly on the server.

It is very likely that it is using rewrite configuration elsewhere, I've seen it a few times including multi-billion dollar turnover company websites :) If the site continues to function then that's the issue, if not then you'll be needing to enable a RewriteDebug log file to see what the problem is, which can only be done by editing main apache configuration and turning it on and off again.

The deny from all statement is just a test to see if your .htaccess is working. If it is then your site would stop with a 500 error. The same test can be achieved by renaming the .htaccess file and seeing if the site still works.

No it just means that the .htaccess file is being read, you need to remove / rename the file to see if the site works to know if there is a configuration elsewhere.

You are going to have to read up about enabling the RewriteDebug log in your virtual host/ apache config to try and work out what's going on. If clean URL's aren't working with a disabled .htaccess then the root .htaccess is probably in control and you'll need the debug log to work out why the boost rules are being ignored.

That article is not quite correct. For one thing you need to disable page caching (the other pages are fine) but you need under performance to turn off the check box for

Cache pages for anonymous users

I also do not understand the comment about length.

boost cache length is entirely dependent on what else you have installed. If you have the crawler component installed then any page that is edited/ updated/ deleted will be regenerated, so the cache length can be infinite if you wish to untick the box "remove old files on cron" which is my personal preference. However if you are doing major stylistic changes then you are going to want to alter that because your cached pages would be with "the old layout".

cron in drupal 7 is pretty much automatic as long as an authorised user logs in, or someone hits a page that boost does not cache. Depends on the frequency of updates to the site and really is most useful for search indexing, although the crawler does use cron it already expires the pages so an anonymous user could generate the pages before cron gets around to it so fresh content is always going to be provided. The other aspect of cron is the checking for available updates, that's a security issue so should be run at least once a week even if nothing else is happening to the site.

So the solution to the issue seems to be that your %{DOCUMENT_ROOT} does not match the filesystem on your cloud server, this would have been diagnosed by the Rewrite log being enabled as it would have given you the path that mod_rewrite was looking for in the filesystem.

I will look into the boost crawler. Seems like the more efficient way to go so the pages are cached automatically upon expiration rather than requiring a user to load the page again before caching (assuming I'm understanding that correctly).

Yes, otherwise you have to rely on cron deleting the pages and anonymous users creating the cache.

So with the "remove old files on cron" option, if its enabled Boost will delete the expired pages when cron is run?

Yes, but I find there is little point to delete stale cache files apart from to save disk space unless the layout has changed, since you still require then an anonymous user to visit the page to re-create the cache. There is a theoretical advantage that the db tables would be "in memory" for non-cached pages, but thay tends to be far and few between.

If its disabled and using the crawler, the crawler will automatically regenerate the cached file upon expiration so the user doesn't need to hit the page first in order to cache it?

Am I understanding that correctly?

No not on expiration, on modification, plus if the title is changed and the page appears in blocks on other pages, then the other pages are also deleted and then regenerated.

In regards to the rewrite log, wouldn't that need to be enabled in the server configurations? That's what I've read. If so, I don't have access to those settings, nor am I familiar with them.

Yes, but it is unusual to have a cloud server with no access to the configuration, that's more like cheap shared hosting.

If you don't have expire then boost will run on cron and you'll need to have the setting turned on for "expire pages on cron run". This would only effect anonymous users but they could get way out of date pages where comments never updated.

One interesting point. If you put your site into maintenance mode, then anonymous users will still see "the site" as the rewrite rules hit the boost cache before going to index.php so the cache needs to be flushed if any major correction work needs to take place.

So to make sure I understand correctly, the cache either gets cleared
when cron is run or when the expire module detects a content update of
some sort. At that point, I've got the boost crawler and httprl modules
installed that will automatically regenerate those cached pages that
were cleared without a user having to hit the page. That correct?

Exactly.

Difficult subject, depends on many things.

• cron would only be triggered by a logged in user (as anon users see boost)
• or a visit to a admin page / like register new users which might be triggered by search engine spiders
• or data being posted to the site by a spam bot looking for a vulnerability (so they can prove useful)

You need it running at least once a week for update checking, possibly more if there is a large amount of new content every day as cron also controls your search indexing.

If you have the crawler then cache everything for a long long time and then turn off the cron setting that removes stale files, then only ever clear the cache if the style of the site changes. The crawler would handle any updates, the only thing that may ever need changing would be if you had the google tracking modules enabled and they changed the javascript code which would then require an old cache clearout. But it will drop your CPU usage etc down to virtually nothing, and speed up everything especially any spiders indexing old information which can be a big drain.

You can ignore those settings for the pages but not for the blocks (though any performance increase is going to be very minimal). That highlights a lack in my knowledge as I do not know if those settings affect the aggregate css and javascript functions inside of drupal.

Yes a menu is a block so is controlled by drupal's caching mechanism not boost's. Boost caches the output to the browser at one instant, if you move your links around or play with stylesheets, boost will either not find the new styles or the links will be the old ones just like if you saved a page to your hard drive.

That all looks fine as long as you remember that menus are blocks, and that themes work exactly the same way, boost will show the "old one" unless content is updated. It is only my personal opinion that cron should be unticked, but it's based on seeing sites that are spidered and using thousands of CPU cycles to generate old content which can run into thousands of pages.

Look through the boost settings and there is a tick box, "remove stale files on cron", I don't think that should be ticked.

The crawler only crawls for changes, anonymous users generates the cache. It's a frequent naming error.

There's lots of ways of doing it from paid for systems, to submitting your site to a search engine which will spider it. There's not much when you're on shared hosting but there is a thread somewhere here on some PHP that people have put together to mimic a spider.

This module may become deprecated in favor of a more universal solution (like Boost integration with the Cache Expiration module).

As far as I remember it serves the same function as the crawler and the cache expiration module so is not a core part of boost, it would not be likely that boost would be directly integrated with cache expiration as other modules rely on cache expiration, the change the boost 6 to boost 7 was a total redesign that took a very large piece of code that was quite difficult to manage and to split it into discrete chunks. I would doubt that one would go backwards and I suspect that boost_expire was created without fully understanding the functioning of the crawler as it appears to be a duplicate project.