We have developed a patch that is likely to be accepted into Drupal 4.7, http://drupal.org/node/45414. The static file caching patch reduces load on the server in four ways.

1) Database load: Anonymous users are served static pages directly with out contacting the database for session management or other database queries, reducing the anonymous load to 0 and allowing the query cache to be used more effectively for authenticated users.

2) Disk IO load: Pages being served by Apache or a web server will take advantage of the operating systems internal page caching mechanism meaning most popular pages will be served directly from memory and not from disk reducing disk IO latency.

3) PHP CPU load: If the static caching patch is configured much of the Drupal PHP that needs to be interpreted or even loaded load from an Op cache will not be done as Drupal is largely bypassed in serving pages.

4) Apache thread load: Drupal's PHP apache threads are at least 16MB each. Static pages in a directory can be re-directed and served from a separate web server that will use 1M threads for anonymous pages meaning that expensive threads for Drupal can be reserved just for expensive operations.

These results have been validated with a similar file caching patch for Wordpress.com. Please test this patch and help us get this patch into Drupal core. If you are interested in funding this kind of development please contact me.

Here's is a quick overview from Jeremy.

Find a patch against CVS HEAD with more details as to how
it works and some benchmarking results here:
http://drupal.org/node/45414

A quick summary from my most recent benchmarking, under a
heavy siege load:

Database cache:
1,000 pages in 34.1 seconds. That's an average of 29.38
pages per second, with each page loading in 34.12 ms.
Under this load, 7.7% of the page requests failed.

File cache:
1,000 pages in 28.7 seconds. That's an average of 36.1
pages per second, with each page loading in 28.54 ms. 0
page requests failed.

File cache with FastPath:
1,000 pages in 5.23 seconds. That's an average of 191.12
pages per second, with each page loading in 5.233 ms. 0
page requests failed.

(FastPath is a new bootstrap phase that allows the display
of file-cached pages for anonymous users without bootstraping
the database or sessions)

Comments

ymcp’s picture

Very interesting... I had been experimenting with apache rewrite rules to serve static copies of some pages when the server was under heavy load...

I'll like to try your patch & see how the performance compares to my (half-finished) "solution".

Any chance of a back-port to 4.6, or is it too 4.7 specific?

smazsyr’s picture

I don't understand the ins and outs of it all technically, but my site has been Slashdotted 10 times this year and this could help tremendously. There have been times when I've manually created a static HTML page of a story posted on my Drupal site, just to lower the CPU load.

In addition, if it works out as planned, this could help a lot of mid-sized sites who can't quite justify the expense of going to a dedicated server but are bottlenecking on their shared or virtual setups.

I'd be very interested to hear from Dries and other OG Drupalers their thought about this approach.

kae’s picture

what host do you use? have you tried site5?

nathandigriz’s picture

Nice looking and very much needed. The only thing I can see that would be necessary before I test it would be more complete documentation and commented code.

The reason I say this is because I was looking for a way to manually clear the cache. This to me would be an absolute must since Apache user would be the one creating the cache and controlling the filesystem. I would have to write another script to do this since FTP would not work. So it should be built in already.

Being able to clear the cache manually would also be necessary for any module development and testing while running a site under "fastPath".

kbahey’s picture

The devel module has an option to clear the cache.

However, this is for the existing caching, which is database based.

So, it would be nice to add this feature to devel, and it would even be better to to add a cache clear for the new fastpath file based cache.

If Jeremy does not add an option to his patch, I will probably do it at some point in the future.
--
Drupal development and customization: 2bits.com
Personal: Baheyeldin.com

--
Drupal performance tuning and optimization, hosting, development, and consulting: 2bits.com, Inc. and Twitter at: @2bits
Personal blog: Ba

veridicus’s picture

This is EXACTLY what I need! I was just about to start writing a patch to 4.6 to perform static file caching. This is awsome! I can't wait to test 4.7 for my sites and try this patch. Thanks!

---
Gadgets Wanted - Geeks Want Gadgets
DocForge - News, articles, and links for software developers

NITEMAN’s picture

Only a question, this will work on IIS or only in Apache?
It seems to be a great step for drupal!

Administering: Win 2003 SP1-IIS 6.0-PHP 5.0.3 ISAPI-MySQL 5.0-Drupal 4.6.5
Webmaster: www.pesepe.com-Beta.pesepe.com-gamers.pesepe.com

sepeck’s picture

Pull down the patch and try it. There are basic instructions in the developers handbook. IIS users are in the significant minority around here so more feedback is needed. :)

How to setup a test environment

-Steven Peck
---------
Test site, always start with a test site.
Drupal Best Practices Guide -|- Black Mountain

-Steven Peck
---------
Test site, always start with a test site.
Drupal Best Practices Guide

mcduarte2000’s picture

I want it! With this new development, drupal will give a big, big jump!

Miguel Duarte

Webmaster of: Lisbon Guide & Love Poems

yongli’s picture

This actually answers one of my remaining questions for dupal.

mcduarte2000’s picture

Will this patch be part of the core or will we always have to apply it? I'm testing it and it seems to be working just fine.

Miguel Duarte

Webmaster of: Lisbon Guide & Love Poems

olet’s picture

Don't forget user who use 4.6.5
-------
will this work with 4.6.5?
will this work with 4.6.5?
will this work with 4.6.5?
will this work with 4.6.5?
thanks!!!!

Amazon’s picture

If you test it in 4.7 and report results it's more likely to get back ported to 4.6.5.

If you are able to test and respond on the issue we can work together to make it available for 4.6.5.

Kieran Lal

SimonVlc’s picture

Hi all,

first congrats to Amazon for coding this feature. I just installed the 4.7.2 patch, and I´m having a little problem with anonymous users.

When one of this users posts a comment to the site, it didn´t view it until the cache get rebuilt, using the file system cache. I suppose that this is a normal behaviour, but can be evaded?

Thanks in advance, Simon.

eagereyes’s picture

What I don't understand is why the cache even has to be expired with cron. What's the point of deleting cache pages if the corresponding node has not changed? And why not simply delete the cached version every time a node is updated or a comment is added? I think doing this in an event-driven way would make handling the cache much simpler. Or am I missing something?

But this is a great feature, and extremely useful, even for low-traffic sites (appear much snappier).

Christoph C. Cemper’s picture

amazing approach...

my other thoughts on caching/hiperf would be distributed installations with a central database...

what if 1 master node would server 10 client nodes and each client node could cache (static or via mysql) on it's local db...

would server for country-specific hosting as well as load distribution while keeping content consistent...

what do you think?

--- http://www.cemper.com
--- http://weblog.cemper.com
--- http://www.marketingfan.com