We have developed a patch that is likely to be accepted into Drupal 4.7, http://drupal.org/node/45414. The static file caching patch reduces load on the server in four ways.
1) Database load: Anonymous users are served static pages directly with out contacting the database for session management or other database queries, reducing the anonymous load to 0 and allowing the query cache to be used more effectively for authenticated users.
2) Disk IO load: Pages being served by Apache or a web server will take advantage of the operating systems internal page caching mechanism meaning most popular pages will be served directly from memory and not from disk reducing disk IO latency.
3) PHP CPU load: If the static caching patch is configured much of the Drupal PHP that needs to be interpreted or even loaded load from an Op cache will not be done as Drupal is largely bypassed in serving pages.
4) Apache thread load: Drupal's PHP apache threads are at least 16MB each. Static pages in a directory can be re-directed and served from a separate web server that will use 1M threads for anonymous pages meaning that expensive threads for Drupal can be reserved just for expensive operations.
These results have been validated with a similar file caching patch for Wordpress.com. Please test this patch and help us get this patch into Drupal core. If you are interested in funding this kind of development please contact me.
Here's is a quick overview from Jeremy.
Find a patch against CVS HEAD with more details as to how
it works and some benchmarking results here:
http://drupal.org/node/45414
A quick summary from my most recent benchmarking, under a
heavy siege load:
Database cache:
1,000 pages in 34.1 seconds. That's an average of 29.38
pages per second, with each page loading in 34.12 ms.
Under this load, 7.7% of the page requests failed.
File cache:
1,000 pages in 28.7 seconds. That's an average of 36.1
pages per second, with each page loading in 28.54 ms. 0
page requests failed.
File cache with FastPath:
1,000 pages in 5.23 seconds. That's an average of 191.12
pages per second, with each page loading in 5.233 ms. 0
page requests failed.
(FastPath is a new bootstrap phase that allows the display
of file-cached pages for anonymous users without bootstraping
the database or sessions)
Comments
very interesting...
Very interesting... I had been experimenting with apache rewrite rules to serve static copies of some pages when the server was under heavy load...
I'll like to try your patch & see how the performance compares to my (half-finished) "solution".
Any chance of a back-port to 4.6, or is it too 4.7 specific?
Very cool
I don't understand the ins and outs of it all technically, but my site has been Slashdotted 10 times this year and this could help tremendously. There have been times when I've manually created a static HTML page of a story posted on my Drupal site, just to lower the CPU load.
In addition, if it works out as planned, this could help a lot of mid-sized sites who can't quite justify the expense of going to a dedicated server but are bottlenecking on their shared or virtual setups.
I'd be very interested to hear from Dries and other OG Drupalers their thought about this approach.
agree
what host do you use? have you tried site5?
Nice looking and very much
Nice looking and very much needed. The only thing I can see that would be necessary before I test it would be more complete documentation and commented code.
The reason I say this is because I was looking for a way to manually clear the cache. This to me would be an absolute must since Apache user would be the one creating the cache and controlling the filesystem. I would have to write another script to do this since FTP would not work. So it should be built in already.
Being able to clear the cache manually would also be necessary for any module development and testing while running a site under "fastPath".
devel module
The devel module has an option to clear the cache.
However, this is for the existing caching, which is database based.
So, it would be nice to add this feature to devel, and it would even be better to to add a cache clear for the new fastpath file based cache.
If Jeremy does not add an option to his patch, I will probably do it at some point in the future.
--
Drupal development and customization: 2bits.com
Personal: Baheyeldin.com
--
Drupal performance tuning and optimization, hosting, development, and consulting: 2bits.com, Inc. and Twitter at: @2bits
Personal blog: Ba
Excellent!
This is EXACTLY what I need! I was just about to start writing a patch to 4.6 to perform static file caching. This is awsome! I can't wait to test 4.7 for my sites and try this patch. Thanks!
---
Gadgets Wanted - Geeks Want Gadgets
DocForge - News, articles, and links for software developers
Very Nice!
Only a question, this will work on IIS or only in Apache?
It seems to be a great step for drupal!
Administering: Win 2003 SP1-IIS 6.0-PHP 5.0.3 ISAPI-MySQL 5.0-Drupal 4.6.5
Webmaster: www.pesepe.com-Beta.pesepe.com-gamers.pesepe.com
www.SceneBeta.com
Pull down the patch and try
Pull down the patch and try it. There are basic instructions in the developers handbook. IIS users are in the significant minority around here so more feedback is needed. :)
How to setup a test environment
-Steven Peck
---------
Test site, always start with a test site.
Drupal Best Practices Guide -|- Black Mountain
-Steven Peck
---------
Test site, always start with a test site.
Drupal Best Practices Guide
This is a dream! :)
I want it! With this new development, drupal will give a big, big jump!
Miguel Duarte
Webmaster of: Lisbon Guide & Love Poems
Nice
This actually answers one of my remaining questions for dupal.
One question
Will this patch be part of the core or will we always have to apply it? I'm testing it and it seems to be working just fine.
Miguel Duarte
Webmaster of: Lisbon Guide & Love Poems
will this work with 4.6.5
Don't forget user who use 4.6.5
-------
will this work with 4.6.5?
will this work with 4.6.5?
will this work with 4.6.5?
will this work with 4.6.5?
thanks!!!!
Looking for partners to collaborate
If you test it in 4.7 and report results it's more likely to get back ported to 4.6.5.
If you are able to test and respond on the issue we can work together to make it available for 4.6.5.
Kieran Lal
Anonymous comments
Hi all,
first congrats to Amazon for coding this feature. I just installed the 4.7.2 patch, and I´m having a little problem with anonymous users.
When one of this users posts a comment to the site, it didn´t view it until the cache get rebuilt, using the file system cache. I suppose that this is a normal behaviour, but can be evaded?
Thanks in advance, Simon.
Delete cache file when page is updated
What I don't understand is why the cache even has to be expired with cron. What's the point of deleting cache pages if the corresponding node has not changed? And why not simply delete the cached version every time a node is updated or a comment is added? I think doing this in an event-driven way would make handling the cache much simpler. Or am I missing something?
But this is a great feature, and extremely useful, even for low-traffic sites (appear much snappier).
amazing
amazing approach...
my other thoughts on caching/hiperf would be distributed installations with a central database...
what if 1 master node would server 10 client nodes and each client node could cache (static or via mysql) on it's local db...
would server for country-specific hosting as well as load distribution while keeping content consistent...
what do you think?
--- http://www.cemper.com
--- http://weblog.cemper.com
--- http://www.marketingfan.com