By john t. haller on

I've been using Drupal for a while now and have had to use some duct tape style hacks to get things working well. One of them was using file caches for specific high-volume pages that aren't modified often and then redirecting those pages via htaccess. For example, I have a page of applications, with a url alias of example.com/applications. This is a heavily used page that is only modified once every few weeks. So, I pull the page as an anonymous user and save it in a cache directory on the server as applications.html. Then within .htacess, I redirect it to this file using the following:
RewriteRule ^applications$ http://example.com/cache/applications.html [L]

This reduced the load on the server quite a bit. It's still struggling badly under load, but it's better. The main reason I had to do this is that I have lots of users (15,000+) and most of them are logged in all the time... making the Drupal cache mostly useless with them. But, this got me thinking... wouldn't it make sense to be able to set specific pages within a Drupal site to treat ALL users as anonymous, thus being able to better utilize Drupal's cache? Or, perhaps, all users except those of a specific type (say 'admin' users)?

I know editing would be an issue (how would Drupal display the Edit tab?)... so just showing it for specific types of users would work. Any thoughts on this?

(I'm in a bit of a desperate need to get Drupal performing better for this large community site, so I'd even be willing to post a small bounty if there's any coding that needs doing)

Comments

vm’s picture

vm commented

Which version of Drupal ? I ask because D5 caches more than anon users. And has some settings not seen in 4.7.x

mfer’s picture

mfer commented

I don't know if any version of caching will help you. You pointed out that with your own file based cache your hosting is still "struggling badly under load". You may want to consider upgrading to a more powerful hosting situation.

The problem with caching a specific page for logged in users is that users in different groups and with different customizations will have very different looking pages. Caching based on all the possible dynamic features would leave with one very large cache store. This is just not a logistically doable idea for a release module to the community or something in core.

But, if you know your configuration you may be able to write a custom module that is able to do just this form of caching. There are, also, modules like the block cache which may be able to help.

You may want to look at the file based caching module. I am not sure of the full abilities of that but it's something to look into.

--
Matt
http://www.mattfarina.com

john t. haller’s picture

john t. haller commented

Let the server admin choose to structure certain pages to look exactly the same whether the user is logged in or not and allow those to be cached. I only cached 3 pages of the most popular set to get the load under control. caching more would in drupal as with anon users would help a lot.

Unfortunately, it's looking more and more like I'll have to drop Drupal at least for site forums. If I were only using Drupal for the site itself and not the forums, nearly all users would be anonymous and performance would be much, much better. I was looking for a solution to approximate that setup without having to drop Drupal for the forums.

I've upgraded as much as I can. I'm on a dual-CPU, raid5 dedicated server. I'm using Drupal 5 with agressive caching. CSS compression/combination. mySQL is setup with the my huge profile (mySQL cache enabled). Very few modules in use. At about 170,000 page views a day, Drupal is falling down under the load. So, I'm looking for anything and everything to try and get things happier. Like I said, the selective caching was just a thought.

Want some portable apps? Firefox, Thunderbird, OpenOffice.org, Gaim, etc.

moshe weitzman’s picture

moshe weitzman commented

improving performance for a heavily used site is non trivial. what is needed from you and your team is detailed analysis of the problem. we can't help you until we have that.

you have to work hard to identify the bottlenecks. you need to know if web server or database or OS are the limiting factor. you need to know which queries are slow. the drupal team loves jumping on these situations, when drupal is at fault. but you have to do a lot of painstaking analysis to get to that point.

so far, you have proposed more caching and thats OK, but it doesn't really tell us where the real problems are.

i do agree though that we need to improve caching for logged in users. here is a drop in block cache module that often helps and here is a core patch. please report back on their usefulness.