Why does Drupal have such a large memory footprint (and how can I reduce it)?

Sorry, the bold text was just for emphasis, I didn't mean for it to be read as shouting. (I thought all caps was shouting)

I am not trying to be accusatory, I just want to understand some of what is happening behind the scenes so that I can be an effective sysadmin for a box running Drupal. I feel it is a legitimate question to ask - the answer would probably be useful to many others as well.

I will also log an issue for this - thanks for the suggestion.

PHP is responsible for allocating memory.

All Drupal does is load a whole bunch of PHP files (from core, modules and the themes involved) into memory. What actually gets done beyond that depends on what those files do. There are data structures loaded for the menu hierarchy, config settings, caches, nodes, taxonomy etc etc. There aren't many people that could go into exhaustive detail about this, and they probably won't be reading this post anyway.

Some pages require a lot more memory than others - usually the big ones are certain pages in the admin section (eg module listings etc). You can use the devel module to see how much memory any particular page used.

As far as I'm aware (you'd need to check this yourself as I don't know for sure), mod_php doesn't release the memory used by Apache processes. So if a certain child process is used to load an expensive page and ends up allocating itself 30MB, that process will still be allocated 30MB even if all its subsequent requests require half that. To mitigate that, you'd need to lower the lifetimes of the Apache processes so that on average less of them eventually hit the expensive pages.

At this stage in your sites development where most requests are from you and are in the admin section, each process will be showing a lot of memory. When much more of your traffic is outside traffic (especially with cached anonymous traffic), there will be smaller proportion hitting the expensive admin pages.

If you really really want to minimise RAM usage and have exhausted all Apache tuning methods you'll need to investigate alternative web server configurations. eg the Apache prefork model and mod_php are renowned memory hogs. I don't know if PHP is thread safe yet though, but I imagine the threaded models would use far less duplicated memory. And you might even want to look at FastCGI and other web servers like lighty or nginx etc

But a very flexible extensible system like Drupal will always use a lot more memory than one that is coded to do very specific things. A lot of Drupal modules are intended for non coders (or coders that want to save time) to be able to do complex tasks just through configuration UIs - eg CCK, Views, WorkflowNG etc etc. The very flexible configuration of these modules uses much more code (and memory)

Sometimes you may be loading a number of large modules to do a couple of small things that could be replaced by one simple 'hardcoded' custom module that uses next to no memory.

It's a tradeoff between the cost of development resources and the cost of deployment resources, and usually the deployment resources are cheaper. You should only spend development resources on optimizing stuff you have identified as a real world bottleneck that is more expensive or impossible to fix with deployment resources. But in terms of deployment costs, RAM generally isn't an expensive bottleneck to fix compared to other bottlenecks (eg database). I suspect you are worrying about this prematurely.

Not many people really worry too much about memory usage with Drupal - most concerns are with the levels of DB activity. Once you install the devel module - you'll probably forget about memory and have a whole new thing to focus on :)