Making your Drupal modules use less memory

http://wtanaka.com/node/7644

I think everyone should be brought to the attention of this issue raised by Wesley Tanaka and start producing optimised modules.

I have tried optimising my current modules and it takes a terribly long time with maybe unexpected results as I do not know how the modules work. Module authors themselves are in the best position to implement the optimisation as they know what can and cannot be excluded from their main code.

Almost all modules are bloated to the extent that users on shared hosts cannot enable every single module since the memory limit imposed has been exceeded. Basically, just removing admin code from the main module file alone can reduce the memory footprint by at least 2% which adds up to a lot if the module is extremely large such as the following:

views 410976
views_ui 438284
buddylist 468552
workflow_ng 486908
uc_product 519848
uc_store 611160
og 752396
uc_order 958924

If all modules are optimised, the memory savings can even be substantial, helping users who are receiving the white screen of death on admin and other pages. As an example, I removed the code from views_ui entirely except for users who have administer access and the memory footprint of the module dropped to 4048 bytes! That is like a 99% improvement.

More importantly, Drupal loads all the unnecessary stuff in the module files on every page which can slow down website performance.

Therefore, I hope future modules releases from everyone can be improved.

Comments

iandickson’s picture

Obviously module makers can't maintain multiple versions and most of us really don't want to start messing around with module code (repeating for every module for every upgrade), but are you suggesting that where a module is usually only open to use by Admin, that a LITE version could be useful, and if applied to a number of modeules, would amount to a really useful saving?

I write as someone who has often faced memory issues on shared servers...

Ian Dickson

Likal.com

atuyo60’s picture

mm.. ya, if a certain code is meant for the admin pages, it should work fine excluding from the main module file. The savings can really be substantial if you are close to the limit which results in blank pages at the admin area.

module makers don't have to maintain multiple versions, they just have to alter their next release to exclude certain code from the main file as not all the code needs to be loaded on every view... I am not very sure how it works too maybe the article's author would know more but it has help me pushed down my memory use by quite a lot although altering the code ourselves can cause problems since we are not familiar with how the module works internally, that's why the creators should do this instead of end-users doing the alteration..

kbahey’s picture

Actually, there are two points here.

The technique advocated by Wesley is to break modules into submodules, and conditionally load the code you need via include.

There are two points here:

1. If you run an op-code cache or accelerator (APC, eAccelerator), this becomes a moot point. What the accelerator will do is pretokenize your modules and store the code cache in memory, then requests for the module will be served from memory which is very fast, and very small (no parsing or compilation is needed). You should be using accelerators if you are on a VPS or dedicated server and/or your site has considerable traffic. Without an accelerator (and even in some cases, with certain APC options), doing an include_once is an expensive operation since it is a disk access, so the gain you get with memory reduction may result in more execution time for the page.

2. In Drupal 6.x, the new menu system allows you to specify the functions associated with a path, and the file to load for that. All core modules have been split up, and some measurement (from August) shows that modules are indeed smaller than Drupal 5 when loaded in memory (details on 2bits.com). It remains to be seen how much

I replied to Wesley here.
--
Drupal performance tuning, development, customization and consulting: 2bits.com, Inc.
Personal blog: Baheyeldin.com.

--
Drupal performance tuning and optimization, hosting, development, and consulting: 2bits.com, Inc. and Twitter at: @2bits
Personal blog: Ba

atuyo60’s picture

wow, thanks for the insight! I am not really familiar with the servers and hosting stuff, I am on a shared host that limits my PHP memory so I have no other choice but to implement such a method or I would constantly get blank pages because of all the modules alone.

Wesley Tanaka’s picture

My personal experience without an accelerator has been that the disk access for reading in the unused code on every page view for every module is worse than the disk access for one or two extra include_once calls for the particular code that is needed.

Splitting modules into submodules could very well hurt performance if you have only a very few, small modules enabled.

But I've generally found it to be a good thing with large numbers of large modules enabled.

Hopefully this all becomes moot in Drupal 6.

Wesley Tanaka | Drupal Stuff