PHP 5.2 introduced a marvelous little function called memory_get_peak_usage(), that returns the maximum amount of memory consumed during the request.
Thanks to the apache_note() function, we can pass down that information to Apache, and have it logged in the request log!
I tested this on a very simple test script (which only outputs "hello" and send the memory usage to Apache): the performance impact of that is *unmeasurable*, even in a bare script like this one. So I suggest we output that information unconditionally (patch attached).
Usage instructions
Add the %{x-drupal-memory-usage}n directory to your logging configuration. To do this, teplace the access log configuration in your Apache configuration (the main configuration or a Virtual Host) by something like:
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\" %{x-drupal-memory-usage}n" combined_memory
CustomLog /var/log/apache2/access.log combined_memory
| Comment | File | Size | Author |
|---|---|---|---|
| #1 | log-memory-peak-usage.patch | 902 bytes | damien tournoud |
| log-memory-peak-usage.patch | 903 bytes | damien tournoud |
Comments
Comment #1
damien tournoud commentedFixed spelling.
Comment #2
Anonymous (not verified) commentednice patch! this will help to with patches like #325665: Cache registry lookups, where its clearly faster with the patch but we want to make sure we're not introducing regression with regard to memory usage.
i applied the patch and setup a custom log format as per the docs, and everything worked as expected. simple patch, so RTBC.
Comment #3
kbahey commentedI am not sure I want to mess with Apache configuration to get this feature on every site/host I use.
But the way, the devel module has a new module in it called "performance" which logs peak memory usage, either in detailed (per page) mode, or as a summary (aggregate across all accesses to a given URL). It logs to the database or to APC (no performance impact).
It also longs milliseconds per page, number of queries, and more.
This is available in the -dev version for D5, D6 and D7.
Should help with the regression part.
Comment #4
damien tournoud commentedWhich "regression" are you talking about?
Comment #5
kbahey commentedRegression testing: when a patch needs to be assessed for memory usage (up or down).
Comment #6
dries commentedMaybe this should be written to the accesslog table instead? If you enable the access logs in the statistics.module, then the "Top pages" view already provides some information that seems relevant. It would certainly be easier to use than apache logs imo.
Comment #7
dries commentedSetting back to 'code needs review' so we can discuss this some more. Thanks.
Comment #9
lilou commentedSee: #335122: Test clean HEAD after every commit and http://pastebin.ca/1258476
Comment #10
moshe weitzman commentedaccess log in drupal is currently unusable for medium to large scale sites. i think apache log makes sense even if it is harder to get at the data.
Comment #11
kbahey commentedIf we make it go to watchdog, the we have options.
- Starting with Drupal 6.x, core has support for logging watchdog to syslog.
- Also, the http://drupal.org/project/logging_alerts project has support for logging watchdog to the web server's log.
So, I say make it a watchdog thing, make it optional, and we are home free.
Comment #12
catchThat sounds better to me.
Comment #13
kbahey commentedThere is another thing that has been since this issue was opened: the devel module includes a performance logging module that logs peak memory usage for each page request. It either logs details (an entry for every page), or summary (a max and average across requests to the same page). It can log either to the database or to APC (in memory).
Check that out too.
Comment #14
damien tournoud commented