Gzip aggregated CSS and JS
| Project: | Drupal |
| Version: | 7.x-dev |
| Component: | base system |
| Category: | feature request |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | patch (code needs work) |
Jump to:
This is a follow on from #100516 - CSS preprocessor (and, originally #81835), which is a patch to aggregate multiple CSS files into a single (cached) file.
This patch (which should be applied on top of the #100516 patch):
- Adds an option to the settings to gzip this cached file.
- If enabled then it gzips and saves a .css.gz file in addition to the regular .css file and...
- Adds a .htaccess rule to use this file if the browser accepts gzip and the gz file exists.
As you can see, it is a very simple addition, but should be very valuable for heavy sites. Apart from the time saved for users, there is a 16Kb bandwidth saving too - potentially several GB of bandwidth a year for a site with several 100K visitors.
I currently have this flagged as a feature request for 5.0, but it could very well be considered a usability enhancement.
Here are my original benchmarks:
-----------------------------------------
<?php
HEAD
Total Transfer Duration Page Duration
Test 1 12798 8756 8454
Test 2 12984 8907 8162
Test 3 12730 8718 7966
Average 12837 8794 8194
Baseline 100% 100% 100%
conditional_css_include_2.patch
Total Transfer Duration Page Duration
Test 1 10633 7111 6828
Test 2 11225 7332 7530
Test 3 11047 7639 7357
Average 10968.33 7361 7238
Faster By 15% 16% 12%
cache_19.patch
Total Transfer Duration Page Duration
Test 1 9160 5453 5175
Test 2 9417 5712 4961
Test 3 9056 5306 5290
Average 9211 5490 5142
Faster By 28% 38% 37%
conditional_css_include_2.patch AND cache_19.patch
Total Transfer Duration Page Duration
Test 1 9595 5615 5333
Test 2 9909 5709 5425
Test 3 9461 5824 5537
Average 9655 5716 5432
Faster By 25% 35% 34%
cache_19.patch AND gzip
Total Transfer Duration Page Duration
Test 1 7429 3495 2741
Test 2 7027 3306 2758
Test 3 6915 3267 3001
Average 7124 3356 2834
Faster By 45% 62% 65%
conditional_css_include_2.patch AND cache_19.patch AND gzip
Total Transfer Duration Page Duration
Test 1 7489 3157 2395
Test 2 6699 3107 2364
Test 3 7250 3013 2255
Average 7146 3092 2338
Faster By 44% 65% 71%
?>* Testing was done locally, to eliminate the variable latency you get on live networks. The Charles web debugging proxy was used to apply a consistent throttle and latency equivalent to a typical 64Kbps connection.
* Browser caching was disabled completely, to simulate an initial page load. I can repeat, with caching enabled (to test http cache freshness checks) if that would be useful.
* Drupal page caching (css caching for #100516) was enabled, and the test was done as an anonymous user. All Drupal modules were enabled (a few of these would more likely be contrib modules in reality). The page cache was cleared before each set of tests, and 2 dummy reloads were made before starting timing.
* The first column 'Total' is the total time spent transferring data, as reported by Charles. This does not include the time saved by browser pipelining of http requests.
* The second column 'Transfer Duration' is the Duration between the first byte transfer and the last byte transfer, as reported by Charles
* The third column 'Page Duration' is the time between the end user hitting refresh and Firefox finishing building the page. This is occasionally less that the transfer duration, which is a little odd, but perhaps certain page graphics (favicons maybe?) are not included in the page build time.
Note that these percentages are compared to the times in my original HEAD benchmark (with no patches) at http://drupal.org/node/100516#comment-162025
Please read the notes on that comment if you have not done so already. Also note that all the conditional css includes patch is doing here is somewhat reducing the cached css size, because none of the conditions are met on the front page.
Here is a summary:
<?php
Test Set Faster By: Seconds Saved:
HEAD 100% 0
conditional 12% 1
caching 38% 3
conditional + caching 34% 3
caching + gzip 65% 5.3
conditional + caching + gzip 71% 5.8
?>Looking at this I am actually wondering if we should be looking at including gzipped css (and js) in core - if not for 5.0 - then certainly in 6.0. While the percentages are skewed by the fact that my test page is pretty lightweight (i.e. not much content, and no user added images) the seconds saved are real, actual seconds that would be saved by a user on a 64Kbps connection, and would be saved no matter what is added to the site and theme in the way of content and additional images. Also note that the conditional includes patch has now been committed to HEAD, but wasn't when I did my initial benchmarks - I am keeping it separate here for easier comparison.
Bearing in mind the statistic that most users will only wait 4 seconds before going to a different site, the application of aggregation/caching and gzip can take the initial page load time from a very poor 7 or 8 seconds, to a very respectful 2 seconds. The difference between these patches is extremely noticeable, the site goes from feeling pretty sluggish to appearing extremely fast.
| Attachment | Size |
|---|---|
| gzip_css_1.patch.txt | 2.62 KB |
#1
Personally, I think this is critical. Initial page load time is a critical factor for first time visitors. This one almost cuts the initial load time in half. Very significant!
#2
I just noticed a typo in the benchmark tables - 'Faster By' for HEAD should of course be 0% (or N/A!), not 100%.
#3
we have outstanding bugs on our gzipped page cache (if first person to view a page has gzip disabled, we still cache that page and send wrong headers). i'd rather see that resolved before we add more gzip.
the implementation here is a bit more interesting since the logic is in .htaccess. is there no performance penalty for this .htaccess check on every request?
also, don't we have to tell the browser that we are sending gzip in the response headers? are we expecting browser to know because filename ends in .gz?
the #description doesn't explain anything about gzip
#4
Subscribing... this should handle CSS and JS files...
#5
There is a different (better, I think) approach in my sandbox: http://cvs.drupal.org/viewcvs/drupal/contributions/sandbox/grugnog/gzip_... - which should deal with css, js and a few other things too.
I think moshe was right about the headers (in terms of following specs), so the .htaccess option is probably out (although it did work cross-browser).
#6
With 4 weeks to go, let's see if we can get this in :-)
Owen, did you want to start with a patch based off of your module? If not, I can try and post one in the next few days.
#7
#8
Subscribing.
#9
Since JS aggregation is now in Drupal 6 core as well, I suggest we tackle both CSS and JS aggregation here.
#10
subscribing...
#11
Subscribing.
#12
This had huge potential but unfortunately didn't make it into 6.x. What's holding up the progress on this patch? Is it because of the outstanding bugs on gzipped page cache in core?
Also I agree with Grugnog2 in #5 about the use of .htaccess method, while it is browser compatible, Dries highlighted his desire to make core compatible with lighttpd and other web servers. We should put some consideration for those too if possible.
#13
On my site a typical page is around 100kb in size. About 75kb of that are aggregated js (50kb) and css (25kb) files, which can be compressed down to about 25kb (with stripped comments 17kb).
Even without stripping comments it would cut it in half for first time visitors. And you can get a whole torrent of those from social bookmarking sites. My puny little page gets about 3000 of those occasionally and I'm currently on some cheap shared host. So, it generates about 300mb instead of 150mb. That's a pretty big difference if you ask me.
Since the number of visitors is constantly rising I really would love to see that kind of thing in core ASAP. Otherwise I'll need to switch to some more expensive hosting plan a few months sooner than actually necessary.
#14
subscribing
#15
This topic is pretty hot nowadays, I read two related discussions on contrib modules for 5.x: this one minify the Javascript aggregated file, and this is about "statical" gzipping method. Today I start to use both the approaches combined, and the result is fine. Since aggregation is a core task, I leave this to a small bash script run on cron job. Should JSMin patch to Javascript Aggregator be reused here?
#16
Got tired of waiting and just added it to my installation. Only requires 14 extra lines in .htaccess and 2 in common.inc. These 2 lines need to be added again after updating, but imo it's totally worth the hassle.
The step by step guide is over here:
How to GZip Drupal 6.x's aggregated CSS and JS files
#17
It seems that Zlib support isn't standard for a PHP installation, at least not on non-Windows OSes. Can someone confirm this?
If this is the case, it might stand in the way of adding this.
#18
Surprisingly you're correct. It's silly, but there are indeed a few installations without Zlib support.
If it's implemented as an option, it would be a good idea to guard the setting (defaults to off) with a function_exists('gzencode') condition.
Fortunately the level parameter for gzencode exists since 4.2 and Drupal requires 4.3.5+. So, at least that part won't be an issue.
#19
Well, Drupal 7 requires 5.2+, so that's definitely not an issue. I would think this would need to be implemented like Clean URLs, where the option to enable gzip encoding would be disabled if the function doesn't exist with an explanation of why.
How about a reroll as an option in the Performance settings page?