Serving Gziped files
| Project: | Javascript Aggregator |
| Version: | 5.x-1.3 |
| Component: | Code |
| Category: | feature request |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | postponed |
Jump to:
Hi,
I'm not a coder or Apache expert but from what I've been reading recently, here are my suggestions for improvement:
I think it would be great if this module could also create a Gziped version of the js aggregation file, and store it in the same files/js folder. And then, serve the compressed js.gz file rather than the big js file.
Once that is done, in .htaccess file one needs to add something like the following code:
<FilesMatch "\\.js.gz$">
ForceType text/javascript
Header set Content-Encoding: gzip [maybe this could be set to Footer also]
</FilesMatch>
<FilesMatch "\\.js$">
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} !".*Safari.*"
RewriteCond %{HTTP:Accept-Encoding} gzip
RewriteCond %{REQUEST_FILENAME}.gz -f
RewriteRule (.*)\.js$ $1\.js.gz [L]
ForceType text/javascript
</FilesMatch>(I have taken this from this blog entry)
From what I understand this htaccess command lines are telling to preference the js.gz rather than the uncompressed js file.
I would do it myself if this module would create only 1 aggregation file, but it creates many different aggregations.
There is also project JSmart that can be helpful (see http://farhadi.ir/jsmart.html) . See also this recent comment: http://drupal.org/node/11128#comment-813518
What is the benefit of doing this?
1) The first time that the JS aggregate is served would now be minimized up to 80%, because it is serving a Gziped version of that file! (and as we know the first impression is always important to the visitors)
2) If you enter a simple .htaccess command like
AddOutputFilterByType DEFLATE text/html text/css text/plain text/xml application/x-javascript application/json
Header append Vary Accept-Encoding... that's NOT good enough!!! because it will ask the Apache server to compress into a new Gzip file the JS and other files, on every request. Firstly, this is potentially slow. Secondly, and importantly, this can become a big load on the server CPU , which is why many hosting companies (like my Hostmonster) disable mod_deflate and mod_gzip (thus making it impossible to Gzip components and get higher scores on YSlow)
But if a compressed Gziped version of JS aggregated file can be created (using JSmart or any similar solution) alongside the original JS aggregation, AND if in .htaccess file it can be told to serve the Gziped version and not the original uncompressed version, then it seems like this coould be a pretty neat solution!
* it would not be as CPU intensive
* it would reduce the traffic volume
* it would provide a solution for many poor souls (like myself) who's host refuses to enable mod_deflate and mod_gzip (and quite understandably so)
* it would dramatically reduce the first time load (and give a good impression to the first time visitors)
any thoughts?
#1
subscribing ;-)
#2
seems to be a good idea... thanks for all the research that you've done... i'll take a closer look once i find the time... :)
i think this will not be trivial to implement... i also think that gzip support depends on the php configuration, right?
#3
Hi,
After posting the original request, I've tried many options and did a LOT of reading around the internet.
(please see my comment here http://drupal.org/node/248909#comment-818261 )
Just to test if it would work, I *manually" gziped all the aggregated files inside /files/js and /files/css (using WinGzip). And inside .htaccess I inserted these lines:
<FilesMatch .*\.js.gz$>
ForceType text/javascript
Header set Content-Encoding: gzip
</FilesMatch>
<FilesMatch .*\.css.gz$>
ForceType text/css
Header set Content-Encoding: gzip
</FilesMatch>
RewriteEngine On
RewriteCond %{HTTP:Accept-encoding} gzip
RewriteCond %{HTTP_USER_AGENT} !Safari
RewriteCond %{REQUEST_FILENAME}.gz -f
RewriteRule ^(.*)$ $1.gz [QSA,L]
Results? The website was flying! Now that the aggregated JS files that weighed 160kb were reduced to 30kb, the pages were loading upto 6 times faster!!! I wouldn't call this much difference "trivial".
The only drawback is that these aggregated .js files are constantly being rewritten. And creating a gzip version of each of those files mannually every time the cron runs, would drive anyone insane. So they must be created automatically.
For anyone with a dedicated server with many powerful CPUs this solution would not be urgent. But for everyone else hosting on a shared packade with 500 other websites hosted on the same box, this kind of feature would be like water to survive.
I don't quite know what you mean by php configuration...
Normally gzip support depends on whether the hosting company has it enabled on their Apache. *Most* hosting companies disable mod_gzip and mod_deflate .
However, if there's a gzip feature running from inside drupal, that shouldn't be a problem at all.
#4
Hi drupalina, interesting...
by trivial i meant it will be quite a lot of work... but i have a project that might need this feature as well, so i will look into the details now...
thats my concern: im not quite sure if this gzip feature is something that will work on every server... what i think is that this gzip support depends on the php configuration (also done by the hosting companies)...
but 160kb to 30kb is something thats worth investing the time...
#5
it's worth looking into JSmart http://farhadi.ir/jsmart.html -- it's an external project -- not related to Drupal!!!
It's designed precisely for sites that run on sites where the hosts have the Apache mod_gzip and mod_deflate disabled.
And I'm sure there are lots and lots of similar open-source tools that can gzip files from inside drupal installation without being restricted by Apache settings.
I haven't managed to getting JSmart fully running even on my localhost at home, but that's because I'm not an IT person (just someone who is not lazy enough to read documentations and do the necessary research)
------
Another (probably very BAD) solution would be to instruct JS aggregator module not to create new aggregation files inside the /files/js folder unless new modules containing JS are added or removed. But this probably a bad idea.
If you come up with watever solution, I'd be happy to test it out on Hostmonster/Bluehost
#6
It is enough to add to .htaccess
mod_gzip_on Yes
mod_gzip_item_include file \.js$
mod_gzip_item_include file \.css$
#7
Impressive approach. Certainly a win with these layers of js scripts I'm looking like needing for all the add-ons we want nowadays.
#8
@Rusland -- the lines that you provided are definitely not the solution. They gave me a Server Error.
Additionally, you seem to miss the main point of this issue and the need to serve ready gzipped files (rather than Gzip them upon every request). Most hosting companies disable mod_gzip !!! So the files must be gziped as a drupal application rather than as an Apache application.
#9
check out this post .... http://drupal.org/node/251140
#10
Subscribing. Optimization (minify via JSMin) plus Gzip can be the best approach.
Since this module generate a UNIQUE_STRING.js file, compression should be placed not on cron run, but just after UNIQUE_STRING.js generation, something like UNIQUE_STRING.js.gz. The Apache rules should be changed to look at UNIQUE_STRING.js.gz when UNIQUE_STRING.js is called.
Minifying thin my aggregated Javascript file from 63,7 KB to 48,8 KB.
Gzipping the minified version results in a 17,6 KB file against 21,6 kB of an only-gzip approach (using best compression rate, 9, when usually apache use a value from 3 to 6).
So, both approaches are useful, and should be combined on the aggregation phase, minifying and then compressing the aggregated file.
#11
Who have cron job privileges on a *nix host can use this simple bash script to gzip aggregated files.
<?php#!/bin/bash
# gzip aggregated files (on cron run)
# cfr. http://drupal.org/node/249211
#
cd /absolute/path/to/drupal/files
# run on files/css and files/js directories only
for aggdir in "css" "js" ; do
cd "$aggdir"
for file in `dir -d *.js *.css` ; do
# if gzipped file doesn't exists, create it
if [ ! -f "$file".gz ]
then
gzip --best -c "$file" > "$file".gz
chmod 755 "$file".gz
fi
done
cd ../
done
# NOTE: THIS IS NOT PHP CODE ;-)
?>
Note: This script relies on unique filenames generated by Javascript and CSS aggregators. So, a gzipped file will be generated only one time and aggzip.sh doesn't refresh the .gz file. This should reduce CPU usage to minimum, compressing only newly aggregated files.
#12
If there is PHP you can GZip whatever you want. I just added a line of code which outputs the same file again - just gzip compressed (via gzencode) and with a ".gz" file extension.
Check my step by step guide for details:
How to GZip Drupal 6.x's aggregated CSS and JS files
#13
Without Zlib support you can't, and hacking the core IMHO isn't the preferable way to implement aggregation in this phase. A supported core patch will solve this issue, but for now modifying .htaccess and creating a shell script could be the more portable solution on some hosting provider, keeping out of core these modification.