Hi,
This is a pageload optimization question.

I've installed YSlow (Yahoo's Firefox plugin, and I'm trying to optimize my website.

So far I'm getting an "F" for Gzip components (this is the default Drupal installation). But I noticed that drupal.org is getting higher marks for that: D or C... and sometimes B. I wonder how Drupal.org has achieved the Gzipping of it's components?
And I wonder what is the best and "cleanest" way to Gzip components?

This article suggests the following alternative:

As an alternative, you could configure your Apache server to automatically compress files.
An example for Apache 2.x: add the following lines to your .htaccess or httpd.conf file:

AddOutputFilterByType DEFLATE text/css application/x-javascript

Does that mean that I have to open my .htaccess file and just put it there and Gzipping is gonna work automatically???

please help...

Comments

goldschmidt.a’s picture

I'm trying to figure out the same thing.
I tried putting that code AddOutputFilterByType DEFLATE text/css application/x-javascript in a new line at the bottom of my .htaccess file, and got an error on my site's main page:

Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, support@supportwebsite.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.
More information about this error may be available in the server error log.

I think this would work if I had an Apahce 2 server, but unfortunately I think I have an Apache 1.3.33 server.

I also tried putting this bit of code in a new line at the bottom of my .htaccess file (from Paul K Egell-Johnsen's comment on http://buytaert.net/yslow):

<IfModule mod_deflate.c>
AddOutputFilterByType DEFLATE text/html text/plain text/xml application/javascript text/css text/ecmascript text/javascript application/ecmascript
</IfModule>

I didn't see any changes on my YSlow gzip rating. I don't know much about the 'deflate' vs. 'gzip' functionality though, but no change in loading time.

Tried looking at various sources already:
http://developer.yahoo.com/performance/rules.html#gzip
http://wimleers.com/article/improving-drupals-page-loading-performance#r...
http://buytaert.net/yslow
http://drupal.org/node/238632

Can't figure it out yet.
Using Drupal 6, but should apply to Drupal 5 and 6 according to http://wimleers.com/article/improving-drupals-page-loading-performance#r....

Does anyone else know how to successfully gzip files to speed up site's loading time?

Thanks,
Andrew G

drupalina’s picture

An update: Since the time that I posted the original post, I've done a LOT of reading and I have spoken to my host (hostmonster). It appears that Hostmonster, BlueHost as well as many other hosts disable mod_deflate and mod_gzip functionalities on their Apache, because it's heavy of the server CPU.

So the lines mentioned above as well as on Dries Buytaert's blog are fine and will work as long as your Hosting company allows mod_deflate and mod_gzip on their servers.

I was guttered!!! But I can understand the reason why hosting companies disable mod_deflate and mod_gzip: the problem with Buytaert's line (and the one that I mentioned above) is that it's not really an optimal solution -- IT IS TOO SERVER INTENSIVE! If that line works, than it will compress all the mentioned filetypes before sending them over EVERY TIME THAT THOSE FILES ARE REQUESTED. This means that the server is told to Gzip the same files again and again and again every time they are requested (this can become quite a load on server's CPU).

But why not gzip that file only once, and serve those gzipped versions thereafter??? :|

What would be a MUCH-MUCH better solution (in terms of server CPU load, and traffic volume reduction, and Speed) is if there was a module that would automatically Gzip every new .css file inside /files/css folder and every new .js file inside /files/js folder (assuming that you're using the JS Aggregator module) . And then Apache can be easily instructed to serve the .css.gz and .js.gz files rather than the originals.
I've tried gzipping these files *manually* with WinGzip and in .htaccess I instructed Apache to serve the gziped files instead of the original uncompressed ones. And let me tell you, the speed of the pageloads increased DRAMATICALLY !!!! The first empty-cache pageload was in 6-8 seconds, while subsequent page loads were in 1.1-2.4 seconds

(the created html files was still being served uncompressed)

The major drawback, however, is that new aggregated .css and .js files are being written after every cron run. So creating and uploading new gzipped versions of those files *mannually* would drive you insane (especially if your cron is running every 15 minutes).

But the fact remains: that IF there was a nice little automated Gzipping module that would gzip those .css and .js files automatically inside those /files/css and /files/js folders, that would be a MAJOR-MAJOR improvement in terms of pageload optimizations. Especially for the poor souls who can't afford a dedicated server.

What would also be cool is if that new module would Gzip the generated html content before serving it from inside Drupal, so that the hosting companies don't have to worry about enabling mod_gzip and mod_deflate on their Apache.

A very similar project that is doing pretty much all this is JSmart (which is not part of Drupal). But I could never get it to work :(

Any drupal coding gurus want to take the challenge of writing something like this module???

gremlinc5’s picture

Hi, I too had the same issue and solved it with JSmart.
It's a great tool, but has to be tweaked a little to fix our (mine?) needs.

To have it working, your server MUST be running mod_rewrite. There are some host that disabled mod_rewrite also. In that case, you're out of luck. Sorry.

You have to change the .htaccess file to redirect every single *.js or *.css request to the JSmart script. Add the following:

<IfModule mod_rewrite.c>
  RewriteEngine on

  #JSmart Condition
  RewriteRule ^(.*\.((js)|(css)))$ /jsmart/load.php?file=$1
</IfModule>

Notice that (probably) you already have everything in place but the RewriteRule. The jsmart directory is where you are uploading JSmart's load.php file.

Next, I found that minification/optimization on JSmart is very buggy, so I removed it. Use YUI Compressor, Packer or your tool of choice.

Because my server is running php inside CGI, the php getallheaders() function isn't available.
I substituted it with an equivalent php loop (see code).

I also added an Expire header to 60 days in the future (just to satisfy YSlow).

I'm posting the stripped down version of load.php I'm using.

/jsmart/load.php file:

<?php
/* JSmart v1.0 Final Release
 * Copyright (c) 2006 by Ali Farhadi.
 * Released under the terms of the GNU Public License.
 * See the GPL for details.
 *
 * Email: ali@farhadi.ir
 * Website: http://farhadi.ir/
 */
/*
 * ACA: Edited to disable minification and support PHP as CGI
 */

//Encoding of your js and css files. (utf-8 or iso-8859-1) 
define('JSMART_CHARSET', 'utf-8');

function header_exit($status) {
	header("HTTP/1.0 $status");
	exit();
}

function header_nocache() {
	// already expired
	header("Expires: Mon, 26 Jul 1997 05:00:00 GMT");
	// always modified
	header("Last-Modified: " . gmdate("D, d M Y H:i:s") . " GMT");
	// HTTP/1.1
	header("Cache-Control: no-store, no-cache, must-revalidate");
	header("Cache-Control: post-check=0, pre-check=0", false);
	header("Cache-Control: max-age=0", false);
	// HTTP/1.0
	header("Pragma: no-cache");
	//generate a unique Etag each time
	header('Etag: '.time());
}

function debug_exit($msg){
	//Show error messages if any error occurs (true or false)
	define('JSMART_DEBUG_ENABLED', true);
	if (!JSMART_DEBUG_ENABLED) {
		header_exit('404 Not Found');
	}
	header_nocache();
	header('Content-Type: text/html; charset='.JSMART_CHARSET);
	header("Content-Encoding: none");
	echo "//<script>\n";
	echo "alert('JSmart Error: ".($msg)."');\n";
	echo "//</script>\n";
	exit();
}

$current_path = addslashes(realpath(getcwd()));

$file = @$_GET['file'] or debug_exit('missing file parameter');
if (!preg_match('/\.((js)|(css))$/i', $file, $file_type)) debug_exit($file." is not a javascript or css file.");
$file_type = strtolower($file_type[1]);

$file = constant('JSMART_'.strtoupper($file_type).'_DIR') . $file;

if (!file_exists($file)) debug_exit($file." not found.\\n(the path is relative to $current_path)");

$mtime = filemtime($file);

if ($mtime < filemtime('load.php')) $mtime = filemtime('load.php');
$mtimestr = gmdate("D, d M Y H:i:s", $mtime) . " GMT";

// ACA: getallheaders not supported via CGI
//$headers = getallheaders();

// ACA: inlined function for getallheaders
foreach($_SERVER as $name => $value)
    if(substr($name, 0, 5) == 'HTTP_')
        $headers[str_replace(' ', '-', ucwords(strtolower(str_replace('_', ' ', substr($name, 5)))))] = $value;

header('Content-Type: '.($file_type=='js'?'application/x-javascript':'text/css').'; charset='.JSMART_CHARSET);

if (isset($headers['If-Modified-Since']) && $headers['If-Modified-Since'] == $mtimestr) 
	header_exit('304 Not Modified');

// ACA: add expires header
$offset = 1296000; // 60 * 24 * 60 * 60 seconds = 60 days
header("Expires: " . gmdate("D, d M Y H:i:s", time() + $offset) . " GMT");

header("Last-Modified: " . $mtimestr);
header("Cache-Control: must-revalidate", false);

$gzip_supported = 
	(isset($headers['Accept-Encoding']) &&
	array_search('gzip', array_map('trim', explode(',' , $headers['Accept-Encoding']))) !== false &&
	function_exists('gzencode'));

$cached_file = 'cache/'.md5($file).'.'.$file_type;

if ($gzip_supported) {
	header("Content-Encoding: gzip");
	$cached_file .= '.gz';
}

if (file_exists($cached_file) && $mtime < filemtime($cached_file)) {
	@readfile($cached_file) or debug_exit("Cannot read file ($cached_file)\\n(the path is relative to $current_path)");
} else {
	$file_contents = @file_get_contents($file) or debug_exit("Cannot read file ($file)\\n(the path is relative to $current_path)");
	
	if ($gzip_supported) $file_contents = @gzencode($file_contents, 8) or debug_exit("Gzipping failed");
	
	$handle = @fopen($cached_file, 'w') or debug_exit("Cache dir is not writable or doesn't exist (cache/)\\n(the path is relative to $current_path)");
	fwrite($handle, $file_contents);
	fclose($handle);

	echo $file_contents;
}
?>

Note: I have no relation to the original author of JSmart. I just modified his source and have no rights on it, but the project got discontinued 2 years ago, so there shouldn't be problem.

Comments marked as "// ACA" are mine. Put this into jsmart directory on your site root and have fun! You don't need the original config.php (I included what was there into load.php). On my host (Register.it), it works like a charm.

You just have to put load.php in jsmart dir and create a cache dir inside it (/jsmart/cache/). Give them proper access rights (check write permissions).

Bye!

drupalina’s picture

Hi,

I couldn't get this to work.

I put the jsmart folder in the root.

I deleted the config.php file

I totally rewrote the load.php file with the code that you wrote above.

I gave 777 permissions to jsmart/cache folder (and all the files that might be in it)

I rewrote my Rewrite secion of .htaccess with you code....

and then I go to the site and it's broken... it says that redirecting was not done properly. After I uploaded my old .htaccess file everything became normal again. But obviously JSmart didn't get working.

I et the feeling that something was clashing inside my .htaccess .. If I e-mailed you my .htaccess file, would you be able to have a look at it?

Thanks...

PS: Here are my Performance settings: my caching is set to Normal and my CSS is also being cached. I'm also using JS Aggregator module with JSmin

gremlinc5’s picture

Hi,
to check if it's a JSmart problem or an .htaccess problem, insert in the navigation bar of your browser http://yoursite/jsmart/load.php?file=path_to_js_or_css_file.
E.g. http://example.com/jsmart/load.php?file=/misc/drupal.js
(check http://yoursite/misc/drupal.js just to be sure address is correct)

It should give you the js file (if you check with YSlow it should have delivered it compressed).

If it gives you an error page, chances are that .htaccess is ok and something with JSmart is broken.

If the file is sent, JSmart is working properly and the only problem is on .htaccess.

FYI, my .htaccess mod_rewrite section actually has the following configs (I'm on Drupal 6):

<IfModule mod_rewrite.c>
  RewriteEngine on

  # If your site can be accessed both with and without the 'www.' prefix, you
  # can use one of the following settings to redirect users to your preferred
  # URL, either WITH or WITHOUT the 'www.' prefix. Choose ONLY one option:
  #
  # To redirect all users to access the site WITH the 'www.' prefix,
  # (http://example.com/... will be redirected to http://www.example.com/...)
  # adapt and uncomment the following:
  RewriteCond %{HTTP_HOST} ^example.com\.it$ [NC]
  RewriteRule ^(.*)$ http://www.example.com/$1 [L,R=301]
  #
  # To redirect all users to access the site WITHOUT the 'www.' prefix,
  # (http://www.example.com/... will be redirected to http://example.com/...)
  # uncomment and adapt the following:
  # RewriteCond %{HTTP_HOST} ^www\.example\.com$ [NC]
  # RewriteRule ^(.*)$ http://example.com/$1 [L,R=301]

  # Modify the RewriteBase if you are using Drupal in a subdirectory or in a
  # VirtualDocumentRoot and the rewrite rules are not working properly.
  # For example if your site is at http://example.com/drupal uncomment and
  # modify the following line:
  # RewriteBase /drupal
  #
  # If your site is running in a VirtualDocumentRoot at http://example.com/,
  # uncomment the following line:
  RewriteBase /

  # Rewrite URLs of the form 'index.php?q=x'.
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_FILENAME} !-d
  RewriteRule ^(.*)$ index.php?q=$1 [L,QSA]
  
  #JSmart Condition
  RewriteRule ^(.*\.((js)|(css)))$ /jsmart/load.php?file=$1
</IfModule>

(I substituted my site name with example.com).
I have rewrite rules for the non-www address, for pretty addresses and for JSmart, and a RewriteBase rule pointing to / (drupal is installed in my root folder).

Note the RewriteBase directive: if you have "RewriteBase /drupal" then you have to put JSmart folder under drupal directory (/drupal/jsmart). If there isn't, try to insert a RewriteBase directive (either to /drupal or to / or your custom folder).

Feel free to mail me your .htaccess file (at chiesa [dot] alberto [at] gmaildotcom), but I'll be away until next Monday, so I will not check your file until then.

Good luck!

gremlinc5’s picture

Reviewing load.php I noticed a potential issue:

on line 56 on load.php:

$file = constant('JSMART_'.strtoupper($file_type).'_DIR') . $file;

the "constant()" part has to point to your Drupal root.
So change it in:

$file = "/" . $file;

If Drupal is on the root of your site.

I think it's working on my site because it was never restarted on last 3 months... sorry.

I hope I'll be releasing a more user-friendly (and hopefully bug-free) version of JSmart as a Drupal "module" next week. (the " because, in fact, JSmart works on a different level of Drupal and is completely independent from it). I just make it work for my site and did not worried about making it configurable by others. It's just 100 lines of code, so it's not complicated, but having all the configuration options near the top of the file would be better.

Let me know if there are any news.

Gotta go on holidays, now. Be back next week.

Bye!

gremlinc5’s picture

To address this issue, I created a Drupal module based on JSmart.

It's called SmartCache and you can find it here.

farhadi’s picture

Hi all,
I am the author of JSmart.
I have released a new version of JSmart and renamed it to "SmartOptimizer" that is more efficient than JSmart.
Check the project page for more information.
By the way, I have wrote a how-to for installing SmartOptimizer on drupal that you can see it here.

Rob T’s picture

SmartOptimizer is working well for me. Very easy to install... my YSlow score jumped 12 points.

mimetic2’s picture

Hi,

For some reason my components are gzipped when i'm not logged in but when i am logged in its not gzip. Does anyone know the culprit?

edit: using mod_deflate in apache.

SocialNicheGuru’s picture

I ran Yslow and am now trying to figure out how to effectively gzip items

http://SocialNicheGuru.com
Delivering inSITE(TM), we empower you to deliver the right product and the right message to the right NICHE at the right time across all product, marketing, and sales channels.

SocialNicheGuru’s picture

I ran Yslow and am now trying to figure out how to effectively gzip items

http://SocialNicheGuru.com
Delivering inSITE(TM), we empower you to deliver the right product and the right message to the right NICHE at the right time across all product, marketing, and sales channels.

Mikhailov Anatoly’s picture

Mod_deflate in apache2 is pretty much the same as mod_gzip in apache1.3, and mod_deflate is included with the apache2 source package. Both modules allow compressing of the apache server on the fly

http://railsgeek.com/2008/12/16/apache2-httpd-improving-performance-mod_...