Cache with zlib results in double compression

Sounds good. Are you planning on posting a patch?

Subscribing

Here's another 'victim' of this bug, after the 4.7 'fix'
http://ivrpa.org (live site, cache disabled)

Drupal 4.7.6
PHP Version 4.4.2
Apache Version Apache/1.3.34 (Unix) PHP/4.4.2 FrontPage/5.0.2.2510
mod_zlib version 1.1.4

zlib enabled in .htaccess

php_flag zlib.output_compression On
php_value zlib.output_compression_level 5 

Nothing special in settings.php

Drupal 5.1
Apache/1.3.34 (Debian) PHP/5.2.0-8+etch1 mod_fastcgi/2.4.2

ZLib Support enabled
Stream Wrapper support compress.zlib://
Stream Filter support zlib.inflate, zlib.deflate
Compiled Version 1.2.1.1
Linked Version 1.2.3

Directive Local Value Master Value
zlib.output_compression On Off
zlib.output_compression_level 2 -1
zlib.output_handler no value no value

mod_gzip disabled.

I can't repdroduce this problem, tried with several clients (ff, wget, lynx).

I am wondering if this is related to http://drupal.org/node/111697

Yesterday I turned on cache and today some user(s) have problem - IE suggest to download file... actually this file is gzip-archive with HTML inside.

On my system IE one time also do so... and I turned off Drupal cache, refresh page, turn on Drupal cache again. After that problem on my system is disappeared and now I can't reproduce this 'feature' again...

At first for stability I make dirty hack of bootstrap.inc: change 532 line:

if (@strpos($_SERVER['HTTP_ACCEPT_ENCODING'], 'gzip') === FALSE && function_exists('gzencode')) {

to

if (function_exists('gzencode')) {

But now I devise other method w/o hack: add to sites/default/settings.php line:

unset($_SERVER['HTTP_ACCEPT_ENCODING']);

This line forces Drupal cache system turn off compression.

PS: Sorry for my english

In interesting proposal. I guess this comes down to looking at if an average server is likely to be CPU bound so as to make an impact if all pages are gzipped without caching - a tricky call to make.

I confirm I experience this problem with MSIE 6, but not with firefox. MSIE users will see ASCII characters when I add php_flag zlib.output_compression On in htaccess.

I meant MSIE 7, not 6. And I have drupal 5.1.

I have found very interesting function, which solve this annoying problem. It is _decodeGzip function written by Richard Heyes in PEAR Package: HTTP_Request v.1.4.0 http://pear.php.net/package/HTTP_Request/

"The real decoding work is done by gzinflate() built-in function, this method only parses the header and checks data for compliance with RFC 1952"

I check this and it works for me. No more random text or "Warning: gzinflate(): data eror".

I have reproduced error with random ASCII text (or with gzinflate warning) when I add unset($_SERVER['HTTP_ACCEPT_ENCODING']) to settings.php (suggestion #10 by edhel).

Patch is dirty and need to be better ported to Drupal.

thanks kndr.

all that nasty code is probably whats needed in order to safely perform gzip from within a php app like ours. i always thought, and still think, we should leave this to webserver, even at the expense of some extra burned cpu cycles (for those who want compression).

Fix the indentation.

Unfortunatelly last patch with function _decodeGzip isn't solution. Today, I've got random ASCII text instead of page content. I tried turn off zlib compression. When I did it in settings.php, I got blank pages (which were previously cached) but when I set php_flag zlib.outputcompression to off in .htaccess file, page's content looks good. I don't understand this. Very strange.

kndr, can you post your system's configuration?

Adding some logging would be helpfull too to finally understand this issue.

I don't know what kind of configuration is important for you. My webpage runs with Drupal 5.1 on webhosting environment:

Debian GNU/Linux
Apache 2.0.54
PHP 4.4.1
eAccelerator 0.9.3
Zend Optimizer v2.6.0
zlib 1.2.2

I've got blank pages with FF 2.0 and IE 6.0 but now, when I turned off zlib in .htaccess everything looks good.

It seems like cached gzip serving could quite easily be pushed out to a contrib, what with our pluggable compression.

Not 100% sure how I feel about this, but I guess small sites generally don't get enough hits, and bandwidth is cheap enough nowadays that you would already be getting CPU warnings/bannings by the time you hit the bandwidth limits on a cheap shared host. Larger sites generally have more resources to install a pluggable gzip-ing caching engine (which could be mysql, memcache or file based) and deal with any obtuse server issues.

How does this sound to others?

Cache insertions check zlib_get_coding_type(), but cache gets do not. Maybe this is the problem?

I'd like to avoid any reorganizations, such as pushing this to contributions, in Drupal 5.x.

The code looks the same in Drupal HEAD, so bumping the version number.

Subscribing + Wondering if there is any progress on this issue?

For me, the only working solution was add "php_flag zlib.output_compression off" at the end of .htaccess file. Other attempts fell flat.

For me, +1 on that idea...

Is there a way PHP can query if Apache is trying to zlib the data? If so - why not disable Drupal's zlib-ing if Apache is doing it?

Poor little abandoned issue. Let's make this optional.

I think we need to explain the user what compressing does to your Drupal site, and why you might want to turn it on. The current form description assumes that the user is knowledgeable about this.

I'm still not 100% that an option is the best thing here.

I haven't tried this code - however it "mentally" compiles and the descriptions make sense. I hope this gets into core!

Question...

Would one of the large advantages of moving gzip compression from application (Drupal) to server (eg Apache or IIS) be that compression would apply to ALL pages rather than simply anonymous cached Drupal pages (as compression is an act done on content ready to be served)? This could therefore, in theory, cut bandwidth usage of HTML content by up to about 75%?

Yes but you do not always have access to the Apache config to set that up.

@chx: That is true - but in that scenario would the Drupal Compression not be better?

So the two situations would be:
1) Apache CAN gzip and you have access to the config to enable... Disabled Drupal GZip and enable Apache. This allows for compression on ALL pages, not just anonymous drupal cached pages.....

2) Apache CANNOT gzip - fall-back to default drupal behaviour - internally gzip (if available) for cached pages only.

I've committed a slightly modified version of this patch to CVS HEAD. Thanks.

I haven't had a chance to look at your patches, but does this disable zlib.output_compression inside the Drupal PHP code? I posted a new bug (which you can close since you are already fixing this), which (amongst other things) make sure zlib.output_compression is disable if the cache object is cached already. See http://drupal.org/node/187912.

The problem still exist if you select output_handler = ob_gzhandler.
Take a look here http://drupal.org/node/187912#comment-643604 i have submitted a patch that checks
if we are using output_handler or zlib.output_compression.

Hm, why reopen this when you have another issue for discussion?

subscribing...

This patch doesn't appear to fix the issue for me, and it's the same issue that's been causing me problems for almost 3 years. Cache misses from clients that don't support compression (like crawlers) generates a cached page that doesn't work for future requests from a browser that does support compression.

Also, unless it's been fixed/changed in PHP very recently, you can't disable zlib compression from within PHP code, just from the php.ini file. Output buffering has already begun, so it's too late to stop it in runtime. I've seen two or three patches against Drupal that attempt to change this setting to fix this bug.

Here's a suggestion for a fix: stop attempting compression within Drupal's code and leave it up to PHP or the web server. Is it really such a boost in performance to have to compress and decompress every page on its way in and out of the (database-based) cache?

I'm not sure which post is best for my issue but it seems to apply here:

I was running into a problem with Drupal Caching on causing IE6 to display blank pages: http://drupal.org/node/227294

The patch offered as a solution causes my pages to render as gibberish as posted in comment #32 by Caleb when accessing the front page with Safari, Firefox, or Opera with caching enabled.

As of now, the only solution that solves both is to disable caching. I'd like to be able to enable it. Is there a solution that fixes both issues?

I'm having issues with zlib double compression. I applied a patch in my previous post which I believe may have caused the double compression in FF but I'm not a coder so I'm not sure how to remedy the situation.

I'm on apache 1.3, drupal 5.10. Is there an easy way to disable zlib? I tried my .htaccess and it gave me 500 Internal Server Errors.

I also tried my sites/default/settings.php file with ini_set and that didn't work either.

I'm on Godaddy, I have access to my php.ini file on the site, but running phpinfo says that the loaded file is php5.ini which is not in my root directory...

duplicate of http://drupal.org/node/187912

I'm still having this problem in drupal 5.7, will upgrade to drupal 5.x soon.