Image caching, 304 headers

i think this would be an interesting feature but the apache specific nature of getallheaders() is kind of a problem...

I've just got it working with imagecache (patch for it is coming). I'd like to see it for the image module too.

Here is the code I've used:

// normal image viewing code
//..
  if (is_file($destination) && $fileinfo = stat($destination)) {
    if (function_exists('mime_content_type')) {
      $mime = mime_content_type($destination);
    }
    else {
      $size = getimagesize($destination);
      $mime = $size['mime'];
    }
    $headers = array('Content-Type: '. mime_header_encode($mime));
    imagecache_cache_set_cache_headers($fileinfo, $headers);

    file_transfer($destination, $headers);
  }

/*
 * Sets file headers that handle "If-Modified-Since" correctly for the given fileinfo
 * Most code has been taken from drupal_page_cache_header()
 */
function imagecache_cache_set_cache_headers($fileinfo, &$headers) {
  // Set default values:
  $last_modified = gmdate('D, d M Y H:i:s', $fileinfo[9]) .' GMT';
  $etag = '"'.md5($last_modified).'"';

  // See if the client has provided the required HTTP headers:
  $if_modified_since = isset($_SERVER['HTTP_IF_MODIFIED_SINCE']) ? stripslashes($_SERVER['HTTP_IF_MODIFIED_SINCE']) : FALSE;
  $if_none_match = isset($_SERVER['HTTP_IF_NONE_MATCH']) ? stripslashes($_SERVER['HTTP_IF_NONE_MATCH']) : FALSE;

  if ($if_modified_since && $if_none_match
      && $if_none_match == $etag // etag must match
      && $if_modified_since == $last_modified) {  // if-modified-since must match
    header('HTTP/1.1 304 Not Modified');
    // All 304 responses must send an etag if the 200 response for the same object contained an etag
    header("Etag: $etag");
    // We must also set Last-Modified again, so that we overwrite drupal's default Last-Modified header with the right one
    header("Last-Modified: $last_modified");
    exit();
  }

  // Send appropriate response:
  $headers[] = "Last-Modified: $last_modified";
  $headers[] = "ETag: $etag";
}

With that function it should be not hard to integrate it in image module too.
Note that this solution should work with APACHE and LIGHTTPD, as the used server variables are described in the CGI interface. drupal_page_cache_header() uses them too, so they should be save to use. I've tested my solution with lighttpd - it works :)

I've just added an improvement to my code above: Setting content-length headers.

To do so just add the line
$headers[] = 'Content-Length: '. $fileinfo[7];
before the call to imagecache_cache_set_cache_headers()

I'm posting this here, but maybe we need another thread about this... although it is Apache specific...

If you are using apache mod_deflate then no matter how you try and configure it not to compress images it still seems to if the output is a stream from a PHP script. This seems to be a known problem. It's discussed in http://codex.gallery2.org/Gallery2:Performance_Tips for example:

Disable mod_deflate and other compressors for the gallery folder. The reason is that mod_deflate tries to compress the output of G2 before it's sent to the user. And G2's main.php not only outputs HTML, but a lot of binary data (images) as well. Trying to compress this does not only cost a lot of CPU, there's also almost no gain in bandwidth. G2 sets proper content-type headers, but mod_deflate ignores them, even when you try to configure it properly.

Anyhow, I eventually found the answer here: http://www.php.net/manual/en/function.apache-setenv.php#60530

So easy when you know how! Just insert a call to apache_setenv('no-gzip', '1'); before returning the modified headers in image_file_download works wonders. Have tested this with Apache/2.0.54 w' PHP 5.0.4.

subscribing...

I've been writing some custom modules that generate graphs. As part of this have been looking at how the caching and the If-Modified-Since header works and have discovered something a bit useful.

It would seem that when you tell the server to return code 304 (not modified) unless you also include the other headers (cache-control, etc) some browsers (well, Firefox at least) and I'd imagine some network infrastructure too, forgets about the previously cached version of the image.

The upshot of this is that a user flicking between pages, which lets say all have the same image in a block on the left, causes a response 200 (image is sent), then 304 (not modified), then 200 (image is sent again), then 304, etc.

With the code change below though this doesn't happen, things work as you'd expect. Ie. response 200 sent first time, then 304 on all subsequent pages. Note this also includes the Apache gzip fix mentioned above:

function image_file_download($file) {
  $size = image_get_info(file_create_path($file));
  if ($size) {
    $modified = filemtime(file_create_path($file));

    //apache only
    $request = getallheaders();

    if (isset($request['If-Modified-Since'])) {
      //remove information after the semicolon and form a timestamp
      $request_modified = explode(';', $request['If-Modified-Since']);
      $request_modified = strtotime($request_modified[0]);
    }

    //enable caching on this url for proxy servers
    $offset = 60*60*24*30; // one month
    $headers = array('Content-Type: ' . $size['mime_type'],
                     'Last-Modified: ' . gmdate('D, d M Y H:i:s', $modified) . ' GMT',
                     'Expires: '. gmdate('D, d M Y H:i:s', time() + $offset) .' GMT',
                     'Cache-Control: public');

    // Compare the mtime on the request to the mtime of the image file
    if ($modified <= $request_modified) {
      $headers[] = 'HTTP/1.1 304 Not Modified';
      foreach ($headers as $header) {
        header($header);
      }
      exit();
    }

    apache_setenv('no-gzip', '1');
    return $headers;
  }
}

As a patch, where should this end up? Cos image_file_download() looks quite different from above, both for 5.x-2-dev and 6.x-1-dev.
You don't check for user access rights, for instance.

I think access rights, etc. are catered for outside of this function. The 'stock' image file download in my module looks like this:

function image_file_download($file) {
  $size = image_get_info(file_create_path($file));
  if ($size) {
    $headers = array('Content-Type: ' . $size['mime_type']);
    return $headers;
  }
}

This is quiet old, the ID atop reads:

// $Id: image.module,v 1.209.2.36 2007/07/06 15:22:19 drewish Exp $

Please submit a proper patch. See http://drupal.org/patch for details.

Please find attached a patch just rolled against 1.9 of image. Looking in CVS the image_file_download function looks unchanged between 1.9 and HEAD so this patch should work for D5 and D6 versions.

You might want to check for other servers and make sure it works with something other than Apache. I can confirm this works in D5 with Apache and 1.9 of image.

The attached patch worked for me, with offset adjustment, for 6.x-1.0-alpha4.

I look forward to seeing this incorporated into the next release of Image.

I upgraded to 5.x-2.0-alpha3 over the weekend in preparation for a move to D6 and can confirm that patch in #10 works for that version too.

Could someone please roll this (or a variant to check for Apache) into the standard release?

Hadn't we better go with the approach from #2, which works in at least apache + one other server?

Given that (a different) Image module is in Drupal 7 core, the contrib Image module remains minimally maintained now. It won't see any new features. Instead, we're trying hard to make upgrades to D7 work. See project page for details. Thanks!