ImageCache + Varnish + Origin Pull CDN causes problems

Subscribing, same problem here. This is not limited to the CDN module, it's also a problem which occurs with the Simple CDN module. By patching or overwriting the imagecache_create_url() function no presets will be processed.

I changed the category to “bug report” and the priority to “major”. In my opinion this is a major key-feature of the CDN module and therefore has to work properly. I'm investigating further to find a solution for this, maybe I can help a bit.

Okay, I found a way that works for my configuration.

imagecache.module

      //
      // Added logic to support the CDN integration module.
      //
      $request_file = variable_get('cdn_drupal_root', realpath('.')) .'/'. file_directory_path() . '/imagecache/'. $presetname .'/'. $path;
      clearstatcache();
      if (module_exists('cdn') && file_exists($request_file)) {
        return url(file_create_url(file_directory_path() . '/imagecache/'. $presetname .'/'. $path), $args);
      }
      else {
        return url($GLOBALS['base_url'] .'/'. file_directory_path() .'/imagecache/'. $presetname .'/'. $path, $args);
      }

1. I create the full path to the file on my server
2. I clear the file status cache to ensure PHP is really checking the file
3. I not only check if cdn is active but also check if the file exists

Although I was pretty sure this couldn't be true because I've tested it in the past, I actually created a whole new Drupal 6.19 site + Image + ImageCache + ImageAPI + CDN integration. It does work just fine. Or at least, I can't reproduce it in my simple setup, maybe this problem is only popping up because you're combining these modules with some other, buggy modules?

I need to see hard proof that it's failing. I.e. if you want this to be fixed, then you'll have to provide me with a test site to which I have full SSH/SFTP access on which the problem is already being reproduced.

Hi Wim,

I've managed to reproduce this very same problem on my site using Voxel's origin pull CDN. I'm not sure what the cause is- I found that I can fix the problem for individual images by calling them using the server URL rather than the CDN URL- this generates the image, and then the CDN URL works, but when the same request comes from the CDN for some reason it doesn't seem to parse it.

I cannot for the life of me figure out why it would do this.

Hang on. I have a thought- Fleshgrinder + deadrich - are you using varnish as well? I am. It may well be that varnish is playing a part here...

That's very likely indeed…

Can somebody try pointing their CDN at the non-varnish port? (8080 for most people). I think this might fix it but unfortunately Voxel's crap CDN doesn't permit the entering of an origin URL with a port number.

OK I have successfully experimented and found that it does indeed appear to varnish that is causing the grief. If you set your CDN to pull from apache directly, bypassing varnish (i.e. on port 8080), this problem goes away.

Great to know! I documented this right away. Thanks for pulling through and getting back to me!

http://drupal.org/cvs?commit=416680

Woops, that should've been "Origin Pull CDN" and not "Push CDN".

http://drupal.org/cvs?commit=416682

Just to report that the patch given in #2 has effectively fixed the problem with uploading images with Varnish + Drupal 6 + ImageCache 2, but I also had to remove the check on ImageCache as hinted here: http://drupalcode.org/project/cdn.git/blobdiff/60609b47275c6af77f8600fcb....

Thanks guys, this was really starting to drive me mad!

Reporting again, I have been using Varnish + D6.22 + ImageCache for about a month now. At the beginning, I had applied the patches indicated here as reported in the previous comment.

However, after a while, the original problem appeared again (mostly when people started editing in different places) with no explicable reason.

This problem is also related to comments #52 and #72 of report http://drupal.org/node/570132.

Comment #52 in particular mentions a hack which calls the imagecache_build_derivative() again inside the code indicated here, but doesn't give any code snippet.

After long hours of trial (enabling and disabling this incredible CDN module on a production site during quiet hours), I have been able to verify that the problem is the ImageCache module does not generate the corresponding resized image, even with the patch mentioned as a first comment here, so I reproduced the imagecache_build_derivative() patch and that worked like a charm. Here is the modified code. I'm not submitting it as a patch because I think it's both an ugly patch (it just add one check instead of fixing the problem in the first place - but I couldn't find that first place where it happens) and it's mostly an imagecache patch which only happens when enabling CDN. Even then, it works so I applied it on several production sites (running several thousands visits per hour).

      //
      // Added logic to support the CDN integration module.
      //
      $ic_rel_path = file_directory_path() . '/imagecache/'. $presetname .'/'.$path;
      $request_file = variable_get('cdn_drupal_root', realpath('.')) .'/'. $ic_rel_path;
      clearstatcache();
      if (module_exists('cdn')) {
        // There is an issue somewhere with ImageCache not generating the resized
        // images, so make sure it exists, otherwise re-call the derivative builder
        if (!file_exists($request_file)) {
          // Get imagecache preset details from name to give to imagecache_build_derivatives()
          $preset = imagecache_preset_by_name($presetname);
          imagecache_build_derivative($preset['actions'], $filepath, $ic_rel_path);
          // imagecache_transfer($ic_rel_path); // this seems to cause unwanted headers to be sent
        }
        return url(file_create_url($ic_rel_path), $args);
      } else {
        return url($GLOBALS['base_url'] . '/' . $ic_rel_path, $args);
      }

Bringing this back and assigning to ImageCache.

patch to generate image if the url is altered. probably will affect performance somewhat since it will check if the file exists every time it generates the url.

Is this still a problem when using version 2.0 beta 12 of the ImageCache module? It supports the CDN module natively (without requiring a patch).

Closing due to lack of response.

I still have this problem and i'm using latest release candidate for D6.

I have Varnish too, can anyone explain where to apply patch #13 to get things work?

Sorry, it was a problem in my custom htaccess rules.