image file with '+' sign or & in name not found by imagecache

Same problem experienced here.
Are there orther signs with this problem?

We've also experienced this problem.

I'm seeing the same issue. This seems similar to an issue that was filed back in 2007 for imagecache: http://drupal.org/node/129433

In addition to problems with + signs in filenames, I'm also getting a similar problem if there is an ampersand character in the filename. I realize that these are special characters in URLs, but it's pretty hard to get users all to know and follow file naming restrictions when you're running a site with more than just a few content creators. I think it's important, therefore, to fix this.

Here's an example of an error message that I'm getting:

subscribing

I've created an issue and proposed a patch for the file handler in drupal core to rewrite filenames and replace plus signs, ampersands, and whitespace chars with a dash, I've tested the patch to confirm that it does indeed fix this issue.

#791780: Special chars should be removed/replaced from filenames on upload.

Should this issue be marked as a duplicate?
I was wondering if there is a chance of your patch being added to Drupal 6 core?

The real bug is not that Imagecache cannot find an image named http://example.com/files/imagecache/profile/black+white.jpg, because this is looking for the black white.jpg image (+ means space in an URL encoded string). The bug is that the link is generated wrong (the + should be encoded as %2B).

Fair enough. So that would make it an imagecache bug, and not a core Drupal issue. I think ideally, though, it should be fixed in both places. Drupal core (or whatever handles file uploads) ought to rename filenames so that they don't contain plus signs, ampersands (or slashes, and maybe some other special characters I haven't thought of). AND imagecache ought to encode filenames so that files which HAVE been saved with plus signs, ampersands, etc. don't produce "not found" errors.

I think renaming should happen at the time of upload because it will reduce the likelihood of problems with a variety of modules in addition to imagecache. But imagecache should handle it anyway, because there are existing websites with already-uploaded files.

That's why the -dev version requires transliterate module which removes spaces from uploaded files.

@drewish - haven't tried -dev version but as comment is written it doesn't sound like that would fix the issue.

the filename doesn't have spaces in it. for my test of this issue anyway, the filename actually does have + signs in the filename. image_field handles this ok (suggesting not an issue with core file upload); but imagecache doesn't

Encoding + with %2B doesn't help, as that is already the case I found. I.e. the image I try to retrieve is:

http://example.com/sites/default/files/imagecache/preset/Teal%20UPF%2050%2B_Performance%20Insert_0.jpg

However, after Apache rewriting this is how it ends up in the q parameter:

sites/default/files/imagecache/preset/Teal UPF 50 _Performance Insert_0.jpg

I.e. the + sign has become a white space.

So perhaps we need to double encode the + character or at upload do away with the + character. As I need to fix it, expect some kind of patch shortly.

FYI, this guy explains the problem perfectly.

In #481838: Plus signs ("+") disappear in URL when using clean URLs with the Apache webserver the issue is also recognised, and it contains a suggestion for a fix and that's the one I have applied, as that is the easiest: in .htaccess I've changed the rewrite rule to:

RewriteRule ^(.*)$ index.php?q=$1 [L,QSA,B]

Plain imagefield doesn't have the problem, so it's definitely imagecache that should be generating the correct URLs/filenames.

There's a patch with a different approach in #880098: Escape characters in filenames are escaped and mis-interpreted by ImageCache.

Thanks, your quick fix worked like a charm.
Can you please tell us more about the B flag in .htaccess rule?

Thanks,
Gabriel

http://httpd.apache.org/docs/2.2/mod/mod_rewrite.html#rewriterule

'B' (escape backreferences)
Apache has to unescape URLs before mapping them, so backreferences will be unescaped at the time they are applied. Using the B flag, non-alphanumeric characters in backreferences will be escaped. For example, consider the rule:

RewriteRule ^(.*)$ index.php?show=$1

This will map /C++ to index.php?show=/C++. But it will also map /C%2b%2b to index.php?show=/C++, because the %2b has been unescaped. With the B flag, it will instead map to index.php?show=/C%2b%2b.

This escaping is particularly necessary in a proxy situation, when the backend may break if presented with an unescaped URL.

subscribe

Confirmed, we run into this on https://association.drupal.org/.

I just moved a site to nginx and ran into this issue due to an image with an ampersand in the filename. For some reason it was working with Apache2 but not after moving to Nginx. There is no B parameter in .htaccess so I'm not sure why it was working with Apache. Regardless, the htaccess workaround mentioned above will not work with Nginx so this should really be fixed.

Same issue running Nginx under the following conditions:

1. Uploading a file called hello+there.jpg

This uploads the file 'hello+there.jpg' to /sites/default/files/hello+there.jpg fine. BUT - generates following error in log:

Unable to generate the derived image located at public://styles/thumbnail/public/hello there.jpg.

2. Using Media with YouTube link does not create a thumbnail of video.

e.g. Unable to generate the derived image located at public://styles/square_thumbnail/public/media-youtube/rF1X12PE6PY.jpg.

In Nginx you have two options, either you go back to good'ol rewrite (it's uuugly) or you move along and beyond ;) and use Lua.

I'll update my config tomorrow. Someone else complained about a similar issue of more generic filenames on the issue queue.

To get a quick fix for your problem do on the imagecache location:

    location ~* /imagecache/ {
        ## Image hotlinking protection. If you want hotlinking
        ## protection for your images uncomment the following line.
        #include sites-available/hotlinking_protection.conf;

        access_log off;
        expires 30d;
        ## Fixes the fact that Nginx does exact matching. Escape happens on rewrite or
        ## regex location matching. All URI is escaped now.
        set_by_lua $no_slash_escaped_uri 'return ngx.escape_uri(ngx.var.no_slash_uri)';
        try_files /$no_slash_escaped_uri /index.php?q=$no_slash_escaped_uri&$args;
    }

Thanks perusio!

Will be moving along and beyond with your config update :) and I will keep you up to date on my testing.

Thanks for your work btw on all you do.

Hello Tim,

Support for escaped URIs commited.

Thanks for your interest and kind words.

Tagging as D6 stable release blocker.

We ended up struggling with this one and thought this might help others. Mainly an inspiration from @perusio.

Our Nginx config is mainly based on http://wiki.nginx.org/Drupal

This is for D6:

location ~ ^/sites/.*/files/imagecache/ {
    set_by_lua $unescaped_uri 'return ngx.unescape_uri(ngx.var.uri)';
    try_files $unescaped_uri @rewrite;
  }