Mimemail does not currently do any conversion from URLs it finds in files into paths. This works for a majority of cases, but will fail in two scenarios:
1. The site is not in the top level directory of the domain. This means that all URLs will be prefixed with drupaldirectory/, yet since the script is run inside that directory, in order to find the file that component of the path must be removed.
2. The URL is an absolute URL, ie contains http://www.website.com/. This might actually work if url fopens are allowed, but is needlessly inefficient.
This patch fixes the issue, by removing the prefix for both scenarios.
| Comment | File | Size | Author |
|---|---|---|---|
| #13 | mimemail_url4.patch | 3.8 KB | jerdavis |
| #11 | mimemail_url3.patch | 3.42 KB | jerdavis |
| #9 | mimemail-url2.patch | 2.7 KB | matthewbot |
| #7 | mimemail-url.patch | 2 KB | jerdavis |
| #6 | mimemail-convert_urls8.patch | 2.49 KB | matthewbot |
Comments
Comment #1
allie mickaTHANK YOU for looking into this, matthewbot!
There have been a number of vague and unsubstantiated issues surrounding this, and it's super-wonderful that someone ( you ) took the time to nail it down AND provide a patch. The only thing I could have asked for is if you had followed up on one of those instead of creating a duplicate. No matter, we'll close all of them when we can.
When we've got time we'll check this out and check it in.
Thanks again!
Comment #2
matthewbot commentedWell, I did read through a few issues, but I could never determine if they were really experiencing the same problem as me. Anyway, I've got a second version here. I think I'm done with new versions, so please review this one. This one includes code to prevent people from ../ing their way out of drupal. The helper function can also translate relative paths when given a path to base them off of. This should come in handy when implementing CSS parsing.
This is a new issue, but what its doing, taking a file name from the user, reading it off the server and spitting it into an email, probably needs more safeguarding. I think you could, for instance, email yourself a config file with a carefully crafted image tag. Unless filtering already exists in the code and I just need to look harder.
Comment #3
jerdavisI've re-worked this patch a bit after reviewing it and discussing it with Allie. I cleaned up some of the path checking code from mimemail_url_to_file() and also made it so that fully qualified files would be passed out directly and referenced remotely rather than be embedded in the email.
I've tested the patch and everything seems to work fine, however the patch in it's current state does not actually change existing functionality much unless the site is hosted in a sub folder rather than at the domain root. As I do not have a site with this configuration setup to test on right now I have not committed the patch. I'll need confirmation that files are attached properly for a sub folder hosted site prior to committing. I'll try to test this soon unless someone beats me to it!
Thanks for all your work Mathewbot!
Comment #4
matthewbot commentedWell, its a lot shorter. I guess rejecting any url with a .. in it could work, but there are valid uses for it. Themes might use it in their stylesheets, for instance. I guess that could become a separate patch if its needed?
Well, I tested it. It had a parenthesis in the wrong place, but after that it worked. It no longer attaches files with absolute URLs, but I guess that was intended. (though I thought we really did want to attach everything?) I'm a fan of functions with well defined inputs and outputs; your function sometimes returns NULL, sometimes returns paths, and sometimes returns urls. I changed it so that it either returns a path if it could be found, or NULL if not. The NULL will cause _mimemail_file() to pass the URL through anyway. Returning the URL as a filesystem path will still cause some servers with url_fopen on to potentially download files from itself and attach them.
Anyway, this works on my system, but I did make a few minor changes (its even shorter now), so someone else should probably test it too.
Comment #5
matthewbot commentedHmm, issue when $relpath is set and the URL is absolute.
Comment #6
matthewbot commentedI really need to read my code more carefully.. This is my final answer.
Comment #7
jerdavisSpent some more time on this tonight, and came up with a variation on your patch matthewbot. Basically, I've tweaked the $pattern matching for _mimemail_extract_files() to pick up variations a bit cleaner.
I've done pretty extensive testing of this patch from a sub-directory install of drupal, and some testing from a domain install. I'm not quite ready to commit this yet, but it's looking good. Non-absolute images from within a sub-directory based installation did not embed at all prior to applying this patch.
If we could get some more testing on this patch, I'll look at it a bit more tomorrow and then we'll approve and commit a variation.
Comment #8
matthewbot commentedJust took a quick look, and I think we need a urldecode(). I can't test or make a patch right now, so someone else might have to take a look. And theres some minor code style violations. there needs to be a newline before an else, and 'if(' should be 'if ('.
Comment #9
matthewbot commentedWorks great on my subdirectory test system. Further testing shows that a urldecode() is in fact needed, because otherwise it will look for a file containing the %20 or whatever instead of a space, so I added that in. Also fixed the code style issues.
Comment #10
jerdavisCan you describe to me a bit more what the purpose of the $relpath argument is on mimemail_url_to_file()? As we're not actually calling mimemail_url_to_file() with a second argument anywhere, I'm not seeing the point to this addition and the associated elseif in the function.
For clarity sake, I'm considering combining mimemail_url_to_file() and mimemail_url() into one function for handling url processing - so I'd like to understand more why you added this and what your argument is for keeping it.
Comment #11
jerdavisI spent some time tonight trying to normalize things a bit. I've consolidated mimemail_url_to_file() and mimemail_url() into mimemail_url() so that all of our URL/path parsing and normalization is in one logical place. I've also removed the obsoleted mimemail_urls() and mimemail_uri() functions described in http://drupal.org/node/293619. Further, after consolidating the URL parsing, I've implemented fixes for the spaces issues described in http://drupal.org/node/43613.
This patch should be complete, and I've done quite a bit of testing locally. I'd like some additional testing/review before I commit. Once we get this committed to HEAD, we'll roll another release then address http://drupal.org/node/236073. Once that's done, we should be ready to proceed with creating a Drupal 6 branch.
Comment #12
jerdavisThere's still an issue when clean urls is off and we're not dealing with absolute URLS as noted in http://drupal.org/node/43613#comment-958556
I'm re-rolling with a fix.
Comment #13
jerdavisUpdated patch.
Comment #14
matthewbot commentedJerdavis, I just don't understand the insistence on using url(). Its API documentation is "Generate a URL from a Drupal menu path". That's completely not what we need. All we need to do is convert a domain-relative URL to an absolute one. The most direct way to do this is simply prepend "http://www.domain.com" which is exactly what my previous patch does. I guess if you tested it enough we can hack it in there, but it seems like we had to devote more code to fixing the things that we don't need url() to do than it would take to simply code what we do need to do myself (my patch turned a domain relative into an absolute in two lines of code). But, I'm all for getting a d6 out, so I say stick with it if it works.
If the answer to the above is that you'd really rather be the only one contributing code, please let me know and I'll stick to testing, because it seems like every time I uncover an issue, write a fix and test it, spending plenty of time to ensure the code is clean and logical (IMO at least), you go and rewrite it with something that is less clean and logical (again, IMO), for no real benefit (that I can discern), and sometimes even break it. I'm going to repeat all the tests I did in my previous post, despite that fact that I've already taken the time to fix all of them, simply because you rewrote my patch.
Its not like I expect every patch I write to go straight in, and I know that my patches are not perfect, but if you can find problems with them, or even don't like how I accomplish things, I'd appreciate an explanation and a correction, or if you don't have time I could correct it. It is frustrating to post code that one has spent time perfecting only to find that it was completely replaced with code that is completely different and doesn't fully fix the issue.
On the subject of _mimemail_url_to_file(), the $relpath argument, which looking back I didn't do a good job explaining, is used to resolved relative URLs "dir/file". Since the function has no knowledge of what the relative URL is based off of, it has to be provided by an external argument. You can't assume it is the domain, and with clean urls on, it even most likely is not the domain. It was going to be part of the CSS url() rewriting (which are relative to the CSS file). One disadvantage of combining two unrelated functions into one mega function is that now doing this will require adding another argument that is only used in certain circumstances.
Anyway, redoing all permutations of clean urls + domain relative/absolute + properly encoded spaces/literal spaces with both images and links with your patch: (this time including just the failures)
Clean URLs off, domain relative URL, literal space: Image encoded correctly, link became http://domain.com/drupalfolder/index.phptestfolder/SPACE%20FILE.txt
Clean URLs off, domain relative URL, encoded space: Same as above
The original link (the space was changed to %20 in the encoded space test): /drupalfolder/testfolder/SPACE FILE.txt
All other combinations got expected results. The space as + issue in gmail is fixed. I'm not sure its a direct result of the patch, but all attached images with spaces in them were shown visibly as attachments (IE, they could be downloaded and viewed on their own) in gmail. I'd post a fix for the clean url bugs, except I still feel like I'd probably be wasting my time, doubly so considering I've already fixed them once before.
Comment #15
jerdavisCommitted to HEAD
Comment #16
Anonymous (not verified) commentedAutomatically closed -- issue fixed for two weeks with no activity.
Comment #17
andrenoronha commentedHey
this topic is a little old and i dont know how the mimemail module is right now in version 1.41.
my problem is that urls with spaces in it receives and extra %20... not only in gmail
for example:
my picture.jpg => my%20%20picture.jpg
my pic ture.jpg => my%20%20pic%20ture.jpg
so the reference is broken...
anyone can help me?
Comment #18
funkytraffic commentedsame problem in the actual version 6.x-1.0-alpha7 like in #17 with absolute image pathin 6.x-1.0-alpha7 as described in http://drupal.org/node/348327#comment-4070000