Hi,
I noticed in my module status report that the mimemail module has a security updates so of course I was keen to upgrade from 1.0 to 1.1. However upon doing this all my emails sent through mimemail are now random characters for example:
PCFET0NUWVBFIGh0bWwgUFVCTElDICItLy9XM0MvL0RURCBIVE1MIDQuMCBUcmFuc2l0aW9uYWwv L0VOIiAiaHR0cDovL3d3dy53My5vcmcvVFIvUkVDLWh0bWw0MC9sb29zZS5kdGQiPgo8aHRtbD4K PGhlYWQ+PG1ldGEgaHR0cC1lcXVpdj0iQ29udGVudC1UeXBlIiBjb250ZW50PSJ0ZXh0L2h0bWw7 IGNoYXJzZXQ9dXRmLTgiPjwvaGVhZD4KPGJvZHkgc3R5bGU9IiI+DQoNCiAgICA8dGFibGUgd2lk dGg9IjEwMCUiIGNlbGxwYWRkaW5nPSIwIiBjZWxsc3BhY2luZz0iMCI+PHRyPgo8dGQ+ICANCiAg ICAgICAgICAgICAgICA8dGFibGUgaWQ9ImF1ZGktaGVhZGVyIiB3aWR0aD0iNjAwIiBhbGlnbj0i Y2VudGVyIiBzdHlsZT0iY29sb3I6IzRENEY1MzsiIGNlbGxwYWRkaW5nPSIwIiBjZWxsc3BhY2lu
That's just a snippet, it goes on...
I looked in the issue list to see if someone else has had this issue but nothing has been mentioned yet so I had to revert back to version 1.0 which makes it work again. I checked in a plain text client and the plain text version is unaffected so seemingly something is broken in the HTML department but I have no idea what.
Surely this can't be limited to just my install as I've made no modifications to the module?
Any help greatly appreciated as having a potential security hole is a major problem for us.
Thanks, Tim.
Comment | File | Size | Author |
---|---|---|---|
#71 | 8bit_encoding_for-1741082-71.patch | 800 bytes | Dave the Brave |
#67 | 8bit_encoding_for-1741082-67.patch | 1.43 KB | joelpittet |
#44 | 8bit_encoding_for-1741082-42.patch | 794 bytes | joelpittet |
#36 | smtp.patch | 1.36 KB | akoepke |
#32 | smtp-base64-1741082-30.patch | 794 bytes | thirdender |
Comments
Comment #1
sgabe CreditAttribution: sgabe commentedWhat you see is the Base64 encoded message body and it should be fine. You should check with other mail clients.
Comment #2
Tim_MA CreditAttribution: Tim_MA commentedOkay but I'm using the same client I used with version 1.0 so what has changed between 1.0 to 1.1 to break it.
I'm using Outlook btw, it's pretty popular.
/Edit
Just tested gmail too and that has the exact same issue.
Comment #3
sgabe CreditAttribution: sgabe commentedAs I mentioned the Base64 encoding was added by #321026: HTML messages are broken by line breaks.
I just tested it again and it works fine. I guess the cause of the error is something else. You can see a list of changes in the release note.
Comment #4
Tim_MA CreditAttribution: Tim_MA commentedWell it went from working perfectly to not working at all. The Base64 coding obviously isn't being decoded.
Can you tell me specifically what you changed so I can undo it in 1.1 and test please?
It's definitely the module update as reverting back and forth fixes it and breaks it.
Comment #5
sgabe CreditAttribution: sgabe commentedAlready did.
Comment #6
Tim_MA CreditAttribution: Tim_MA commentedAre you purposefully being awkward?
You linked to a previous issue with 48 replies, could you please just tell me if there is some code I can quickly change in the module to test if it is this issues fix that has broken it for me.
I'm surprised you're not more keen to help because as I've pointed out this is an issue with the module that surely needs fixing?
Comment #7
sgabe CreditAttribution: sgabe commentedIf you revert some changes, you may get rid of the error but that won't fix the issue. Base64 is a common standard and should not be a problem. I am guessing that bug can be related to the base64 encoding but something else breaks your messages. Are you using any other contributed module related to sending messages?
Comment #8
Tim_MA CreditAttribution: Tim_MA commentedI think you're probably right, so using base64 encoding is a new feature?
We do use http://drupal.org/project/smtp actually as we're sending via an external secure SMTP service that requires authentication. How could this be breaking it?
Comment #9
sgabe CreditAttribution: sgabe commentedThe problem is that SMTP uses 8bit encoding in the Content-Transfer-Encoding header for the base64 encoded body.
With default Mime Mail engine:
With SMTP module:
I am moving the issue to the right place. In the meantime you can try to revert the patch in #321026: HTML messages are broken by line breaks or try the PHPMailer module.
Comment #10
Tim_MA CreditAttribution: Tim_MA commentedThank you, I appreciate your help. Looks like PHPMailer could be the answer.
Comment #11
Tim_MA CreditAttribution: Tim_MA commentedI've tried PHP Mailer and I got the same result as before... here's the header if that helps?
/edit
My bad, the SMTP module was still being called. Seems to work fine now, however PHPMailer module ignores the error_reporting setting so on error is displaying a red error to non administrators...not good. Separate issue though.
Comment #12
asonderer CreditAttribution: asonderer commentedHi,
I had the same problem (using Gmail's SMTP), and tracked it down to adding:
$mail->Body = base64_decode($mail->Body);
.. just before "Set the authentication settings" (line ~768 in my version), in smtp.module.
Works like a charm on my test case, don't know if it breaks other stuff ...
Comment #13
asonderer CreditAttribution: asonderer commentededit /
Better inserting the following, to detect base64 (because in some cases it's not 64-encoded ..):
$base64 = false;
$body_parts = _smtp_boundary_split($mail->Body, $boundary);
foreach ($body_parts as $body_part)
$base64 = $base64 || drupal_strtolower(_smtp_get_substring($body_part, 'Content-Transfer-Encoding', ' ', "\n")) == 'base64';
if ($base64)
$mail->Body = base64_decode($mail->Body);
Comment #14
benclark CreditAttribution: benclark commentedI encountered the same problem, and it started as soon as I upgraded Mimemail which included the patch from #321026: HTML messages are broken by line breaks. Rather than rolling back the patch, I stepped through the process and came up with a (possible) fix for SMTP module.
It looks as though under the right circumstances, a base64 encoded text/html body makes it to $body_part2 at line 681. I grabbed the decode_base64() attachment logic from lines 740-742 and applied it to $body_part2. It does appear to fix my specific issue, although I can't vouch for whether this breaks anything else.
I'm attaching the patch file to start the discussion, and I'm open to any/all feedback. Thanks!
Comment #15
wundo CreditAttribution: wundo commentedI don't have a test environment for this, so if anyone could review it I'd really appreciate.
Comment #16
plachI have the same problem on the 7.x-1.x branch.
Attached you can find an almost straight forward port of #14 that fixed the issue for me. The only addition I did is forcing base64 encoding for the whole message (since it seems PHP Mailer does not support specifying a different format for alt and body), to ensure that base64 encoding is preserved, which is needed for #321026: HTML messages are broken by line breaks to work.
I won't RTBC the patch since I wrote it myself but I'm pretty confident it's ok. I didn't test the 6.x version, but since the port is good I'd say the original is too, aside from the encoding issue described above.
Comment #17
plachTo reproduce the issue:
Expected result: a nice HTML mail is showed.
Actual result: an ugly base64-encoded text is showed.
Comment #18
JenniferRader CreditAttribution: JenniferRader commented#16 patch worked great for me. Solved the problem perfectly.
Comment #19
sgabe CreditAttribution: sgabe commented#1964218: Mail body corrupted in HTML format is marked as a duplicate of this.
Comment #20
Dmitriy.trt CreditAttribution: Dmitriy.trt commentedExperienced the same issue with a combination of MIME Mail and SMTP modules. Patch from #16 should be committed, because it fixes the problem.
Comment #21
citricguy CreditAttribution: citricguy commentedI can confirm that the patch in #1741082-16: 8bit encoding for base64 encoded message body fixes this issue. Tens of thousands of successful emails later I believe it is ready to be committed as well.
I can do additional testing if needed.
Comment #22
Dmitriy.trt CreditAttribution: Dmitriy.trt commentedIn our case it appeared broken again after patching. However, I'm not sure what was the actual reason.
We've re-wrote sending class from scratch without PHPMailer at all by just using the message + headers composed by MIME Mail module and low-level Smtp class from this module. Because with MIME Mail all the parsing of the multi-part messages to build it again is not the way it should be. Especially because the result is not really matching the original message and parsing is done in a non-safe ways like matching substrings in a whole text (with both headers & body). We could probably share the code if someone is interested. It could be a patch to MIME Mail or SMTP, or even as a separate module.
Comment #23
a.knutson CreditAttribution: a.knutson commented#16 Patch worked for us too! Thanks!
Comment #24
wundo CreditAttribution: wundo commentedCommitted b9d1d33 to 7.x
Can someone confirm the 6.x works well too?
Comment #25
scresante CreditAttribution: scresante commentedThe patch in comment #16 is for 7.x
Here is the patch I backported for 6.x
Comment #26
e5sego CreditAttribution: e5sego commented16: smtp-base64-1741082-16.patch queued for re-testing.
Comment #28
AnybodyI can confirm that the described problem still exists with the latest versions of this module.
I strongly recommend to use "phpmailer.module" for Drupal 7. After lots of WTF's I decided to change from SMTP to PHPMailer and everything worked fine ad hoc.
Comment #29
cbccharlie CreditAttribution: cbccharlie commentedI added base64_decode checking if the body is in base64.
Comment #30
dscutaru CreditAttribution: dscutaru commentedPatch from #16 worked for me after some changes, replaced
$body_part2_encoding = $this->_get_substring($body_part2, 'Content-Transfer-Encoding', ' ', "\n");
with
$body_part2_encoding = $this->_get_substring($body_part2, 'Content-Transfer-Encoding', ':', "\n");
smtp 7.x-1.0, mimemail 7.x-1.0-alpha3, D 7.28
another solution is to revert patch #42 from mimemail issue https://drupal.org/node/321026 since it's safe to assume phpmailer will deal with line breaks and as a bonus your server will not have to base64_encode in mimemail, base64_decode in smtp, then again base64_encode in phpmailer
Comment #31
keyframer CreditAttribution: keyframer commented#30 worked for me (#16 did not).
Emails were sent as base64 if it contained a really long line, but this patch + tweak fixed the problem.
Comment #32
thirdender CreditAttribution: thirdender commentedRolled comment #30 as a patch. I don't know why changing that space to a colon fixes the issue, but it worked for us. We were able to still reproduce the problem following roughly the same steps as comment #17, except with 7.x-1.0-beta3 of mimemail and latest git of smtp. Simplenews is not needed to reproduce the issue, just a very long e-mail body.
Comment #33
jlbellido#32 worked like a charm for me! Thanks a lot to everybody for your work in this issue!
Comment #35
joelpittetBased on #33 and #31
Comment #36
akoepke CreditAttribution: akoepke commentedAdding my support to these changes, fixed my issue. Just added a unified patch.
While this has been tested by the community a maintainer hasn't actually reviewed this as yet and committed the changes. Due to this I am changing the status to Needs Review as they still need to commit these changes to the repo.
Comment #38
joelpittetI've tested and am using the patch, hence the RTBC of #32. The maintainer can knock it back if there is something wrong or anybody for that matter.
This may help? https://www.drupal.org/node/156119
What do you mean "unified patch"? #32 applies. Are you adding more things that are need into the patch? My only guess why that fails is it may be made off of 7.x-1.0 instead of 7.x-1.x-dev which I believe already has what you are trying to add hence the patch not applying. Just a guess from a quick look though, let us know:)
Comment #39
quicksketchThanks guys. This fixed the problem for me as well. Like @joelpittet says, #32 applies to 1.x dev and is the one that should be applied. If you're running 1.0, the patch in #36 includes #25 and #32 together.
Comment #40
wundo CreditAttribution: wundo commentedCould someone please re-roll the patch against latest dev?
Comment #41
joelpittetIt still applies it seems.
Comment #42
wundo CreditAttribution: wundo commentedgit pull? :)
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch smtp_0.patch. Unable to apply patch. See the log in the details link for more information.
Comment #43
wundo CreditAttribution: wundo commented(Also, I tried in my local and it didn't work)
Comment #44
joelpittetHere's a clean roll of the patch in #32 against dev if that helps any.
Comment #46
wundo CreditAttribution: wundo commentedOhh, my bad, I was rolling #36 not #32 ;(
Comment #48
wundo CreditAttribution: wundo commentedComment #51
wundo CreditAttribution: wundo commentedComment #52
joelpittetThanks again @wundo
Comment #53
wundo CreditAttribution: wundo commented7.x-1.2 was released, which includes this fix...
Comment #55
rroblik CreditAttribution: rroblik commentedNot working !
The issue is still here in
7.x-1.2
as in7.x-1.x-dev
(see https://www.drupal.org/node/2550151)
Comment #56
wundo CreditAttribution: wundo at Chuva Inc. for Chuva Inc. commentedComment #57
kungfu4 CreditAttribution: kungfu4 as a volunteer commentedHey there,
I just wanted to reach out and let you all know that on upgrading to the beta4 version, I am getting the same encoding errors as has been mentioned, whereas in the beta3 version, this is working as intended.
I will try to take some time and dig into it, but for now, we will be reverting to the beta3.
Just wanted to give you guys a heads up as I'm sure I won't be the only person experiencing this issue.
Thanks
Comment #58
kungfu4 CreditAttribution: kungfu4 as a volunteer commentedHey all,
Updating the status and adding more information. This issue with beta4 is involved around the following contrib modules. We're using:
Mimemailer
SMTP
Simplenews
Workflows
After some sleuthing, interestingly enough, using the Mimemail and SMTP together does not result in this issue when sending a test email from the SMTP module, or when sending out notifications via Workflows email system. But when I try to send a Simplenews Newsletter, OR use the Contact form (default module), I receive the garbled encoding.
Like I said, I'll try and poke at it later, but I'm just trying to bring this to the attention of the group. Sorry for any inconvenience.
Thanks
PS - We are not using Mandril
Comment #59
Anybody@kungfu4 please try the dev version and report.
Comment #60
gunzip CreditAttribution: gunzip commentedSame here. After upgrading to beta4 I sometimes (my guess is, when I have long lines in html body of email)
receive garbled base64 encoded emails. Everything was fine with beta3.
Comment #61
joelpittetSame here only sometimes.
Comment #62
mrconnerton CreditAttribution: mrconnerton commentedI have been having this same problem. After some debugging I have found that the header value had a leading space that I had to trim out. I don't have time to debug futher to see where it's coming from but I simply changed:
if (drupal_strtolower($body_part2_encoding) == 'base64') {
to
if (trim(drupal_strtolower($body_part2_encoding)) == 'base64') {
in smtp.mail.inc and they are sending good for me now.
Comment #63
amir_razavi CreditAttribution: amir_razavi commented#62 has solved my problem for now thanks mrconnerton
Comment #64
nathan.telles CreditAttribution: nathan.telles commented#62 solved the issue for me as well
Comment #65
joelpittetThat's cool that you have a solution to the problem @mrconnerton. Though it would be really nice to know how trimming the encoding would fix this. Why would the encoding have whitespace?
Comment #66
mrconnerton CreditAttribution: mrconnerton commentedLike I said I haven't had time to debug the "why". I'm sure a simple debug_backtrace() would do the trick.
Comment #67
joelpittetok @mrconnerton, here's a backtrace patch that everybody can use to trackdown what's up in their logs.
I'm getting this sporadically as well, so hopefully collectively we can resolve this.
Comment #68
misthero CreditAttribution: misthero commentedsame problem here, randomly mails are base64 encoded
applied patch #67, will report the results if any
Comment #69
Gaurav Chawla CreditAttribution: Gaurav Chawla as a volunteer commented#62 has solved my problem. Kudos mrconnerton
Comment #70
Dave the Brave CreditAttribution: Dave the Brave commentedThe white space can easily come from an earlier line depending on what is setting the encoding. This line actually needs a trim() around it as shown below (line 388 in the version I am using). I can submit a patch if you need it:
388: $body_part2_encoding = trim($this->_get_substring($body_part2, 'Content-Transfer-Encoding', ':', "\n"));
Comment #71
Dave the Brave CreditAttribution: Dave the Brave commentedPatched.
Comment #72
dang42 CreditAttribution: dang42 commentedThe patch in #71 worked like a charm. Thank you very much @Dave the Brave!
Dan
Comment #73
jdanthinne CreditAttribution: jdanthinne at AppTree commentedPatch #71 works for me. Saved me tons of debugging… Please commit
Comment #74
wundo CreditAttribution: wundo at Chuva Inc. for Chuva Inc. commentedCommitted!