Disabling base64 encoding - hotmail compatibility

I attached a patch to do that for drupal 6. It is my first patch so I hope it works :). It is in the function mimemail_extract_files in mimemail.inc

I guess the solution is very similar for the drupal 5 version.

I would be very glad to see this patch to be applied, since there are a lot of issues posted about problems with hotmail in the current setup with base64 encoded html.

subscribing

This patch works fine for me - even under drupal 5.11 (here you will have to apply the patch at lines 124 + 125).
Many thanks!

Dirk

Hi,

I updated the patch.

The default theme function removes all line breaks. That way your email program will insert line breaks after n characters, and break some html by doing that.

Greetings,

Allart

I'd be interested to know how much testing you've done with this change, and what mail servers you're sending your mail out through. While I believe most SMTP servers out there probably do implement the 8bit MIME extension in this day and age, some still may not. When Mimemail was first developed base64 encoding the UTF-8 content being provided by Drupal was still important. If your message is getting passed through to an SMTP server that doesn't handle 8bit encoding you may have a problem.

Jer

(Just ignoring the hotmail thing for a moment)

I'm trying to get straight in my head how come I don't see this problem on my iPhone normally. I'm looking at an email on my iphone from a government organisation, and it's in HTML.

What actually is best practice? Is this an iPhone issue at the core?

Hi jer,

First of all, I'm not an expert at mail servers, but I needed some solution.

We have 4000 (dutch) subscribers to our newsletters, which are sent (almost) daily. We didn't got any reports which are related to the 8-bit encoding of our mails.
We send our mail through postfix-2.4.5 on freebsd.

@sime:
Their mail is probably 7bit (look in the source of the mail).

I've just committed the change from base64 to 8bit for the text/html part. This should show up in the next development release. If everyone could please test this and let me know if there are any issues, it'd be greatly appreciated.

As it turns out, we've been sending the text/plain component as 8bit encoding for some time. I think 8bit encoding is widely enough supported at this point that we should be fine.

Also, I did not remove the new line stripping from the theme function from #5. Text/html parts will not be force-wrapped by mail clients so this shouldn't be an issue.

Jer Davis
Advantage Labs, Inc.

I've found some extra spaces within the text received (email body - here and there, without any common logic - just in the middle of some words) after the last update. After reverting 2 lines (#1 patch) back (to base64 + the next line...), everything seems to be back to normal. I am testing mails with Outlook Express and through squirrelmail though (no hotmail).

Can you attach the raw message source and a screenshot?

Jer

I have the same problem with a patched version of 5.x-1.x-dev to switch from base64 to 8bit. After applying the patch I can send properly to hotmail and all other programs, but I get some random spaces as reported in #10.

The entire message is fine except for a few places - one link gets an extra space in one of the letters, an image gets an extra slash added, little things like that -- it seems random to some extent.

Without the patch I can't send out to hotmail/outlook without it showing up as plain text, etc. - with the patch I get a newsletter (simplenews) out correctly to all readers but the newsletter isn't perfectly formatted.

Can you send some screenshots and possibly a test email to mimemail [at] advantagelabs [dot] com?

I did get it working last night through a combination of removing the preg_replace on the output line at the tail end of the mimemail.module (adding just return $output;) -and- saving the email as "Full HTML" - without I got nothing in the email when I removed the preg_replace (we had been saving them in Rich Text Editor due to our use of FCKEditor).

I'll dig up one of my last tries before I did the route above, which seems to work fine now, and send it along. Thanks!

I have the same problem with plain text showing in Hotmail. However the patch (which I applied by hand to 5.x-1.x-dev yesterday) didn't solve the problem and with 8bit encoding hotmail still shows the text version.

In addition to the patch, I had to do the following (from my message above): "removing the preg_replace on the output line at the tail end of the mimemail.module (adding just return $output;) -and- saving the email as "Full HTML""

So edit mimemail.module and alter the last output line and make sure you save your newsletter as "Full HTML"

Did you try that as well?

Thanks for pointing that out Chad. Sadly this still hasn't solved the problem for me in 5.x-1.x-dev

I'm working with Simplenews in Drupal 5 trying to get HTML mail in Hotmail - I haven't tested Outlook.
None of the info in http://drupal.org/node/310305 or http://drupal.org/node/372710 helped me.
The patch in http://drupal.org/node/225731 (http://drupal.org/files/issues/mimemail-outlook+filename.patch) didn't apply against Mimemail 5.x-1.0 so I applied by hand against that version and I do get HTML mail in Hotmail and Gmail (which are our targets) :D

I wondered if this part was particularly relevant:

- return mimemail_multipart_body($parts,"$content_type; charset=utf-8");
+ // agidise-mod: mixed
+ $content_type = 'multipart/mixed';
+
+ // agidise-mod: removed "$content_type; charset=utf-8"
+ return mimemail_multipart_body($parts,$content_type);

I've attached my working mimemail.inc and mimemail.module in case there is other stuff I've done that I cannot remember (which happens often).... You could look through it and see what stands out. I vaguely remember that I might not have applied the whole patch. I know I tried a bunch of things before I got it working. See if that works for you.

thanks Chad, but it is working as described above so I'm going to deal with some other issues and come back to this later.

I tried this patch and several older ones to no avail.
I'm still having my html emails show up as ugly text with an att.htm attachment in Outlook.
It looks like the last time there was a comment on this was about a month ago.
Is this patch for the most recent dev version of mime mail?

jwaxman,

if you check the date of the last -dev version (April 15 - should be already that one on April 17, when you've submitted your post) and the date of the post with a patch you are asking about (March 17), what could be your conclusion?! ;-))

LUTi:
Well. That makes sense.
I still don't know what I need to do to get HTML mail to be read correctly by Outlook.
Outlook may very well be at fault here. Unfortunately, it's a defacto standard in many corporate environments.
Does any one know of any workaround?

Well my module and mimemail.inc (the zip file) in the comment #19 above works (for d5.x) so long as you also save the newsletter in full html as described in previous comments above that. You could try installing my patched version. At this point I don't appear to be having any problems with any email clients or web clients including outlook, hotmail, msn, etc.

Therein lies the problem.
I'm using D6.
Is there anyone out there for whom D6 mime mail and Outlook 2007 are working?

jwaxman,

what exactly is the source of your HTML mail - simplenews or all Drupal mails? I have an option to process all messages with MimeMail unchecked (there were some issues in the past, still with Drupal 5, if I remember well; and, I simply don't need HTML for system messages as confirmations etc.).

I have Drupal 6, and messages from simplenews seems to be OK (in my Outlook Express) - at least test messages (I haven't created any public newsletter recently...).

I've had to apply a few patches (#1 in this issue, #261028: SMTP Return-Path Setting which obviously is not needed for you, and #438058: Remove line feeds in subject) though.

LUTi:

Thanks for following up.
The source of the messages is indeed simplenews.
"Use mime mail for all messages" is unchecked.
EMail engine is set to "Default mailing engine using drupal_mail()
I'm using the most recent Drupal core and the most recent mimemail (April 14)

What shows up in Outlook 2007 is plaintext with an ATT0001.htm attachment.
Mail send to a gmail account is fine.
The subject line comes across fine in either client.

If it's not the patches I need then what else could I be missing?

jwaxman,

I don't have Outlook (I am using OpenOffice.org instead of M$ Office, so I have to rely on Outlook Express...), so I can not help you, sorry.

I am also not an expert for mail systems (requirements / standards) or for M$ (and all their proprietary "solutions"), so no use of me also here, unfortunately.

I've just tried to help you as much as I could. Now, it is someone else's turn...

And your help is greatly appreciated.
I think my next step is to figure out how to use CVS to download the September release and see if that solves the problem. If that doesn't work then I guess I either need to abandon Simplenews/Mime mail or actually try and understand and fix the problem.

jwaxman,

if you have access to the mail system (mailer), maybe you can try to manually compare the problematic message(s) with other similar ones, which Outlook accepts OK (send a message with exactly the same content from simplenews and from some other client).

Or, you can try to forward the problematic mail from gmail (or some oher client, which displays them OK) to see how that one(s) will be displayed in your Outlook.

Probably there are not much differences, but just some bytes which confuse Outlook.

Mail systems can be quite picky - I've had once weird issues just because someone managed to send a mail with addresses separated not exactly according to some RFC (I am almost sure it was some M$ client involved...).

If you will find the problematic part of the message, it will probably be quite easy to resolve your issue (as a general patch, if it doesn't break any standard, or you for yourself only - as I am forced to do about some other issues I have...).

But, are you 100% sure it is not some (weird or non-standard) setting in your Outlook, which is causing the behavior you are noticing? It surprises me very much that you are the only person on the earth having this issue - otherwise, this thread should be much more active...

LUti
I just tried forwarding from GMail to Outlook.
It works perfectly.
How can I use this information to figure out why sending directly from my site to Outlook isn't working?

jwaxman,

try to compare sources of both mail messages, to see how they differ (neglecting obvious text part differences, introduced when forwarded).

The best would be if you have access to some mailer. If you don't, you can try to install something simple on your PC (also many free mail servers available around...) just for testing purposes. Important is that it handles each mail message separately (maildir), not all together in one file (mailbox). This would make it relatively simple to "forge" messages (manually edit some parts of the message you suspect to cause your issue at the mailer side, to see how your Outlook is swallowing them after). Probably it is something within the body ("multipart" sentences, and around that...) that confuses your Outlook, not in the header...

Also some clients have an option to store messages in maildir (and not mailbox), for example Mozilla Thunderbird (linux version for sure, don't know exactly about Winblow$ one...). This way, you can open them with Notepad (or some other text editor), and compare suspicious parts (I presume you know more or less how a mail message source looks like). But, you can probably not change them with some text editor and get into Outlook to see how it understands them...

Holy moly.
It looks like I've got it.
I was hand-patching rayvaughn's patch from #18 incorrectly.

Using

- return mimemail_multipart_body($parts,"$content_type; charset=utf-8");
+ // agidise-mod: mixed
+ $content_type = 'multipart/mixed';
+
+ // agidise-mod: removed "$content_type; charset=utf-8"
+ return mimemail_multipart_body($parts,$content_type);

around line 320 in the D6 version of mimemail.inc did the trick.

Hopefully this doesn't break formatting in any other client.
Many thanks.

Thanks, it's working here too on D6.
Changed the title hoping to attract more attention.

#33 works for me.
I'm finally able to send HTML mail to hotmail !
Thanks

#33 fixes Outlook 2007 issue of displaying plain text version with HTML version attached as ATT00001.htm. Now Outlook 2007 properly displays the HTML version. Users who opt to receive plain text only get the proper plain text version.

Tried also on a gmail account and it seems to work.

Thank you for the fix--this was driving me crazy!

(working with Drupal 6.10, Mime Mail 6.x-1.x-dev (2009-Feb-22))

Are you sure #33 fix is correct?

I believe the line:
$content_type = 'multipart/mixed';
simply resets all the code before (where $content_type is set to different values - 'multipart/alternative' first, and conditionally changed to 'multipart/related' or 'multipart/mixed' after).

I think just the last line of the patch proposed:
replacing: return mimemail_multipart_body($parts,"$content_type; charset=utf-8");
with: return mimemail_multipart_body($parts,$content_type);
should be enough, or?

If not, some code above is not needed, and should be cleaned out.

This problem definitely seems to be caused because Hotmail (and presumably Outlook, for those who are having similar problems with Outlook) doesn't like the MIME type "multipart/alternative". I have no idea why this is, but forcing the MIME type to "multipart/mixed" fixes the problem for me.

I did this by adding an extra line to mimemail.inc at line 321 (Mime Mail version 6.x-1.x-dev dated 2009-Jun-08):

$content_type = 'multipart/mixed';
return mimemail_multipart_body($parts, "$content_type; charset=utf-8");

This is the same solution proposed by jwaxman in #33, but I don't see any need to remove the UTF-8 character-set setting.

This is obviously a kludge but it solves the problem for the type of newsletters I want to send out. mimemail.inc sets $content_type to "multipart/alternative" as the default setting, then modifies it to "multipart/related" or "multipart/mixed" based on some criteria that I don't really understand. Perhaps using "multipart/mixed" as the initial default would be a safer choice?

There are differences between multipart/mixed and multipart/alternative, as stated in RFC2046, Section 5. Or Hotmail / Outlook doesn't respect RFC (I wouldn't be surprised if it is the case, considering it is all Micro$oft crap...), or a message is not composed properly...

In any case, I believe usage of a charset parameter with multipart/mixed content type can be a bit tricky, as it is generally not intended for this purpose, but to be used with text/plain only (see RFC2046, Section 4.1.2).

In any case, it would be good to compose messages respecting RFC, by my opinion, and not just to try to resolve the temporary issue by chance (maybe such messages could be problematic with some other clients or some mail processors, as Anti-Spam SW or Anti-Virus SW...).

Unfortunately, I don't have enough knowledge (nor Hotmail account / Outlook), so I can not be of a big help in this...

#19 It works for me too. Thanks!!! I was having troubles only in hotmail before overwrite mimemail.inc and mimemail.module

#38 it works for me :)

Just adding one line to mimemail.inc as described in #38 fixes the issue for iPhone Mail. Thanks!

Can someone post the code directly before/after the spot where you add #38? It looks like the same code that is already there... and just pasting it in to 321 doesn't change anything for me.

#33 fixed it for me.... works on gmail, hotmail, yahoo and outlook... what more could I ask?
Many thanks :)

#33 fixed it for me.... works on gmail, hotmail, yahoo and outlook... what more could I ask?
Many thanks :)

I suppose you could ask that the mimemail developers incorporate the patch so we don't have to manually patch every new release. :)

Sure... the last commit was 6 weeks ago... it would be time to include some of these patches :):)

#33 worked for me with the D5 version, fyi. Remove Line 322 and add:

// agidise-mod: mixed
$content_type = 'multipart/mixed';

// agidise-mod: removed "$content_type; charset=utf-8"
return mimemail_multipart_body($parts,$content_type);

Thanks, jwaxman (#33). This fixed the problem I was having when sending to an Outlook 2007/Exchange account. No longer are the emails plain text with html attachment.

I can't get this fixed on Hotmail, I tried all the patches that are suggested above.
I'm using the dev version of Drupal 6 (2009-Jun-08), trying to send an email with Simplenews/Mimemail, the layout is in a table and contains 2 images. Works perfectly for me in Thunderbird, Outlook 2007 and Gmail. But in Hotmail it keeps appearing as a text mail... It's really driving me crazy, I have no idea what this can be.

#33 works for me as well, but I only edited the one line near line 320, which allowed the charset to stay the same (and prevent funky characters showing up in Hotmail):

$content_type = 'multipart/mixed';

subscribe and #33 does fix it for now. It would be good to check on how to properly set $conten_type, not just force it to multipart/mixed in the end.

The edit in #33 seems to have fixed it for us, too.

What's interesting is that we had different behavior in two different Simplenews newsletters on the same server, with (as far as I could tell) identical settings. One was being sent out as

Content-Type: multipart/mixed;

and that one wasn't displaying correctly on Outlook (or Entourage). The other was sent as

Content-Type: multipart/related;
	type="text/html";

and that one WAS displaying correctly. After applying the patch, the successful newsletter was sent as

Content-Type: text/html;
	charset="utf-8"

The whole email is encoded into one lump now, rather than having separate text and HTML parts. But it works in Thunderbird as well as Outlook and Entourage, so whatever....

Using two different servers over 18 months with D5 then D6, we've had to manually do this fix each time (#33 works well for us).

Curious why it hasn't been committed?

And for what it's worth, perhaps the title ought to be changed as it affects many more clients than just hotmail.

Hi there,

An intereasting thread. It helped me a lot.

One thing I noticed after I applied the edit in #33 is that the e-mails on the Iphone showed correctly when fully opened but there was a "This message has no content." in the preview (i.e. list view).

I checked the full header of another html-e-mail that worked and saw that instead of

Content-Type: multipart/related;

it said

Content-Type: multipart/alternative;

After I changed that in the #33 edit, e-mails were also shown correctly in the Iphone preview :-)

Cheers,
Roger

Subscribing this Issue.
I applied the Fix mentioned in #33 and it works for me with Outlook 2007/Exchange and Iphone.

Thanks for this! I also just applied and tested the modification from #33 & it seems to work with iPhone, Outlook, Yahoo, Hotmail, ... and my co-worker's blackberry.

#33 patch seems to have been tested successfully by numerous members of the community. Since applying the patch several months ago, my newsletters come out properly on:
gmail
hotmail
outlook express
Microsoft outlook
yahoo
iphone
blackberry
qq
163
sina
126
....

Any particular reason this patch is not being ported to the lastest version of the mime module?

Many thanks :):)

Duplicate of #372710: HTML emails are text-only in Hotmail This issue has a similar patch, with the same positive effect. There must be something good in this solution, but ... The thing is that issues only attract people who have a problem. The people who will have a problem when a bad patch is committed don't have a voice. And problems occur when under unforseen situations, situations different from the one for which the patch was created. So, the patch (each patch) should be tested under may different conditions, especially conditions for which the patch was not created.

All can help:
Who wants to contribute and to set-up a test-environment which can send various kind of emails (with/without attachement, with/without embedded images, with/without plain text alternatives, ... (I can't think of more variations))?
Who has/wants to set-up many different email clients to receive and verify the test mails on?

(cross posted. Please continue the discussion on #372710: HTML emails are text-only in Hotmail

#33 works for me. Why not just set it as an option in mimemail?

We've used it successfully across Yahoo!, Hotmail, iPhones, iPads, Outlook 2000, 03, 07, 2010, AOL mail, Outlook express, and about 10 others without any issues.

When Outlook and iPhones are broken in the current configuration, is seems like a reasonable cause to create an option in mimemail, say an option/checkbox that says "Use multipart/mixed encoding to improve compatibility with outlook and iphones", so we don't need to keep re-patching? Could we create another option called "use UTF-8 encoding" which would be checked by default?

This would safely accommodate alternative configurations without requiring users to play patch bingo to get it working, which I see has frustrated many users by glancing through the issue queue. I'd be happy to help any way I can.

We have a number of sites using simplenews + mimemail and find it necessary to deploy this patch each time for a new release of mimemail. There are *a lot* of things that can go wrong with html mail ;) but from real life experience, we don't have any indication this particular patch might be bad.

Note that this issue was marked a duplicate of #372710: HTML emails are text-only in Hotmail. The latter is fixed and code is committed to HEAD. If the latest 6.x-1.x-dev does not solve your bugs please change the status of this issue. Comments to duplicate issues may be ignored by module maintainers.

Currently there is no available development snapshot on the project page, you have to checkout HEAD from CVS. If that does not solve your bugs please change the status of #372710: HTML emails are text-only in Hotmail and leave this issue rest in peace.

Wow #33 did the trick.

I have been sending these simplenews emails for months and just accepted the hotmail display issues as a limitation.

Very pleased :-)

@alangbrown: It seems I cannot emphasize sufficiently that using multipart/mixed content-type IS NOT a solution to the Hotmail etc. issues... However this is a duplicate. You should try alpha3 or HEAD, in which this bug is already fixed in the correct way.