I know this issue has been reported in the past (repeatedly), however this is both a bug report and a patch to resolve it in a manner which I think will eliminate it for the foreseeable future.

Issue:

Using Simplenews to send newsletter, I was getting accented A characters when a table in the html was configured with a non-breaking space. The html email looked correct in Outlook and on Gmail, but showed accented characters on the iPhone. Investigating the file I found that the charset within the email was ISO-8859-1. I inserted a specific accented character and examined the result. The accent character was corrupted into a different accent character. Looking at a UTF-8 table of character values, and ISO-8859-1 character values allowed me to determine that the accent character was appearing because the raw data was UTF-8 encoded, and not ISO-8859-1 as indicated by the content-type charset attribute. (The accent A character is a partial encoding of the non-breaking space UTF-8 character.)

As Drupal outputs UTF-8 encoded characters by default I determined the best course of action was to get smtp to default to UTF-8 unless it was being overridden locally. I do this by checking if 'charset' is configured and pre-setting it to UTF-8 if it is not. This is handled in smtp.mail.inc. Whatever value charset is configured to is carried over to the final email.

If there is a reasonable alternative to embedding 'UTF-8' in the patch I would be inclined to utilize it, but I'm not yet aware of a place in Drupal to query the currently in-use character encoding.

(patch to follow once issue # is assigned)

CommentFileSizeAuthor
#5 fix-character-encoding-conflict-1330714-5.patch1.11 KBfabianx
#1 smtp-1330714-charset_utf8.patch788 bytestopham

Comments

topham’s picture

topham commented
StatusFileSize
new smtp-1330714-charset_utf8.patch788 bytes

Patch to modify smtp.mail.inc to use a default of UTF-8 unless charset is configured.

topham’s picture

topham commented
Status: Active » Needs review

Changed Status to Needs Review, after reviewing Status descriptions.

fabianx’s picture

fabianx commented
Status: Needs review » Reviewed & tested by the community

Reviewed and tested, works like a charm, please commit!

The patch does work, because drupal uses UTF-8 everwhere else. PHPMailer has iso-8859-1 as default. This _will_ automatically lead to conflicts.

This patch solves this by always setting "UTF-8", when nothing else is specified.

franz’s picture

franz
any
Location Montréal
commented
Status: Reviewed & tested by the community » Needs work 
+++ b/smtp.mail.inc
@@ -161,11 +161,9 @@ class SmtpMailSystem implements MailSystemInterface {
+		  $mailer->CharSet = isset($vars['charset']) ? $vars['charset']: 'UTF-8';
+		  ¶

please, use soft tabs (i.e. spaces)

fabianx’s picture

fabianx commented
Status: Needs work » Needs review
StatusFileSize
new fix-character-encoding-conflict-1330714-5.patch1.11 KB

Here is re-rolled patch.

fabianx’s picture

fabianx commented
Status: Needs review » Reviewed & tested by the community

As only markup has changed -> back to RTBC

franz’s picture

franz
any
Location Montréal
commented
Status: Reviewed & tested by the community » Fixed

Committed, thanks for the fix.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.