random exclamation points appearing in emails from simplenews

We had the same issue, and it appeared to be related to lines that are to long.

the email messages can't have more than 998 characters on one line. Tinymce is removing all line breaks from the html code and there for all characters are in the same line.

A solution is to change the simplenews module to add a "\n" after so many characters. This is done within the html code and will not affect the layout.

You can change the function sn_mail_send like this:

function sn_mail_send($mail) {
$mail->to = trim($mail->to);
require_once('activeMailLib.php');
$email = new activeMailLib($mail->s_format);
$email->From($mail->from_address, $mail->from_name);
$email->To($mail->to);
$email->Subject($mail->title);

// changed the statement to prevent occational "!" in the sent mails
// original statement : $email->Message($mail->message, 'UTF-8', '8Bit');
$email->Message(wordwrap($mail->message,900, "\n"), 'UTF-8', '8Bit'); // new statement with a linebreak every 900 characters

$email->priority($mail->priority);
if ($mail->receipt) {
$email->Receipt($mail->from_address);
}
$email->Send();
return $email->isSent($mail->to);
}

After this change we never saw the randowm exclamation marks a again.

Please report back if this solution also worked for you...

I invsetigated this a while ago (with sponsorship of developmentseed), and the reason you are mentioning (the 998 characters) is indeed the cause of this behaviour. However, our research revealed that inserting a line break (or using the php wordwrap function) could impact layout in some cases (certain browsers/OS/mail client combinations). Further research learned us that an html-wordwrap is impossible with a 100% guarantee that the resulting html will always be displayed correctly in all mail clients and browsers (webmail systems). Therefore, I wrote a patch which checks for line length, and if a line is found which is longer than 998 chars, the mail is base64 encoded. This always solves the issue.

I will commit this patch next week.

cheers,
Dries

What is the status of this patch? I am using v 1.42.2.5 2006/06/03 08:28:26 and it is not there...

FYI, I fixed the problem (temporarily) by ordering TinyMCE to not remove linebreaks in the source, per this option:
http://wiki.moxiecode.com/index.php/TinyMCE:Configuration/remove_linebreaks

However, this is only a stopgap solution, since even without TinyMCE interfereing there is a chance of long lines being supplied by modules that auto-generate components of my newsletter...

here's the patch. Based on simplenews r 1.10, so it'll probably take some adjustments to fit it in.

Thanks to DriesK for the hard work and development seed for the sponsorship. Here is the patch made for the 4-7--2 branch.
I consider it a new feature and therefore it may not go into this branch but in HEAD instead. It is made for this branch for testing purpose only.

Why not commit it as 4-7--3?

It is a matter of priorities. More branches means more branches to be maintained and who will? RobRoy and myself will apply new features to HEAD. Other branches will be maintained with bug fixes where required.

We have decided that the 4.7.x-2.x-dev branch is now no longer maintained and the code may be removed in the (near) future. Therefore no bug fixes on this branch.
This issue will continue as feature request for HEAD.

Why remove the code? What is wrong with keeping it there even if you are not going to maintain?

How about instead commit the patch as the final 4.7 mod?

I was having this exact issue and I found an amazingly simple fix for this... no patching was needed.

I did the following:
1) Made sure that my HTML newsletter had no line breaks in it. The fact that TinyMCE removes them was actually convenient.
2) Create a special input format just for newsletters. This input format should have only one filter enabled. Check off the box for the Line Break Converter.
3) Try sending the newsletter again.

Even though there weren't any lines to break in my HTML code, the Line Break Converter ended up formatting my HTML so that the raw page source ended up looking very cleanly formatted. No excess
tags were inserted.

I'm not sure what's exactly happening, but for what it's worth, I tracked the problem to the "check_markup" function in filter.module (4.7).

I can't see an applying patch for D7, so active.

Shouldn't we close it ? It seems nobody experienced it in 3 years ;)

right, closing.

The problem is still present in the D7 version of simplenews. It can be fixed with the solution mentioned in #13. This is not a good way though because you will have to write very long strings in the code file where you possibly would want to concatenate a string over more than one line.

OK, back to the topic....

This is one of the traditional mail transport problems that remain till today.
In the RFC 2045 there's plenty details about this issue with solutions like base64 encode (robust) and an alternative try with quoted-printable encoding..
http://www.ietf.org/rfc/rfc2045.txt
Search for "lines" or "quoted-printable" and "base64".

With the current WYSIWYG implementation problem (dropping newlines, outputting all tags on a single line ... doh!) the non-compliance is urged to be fixed ASAP since it is subject to happen in most cases...

Without a mimemail encapsulating module via drupal_mail(), it would be the job of php mail() function that sends plaintext as a mail, to encode the mail message appropriately to make it fit a SMTP transport. (Or php should state about this limitation in the docs...)

However with an advanced mime module such as MimeMail or HTMLMail, it's the job of that module to make the payload fit the standards or adequately encode it if it is not fitting. Using base64 in this case would always work although the mail is no more readable for humans without decoding it. For plaintext representation, purists might still prefer quoted-printable with the caveat that it is not guaranteed to work... (e.g. when containing links >76 characters)

Thus, pushing this issue into mimemail queue first and requesting a linelength check... If a line is too big, the mime part SHOULD be encoded as base64 with appropriate part Header.

The same should also be checked with HTMLMail and an issue should be added if it doesn't fit the standard.

Marking as a duplicate of #321026: HTML messages are broken by line breaks.

Well, this issue was older, but still marking it as duplicate then as sgabe tried to...

I'm still seeing this error in 7.x-1.0. Has there been any fix that was ported over to D7. It doesn't look like the previous patches can be ported over.

Don't move things around and change versions!
It's marked as duplicate, so why don't you participate the discussion in the related issue?!

Simplenews will NOT solve this problem. It's the job of the mail transport layer.

Hi everyone, Hi DriesK. We really appreciate your work on this module. We @ Washington University (USA) use this module to send small newsletters to a listserve ... its very handy, however for longer emails, we are seeing the random exclamation marks. I see the patch above however its for an older version of the module.

Do you have a patch for the most recent simplenews module?

Or point us in the right direction to tweak the module to remove these exclamation marks.
I know you did make a previous comment "I wrote a patch which checks for line length, and if a line is found which is longer than 998 chars, the mail is base64 encoded. This always solves the issue."

The patches i tried are not working as they are for older versions of the module.

I'm still new in Drupal, but i do understand OOP.

Thanks again for your work.

Thulani