Closed (fixed)
Project:
OG Mailinglist
Version:
6.x-1.x-dev
Component:
Code
Priority:
Normal
Category:
Bug report
Assigned:
Reporter:
Created:
28 Nov 2012 at 20:36 UTC
Updated:
27 Apr 2013 at 11:17 UTC
Jump to comment: Most recent file
This bug describes problems with displaying high ASCII chars when posting from a mail client (tested with Apple Mail)
Using Postfix as MTA
Case 1 - The email body and the Subject line contain high ASCII chars
Result:
Case 2 - The email body does not and the Subject line does contain high ASCII chars
Result:
I vaguelly remember this problem existed also when I used mailgun
| Comment | File | Size | Author |
|---|---|---|---|
| #27 | test.eml_.zip | 2.51 KB | sano |
| #25 | Screen Shot 2013-04-19 at 20.30.39.png | 152.71 KB | sano |
| #16 | test2.zip | 331.73 KB | sano |
| #3 | commentPost-good.png | 92.21 KB | sano |
| #3 | commentPost-bad.png | 108.55 KB | sano |
Comments
Comment #1
mahfiaz commentedI backported the fix from 7.x branch. Please test it whether it works now and let me know.
BTW, this is a lazy fix and will create emails which are incompatible with 7-bit email transport systems and applicable RFC documents (which pretty much should be extinct by now, but you never know).
Comment #2
sano commentedThat seems to have solved the problem :-)
Thank you
Comment #3
sano commentedIt looks like the code handling responses to new posts via mail client still has some problems. These are posts that appear as comments on the web site. The attached screenshot commentPost-good.png shows the post as it is delivered to the admin, the commentPost-bad.png shows the same message delivered to the person who made the post.
Comment #4
mahfiaz commentedPlease check if your template contains
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />definition in the HTML header.Comment #5
sano commentedNot sure what template you mean, but I checked the raw source of the problem email message and noticed line:
Content-Transfer-Encoding: 7bit.
Isn't this what you are writing about in your original reply to this post (#1)? The 2nd email, that is correctly formatted, shows:
Content-Transfer-Encoding: 8bit.
I checked with other list members and the ones with google.com email addresses do not see any problem.
Comment #6
mahfiaz commentedAre these screenshots of the webpage or of your email client?
Comment #7
sano commentedThey are screenshot of email messages in my email app (Apple Mail).
Comment #8
mahfiaz commentedCould you please post the full sources of both of these emails somewhere online, e.g pastebin.com
Comment #9
sano commentedHere is a sample email:
http://pastebin.com/tnERnHH4
When we manually force the mail client to use UTF-8 encoding for a message, then all fields are displayed correctly. When we leave it to mail client (Apple Mail) to determine encoding, we end up with a raw message that contains a mixture of encodings (see
Subject: =?WINDOWS-1252?Q?Inform=E1cie_o_n=E1v=9Atevnosti_port=E1lu?=)We have done more testing and have generated (wireshark) dumps and I can send them to you as well if necessary.
Comment #10
mahfiaz commentedThere is no need for wireshark dumps. But it is still unclear what exactly were you doing and how that happened.
So this is the mail you sent to the website, it was posted to the web and it looks good, also it was sent to your email address, but there problems arose with encodings. Is that right?
Comment #11
sano commentedNot exactly. Once received and processed by ogm, I noticed the following issues:
W3Rlc3Rncm91cF0gSW5mb3Jt4WNpZSBvIG7hdpp0ZXZub3N0aSBwb3J04Wx1I had a case, where 2 copies of the posted message were distributed to the list, but I can not reproduce that behavior, so will ignore it for now.
Comment #12
mahfiaz commentedCould you please test if this works now?
BTW, why are you building a new site using drupal 6?
Comment #13
sano commentedIt is better, but there is still some problem. I made 3 tests and am attaching the test files in one zip archive here.
The out files are the email messages I sent from my Apple Mail mail client. The in files are the ones received from ogm in my mail client.
Included are also screenshots of all messages as png files, bearing the same names (sans the extensions) as the "photographed" messages. I hope this is clear.
Why am I using D6 - I actually built that portal 4 years ago, I just added the ogm module recently. Sorry to be causing trouble :-(
I could not attach the 500 k zip file here, please download it from http://www.strings.net/files/test.zip
Comment #14
mahfiaz commentedDon't worry, you probably just are the first one who actually uses og_mailinglist in non-english environment (except me with 7.x branch).
I'll take a look.
Comment #15
mahfiaz commentedI committed two changes, which ought to fix problems with messages 1 and 3. Thank you so much for giving it good testing and bearing with me.
What there anything wrong with the second one?
Comment #16
sano commentedHi Mattias,
I verified the build and can confirm that the reported problems were fixed. It is true I did not have to include the test case #2 because it was processed correctly - I do not remember why it end up in the test folder (probably because it was late when I was submitting it :-) )
I am attaching one more error report. 2 test cases are included, that differ only in the subject line of the "out" messages - one has accents and the other does not. The responses are the same. So it seems it would be sufficient to include in the test zip just one, but in the past having a message with the same body text but different versions (accented/not accented) of the subject line produced different responses.
Thank you
Comment #17
mahfiaz commentedI should be fixed now. Thanks.
Comment #18
mahfiaz commentedLOL, it should be fixed :)
Comment #19
sano commentedHi, I tested the new build and confirm that also the error issues I reported were fixed. Thank you. :-)
Comment #20
mahfiaz commentedI'm glad we got is sorted out, thank you for your cooperation. Just let me know if you find any other bugs.
Comment #22
sano commentedHi, I discovered more untranslated strings in the /og_mailinglist/og_mailinglist.module
For example "You were successfully unsubscribed from the post" on line 257
Comment #23
mahfiaz commentedI committed fix for all the strings I found in that file. Just keep these coming :)
Comment #25
sano commentedHi, I have to report another encoding problem, this time in digest emails. Not sure what to send along, but maybe the attached screenshot will suffice?
Comment #26
mahfiaz commentedsano, could you please attach the source code of this digest email? That would make my life easier a bit :)
Comment #27
sano commentedAttached is a digest email I received most recently. It is not the one that I sent a screenshot of before (hopefully that does not matter. If it does I will produce a screenshot of this one too).
Comment #28
mahfiaz commentedI took a closer look and have to admit it is strange. Your website uses UTF-8, mail contains UTF-8 header, but somehow the data ends up being in some other encoding. I don't know how this could happen, in og_mailinglist_digest_email just $node->title is appended to mail body, nothing is done with it.
To be sure I tried digest on my d7 site and it worked fine, all non-ASCII characters showed up just as they should (and as for digest code in ogm d7 is very similar to d6). To be able to debug it, I would have to do it on your server or somewhere where it is reproducible. If this is possible, use the contact form here on drupal.org.
Comment #29
mahfiaz commentedIt turned out that since strings for digest mails were not marked for translation, these were directly translated (the hacky way) and the file was in some strange encoding. Always use UTF-8 whenever possible, especially when dealing with Drupal.
So I marked these strings to be translatable in d6 branch as well. If there is something still missing, please let me know.