I found that all style definitions from my mail.css containing spaces (e.g. "border: 0 none;") are not "compressed" correctly into my HTML mails.
The problem, however, is not in the Mime Mail CSS Compressor module, but in the last line of function template_preprocess_mimemail_message() in mimemail.module:
$variables['css'] = str_replace(' ','', str_replace("\n", '', $css));
It converts "border: 0 none;" to "border:0none;" which of course cannot be interpreted by the mail client.
As a quick fix is changed the line to:
$variables['css'] = str_replace("\n", '', $css);
Comments
Comment #1
gregarios commentedMy issue is a duplicate of this one. Sorry. Subscribing. Here was my writeup:
http://drupal.org/node/475830
Comment #2
gregarios commentedI might also add, that it looks like the code your fix is applied to is meant to strip out multiple white spaces, but actually strips out all whitespaces. Maybe the code should be more like:
$variables['css'] = str_replace(' ',' ', str_replace("\n", '', $css));?Comment #3
bright8 commentedI was having the same issue. I have made the modification suggested by gregarios and now the css in my newsletters looks correct and is interpreted correctly by the mail clients I've tried so far.
Thanks for pinpointing the fix mindgame and gregarios.
Comment #4
gregarios commentedI'm bumping this to critical, then... since this is a show-stopper bug and a fix is in this post.
Comment #5
pillarsdotnet commentedChanging to "needs review" since a patch is attached.
Also see #443964: Skip style sheets with print media
Comment #6
cameronp commentedme fail sorry about that
Comment #7
moritzz commentedThanks for the hint.
I didn't check the later patch, but the very first solution seams just fine for mime mail's core module. (Because spaces are part of the CSS selector syntax, removing any of them without semantical parsing seems fatal in any case.)
Comment #8
gregarios commentedWhich solution is the "very first solution" you are referring to? "Multiple spaces" are not part of CSS in any way.
Comment #9
moritzz commentedI'm referring to the one in mindgame's original post: Multiple spaces are not, but single spaces are.
Comment #10
miro_dietikerI opened a different case, namely: #581142: truncated mail due to css
My original issue was not css space removement, but much more mail truncation.
Please please read my post and understand that removing any newline is NO solution for SMTP due to RFC limitation of 998 bytes line length.
So finally i see no gain in removing a few bytes such as newlines and spaces, except introducing an incredible amount of potential problems. Instead mimemail should implement line length checks and add them manually or do base64 encoding...
Please simply switch this to the full unmodified css content.
Comment #11
scottrigby+1 for #10
Comment #12
gregarios commentedI'm ALL for that! Please, Allie Micka, or one of the other maintainers...
Comment #13
moritzz commentedGood point, Miro.
Comment #14
moritzz commentedHere's the patch for the latest dev release (2009-Jun-08):
Comment #15
gregarios commentedI think having it remove multiple spaces (leaving all as single spaces) is still a good thing. Leave line-endings though. can you update the patch to reflect what I wrote in the space-removal part of #2?
Comment #16
moritzz commented@gregarios. Here you go. I can't test the patch myself for now. Could you please verify that this works? Thanks, Moritz
Comment #17
gregarios commentedThe patch in #16 doesn't seem to do anything.
In testing, I tried a few mods myself to see if I could get it to work right. The string you are modifying doesn't seem to do anything at all. Even if I change it to be like the patch in #14, there is no change... all the CSS is still compressed.
I believe the fault lies in the mimemail_compress modules. I've tracked down another somewhat related issue to this, where certain CSS containing links gets corrupted as well:
background-image: url(http://domain.com/image.jpg);turns into:
background-image:url(http;Somehow, in the mimemail_compress.inc file, there is code that is reading all colons as CSS seperators, and using the colon in a URL incorrectly, causing it to be corrupted.
I believe all the substitutions with whitespace are also done in the mimemail_compress.inc file as well... could you take a look in it? I tried to spot the defects, but could not due to my lack of enough PHP experience.
UPDATE I believe the string
list($key,$value) = explode(':',$def);found in the mimemail_compress.inc file may be to blame for all CSS URL corruption.Comment #18
miro_dietikergregarios:
This issue is NOT about mimemail_compress. It's about wrong behaviour of plain mimemail.module - without mimemail_compress activated.
After having this basic functionality fixed you might think about enabling mimemail_compress (and also look into depth of much other strange things it does ;-) ) .. You don't need mimemail_compress. I strongly recommend you to add an own mail.css file and get rid of other css definitions that way.
Comment #19
gregarios commented@miro_dietiker:
My mistake... the last patch (in #16) does work.
Comment #20
moritzz commented*whew* Thanks for testing @gregarios
Comment #21
gregarios commentedAs a matter of fact I disabled the CSS Compress module at your suggestion and went with a mail.css file. WAY happier with the results. :-) (Now I wish mail.css could be named differently, but oh well.)
Comment #22
gregarios commentedAlas, this has not been fixed in the newest version... and the patch in #16 would appear to not work on this version as well.
The patch needs to be rewritten to work on the same BUG CODE which has been moved to the mimemail.theme.inc file now. Disappointing.
Comment #23
gregarios commentedComment #24
moritzz commentedHere we go @gregarios. For future reference: http://drupal.org/patch/create ;)
Comment #25
gregarios commentedThanks omo for the patching help. Your patch works great! Now, how can we get this into alpha2? :-)
Comment #26
sutharsan commentedPatch works as advertised. Let's move this one in.
Comment #27
ericbellot commentedFor a CSS code cleaning, this line it's better :
$variables['css'] = preg_replace('#\s+#s', ' ', preg_replace('#\s*([:;,\(\)])\s*#s', '$1',$css));- remove all space before and after CSS separator : ":", ";", "(", ")" and ","
- replace multiple spaces and similar (
\n,\r, etc.) by one spacePatch with comment #28
Comment #28
ericbellot commentedPatch for comment #27:
Comment #29
miro_dietikerI don't think we should do any complex css cleanups in here.
Just make it work as minimal as possible. Without need to check complex specific replacements to understand if it works now or if it still doesn't in some specific cases.
Specific advanced cleanups should be integrated the way as css Compressor module does.
Comment #30
scottrigby#24 works here too. nice work guys
Comment #31
miro_dietikerAnother duplicate of this issue
#436474: Error with some mail server: 500 Line too long
Comment #32
benkewell commentedusing patch from comment #28 and work great
Comment #33
cspitzlay@#27,#28
Sorry, but replacing line endings with spaces is wrong
as it can lead to emails that are invalid according to the RFC as pointed out by miro_dietiker
in issue #581142: truncated mail due to css .
Lines are not allowed to become longer than 998 octets.
I just experienced this problem (with the original module) in the real world:
* several mail servers rejected the emails saying it contained a NUL character
(I think that is an artifact of the too long line that got truncated somewhere along the way)
* several web frontends displayed only a mutilated version of the email.
Removing any compression attempt from this preprocess function
fixed the issue for me. So I agree with miro_dietiker (#10 and #29 in particular).
Comment #34
alexanderpas commentedFirst of all, I'm not on my usual drupal working location so i'm unable to generate a patch here.
If you replace
With:
It should fix all the issues at hand.
Comment #35
alexanderpas commentedand here it is in patch format ;)
Comment #36
gregarios commentedI'm rephrasing this post:
This issue was in no way fixed in this new beta, and it seems inexcusable.
@alexanderpas, in answer to the next post:
Your patch is a great fix for this module, however, the number needs to be reduced to around 700 or so, despite the "standard" because indent spaces that are introduced can take up quite a few spaces before the CSS lines begin. No need to push the envelope anyway IMO.
I changed the 990 to 700 in your patch and it worked very well. 990 caused some lines to be truncated by the server, introducing a "!" into the line endings.
Comment #37
alexanderpas commentedComment #38
gregarios commentedSee issue http://drupal.org/node/743150 as many of these issues are being addressed in a bootleg patch.
Comment #39
sgabe commented@gregarios: If you were referring to the 6.x-1.0-alpha2, that is "just" a new security release, doesn't contain any fixes or new features. The beta1? in this #743150: Mime Mail 6.x-1.0-beta1? contains patch from #24 and seems to work just fine. Can you address a problem with that one? Or alexanderpas's workaround would be just better?
Comment #40
sutharsan commentedThe patch in #35 does fix the issue of too long lines, but hardly compresses the css file. We can do more to compress the file. Reduce multiple spaces (3, 4, 5, 6) to a single; Remove comments; remove unnecessary spaces. I found some articles which may be helpfull: http://websitetips.com/articles/optimization/css/crunch/ and http://www.lateralcode.com/css-minifier/
Comment #41
gregarios commentedBoth patches work satisfactorily. Neither have problems that I can see. I used the patch from #24 for a long time with no problems, and I'm using the one from #35 right now. But, the one in #35 needs the number to be less -- more like 700 instead of 990.
Comment #42
gregarios commentedI find it easier to just save the mail.css file the way I like it, in a readable-compressed format. Its not like it'll have to be a huge CSS file anyway. Compression can be left to the Mime Mail Compress module, right?
Comment #43
sgabe commented@gregarios: I second that, just like miro_dietiker wrote too. In here we should leave the mail.css as it is. If we agree, can we stick with patch from #24 and mark this as RTBC?
Comment #44
gregarios commentedAbsolutely.
Comment #45
alexanderpas commentedthis patch should fix the issue (stated in #40) of multiple spaces (2,3,4,5 or even more) and i've lowered the wordwrap number (to 700 as suggested #36 and #41), also, it is availble in two variants, the first, doing things step by step, and secondly, doing all in one single preg_replace
A quick note about the patch in #24:
It can be slow, as the replacement pattern also replaces each single space with a single space.
Comment #46
sgabe commented@alexanderpas: The step-by-step variant works as advertised, but the second variant with
preg_replace()doesn't handle every new lines.However, if we don't use the CSS as it is, we could do more (remove comment and unnecessary spaces, etc.) according to the articles found by Sutharsan in #40.
I tested the attached patch with a style sheet formatted like disaster and the result was very nice.
Comment #47
miro_dietikerIn my opinion we're discussing here about two totally different issues.
First of all: MimeMail is broken - crititally. And we have a very simple fix ti make it work in many many situations.
Remember - The system is pluggable so we don't need to compress, optimize, ... We should do minimum work. If possible standards compliant. For this fix - urgently needed - we don't need to delay anything while discussing about academic optimums.
Second:
We're talking about optimized ways and improvements in css handling. Fine. Those things should be tested with maximum amount of examples and checked very well. For example i'm pretty sure, example #46 will not work perfectly in every case.
Example:
.this_is_a_first_class, .this_is_a_second_class, ..... _this_is_class_number_999 { key: value; }
I'm pretty sure this will result in a too long chunk since ", " is getting replaced with ",". So class lists result in unwrappable results. Generally, i don't see any reason to remove single spaces around formatters for regular cases. This is ugly.
Then about wordwrap: it does no enforcement at 700. This might be ok if transport layer will allow more than 700 characters and unwrappable concatenated things without spaces occur - but this is neither standards compliant nor will this be a stable solution.
I know - if we enforce wordwrap to wrap (4. param = TRUE) it will break css in alien cases. But this is mail. Standard compliance regarding transport layer is hightest priority.
We shouldn't do tricky things here. We'll introduce new problems in yet unknown cases. We can't be semantically perfect since we don't have a semantic css parser - so don't even try to so this in mimemail core.
In my opinion: either trivial solution is ok (known to break for very long lines in css) - or removing double spaces might be OK and applying wordwrap too. Let's move everything else into custom extensions.
Comment #48
sgabe commented@miro_dietiker: As I wrote before I agree with you. IMO: lets stick with the patch from #24 and when we finally have this (among other patches) committed in a new release, we can discuss this in another issue.
Comment #49
gregarios commentedRE: #48
I second that.
Comment #50
alexanderpas commentedSome explanation about my choice of wordwrap and the space removal method.
As wordwrap only allows breaks on spaces etc., I decided to change, tabs and newlines into spaces, and only remove duplicate spaces, leaving a single space where used to be a spacing area giving wordwrap enough points where to wrap.
So, basically, the only way you could break the wordwrap step, is when there is a 700+ byte long string in your original CSS without spaces, on a single line.
Advanced methods, like those in #46 and the articles from #40 can, and shouldprobaly, be implemented in the CSS compressor module.
the patch in #24 is not good, since it replaces every single space with a single space, in addition to duplicate spaces, which is completely unneeded and causes slowdown.
reuploading the step-by-step variant of #45, as this seems to be the patch giving the best results, without advanced techniques.
Comment #51
gregarios commentedWhere do you get this data? I've never seen a slowdown from code that can replace instances of itself with itself not handle it smartly by skipping such occurrences automatically. No matter what the replace function is, it must scan all the text it is processing anyway... so why would this create a slowdown when it isn't writing any replacements until it spits out the data at once at the end of it's process? IE: It isn't like a loop that traverses over the text for every character one at a time...
Comment #52
miro_dietikerThe situation about memory and runtime is much worse than one might expect.
Look at this:
I've wrote some scripts. One setup for time/memory measurement and one script for each replace method.
Input is always a 10*1024*1024*' ,' (30M) sized text containing "space-space-comma" 10M times.
So finally 10M replacements are getting into place. The third is only to compare what happens if we use trivial str_replace which doesn't work for 2+ spaces.
A measurement script that gets run (in this case the first) looks like:
Results:
So the trivial preg match that matches also single spaces is faster in case of all-double-spaces. That's the cost of the more complex preg.
Now if we make 50% double spaces and 50% single spaces (5M times " , ,,"), see this:
Oops... So again... What will happen if we have single spaces only (10M " ,,")?
Oops again. It gets almost the same speed - even if #1 has no more matches! So the regexp interpretation complexity is by far higher than the replacement overhead - if however something like that is measurable.
Interprete it on your own...
Comment #53
sgabe commentedI was about to do some benchmarking on this to see clearly. If it comes to speed, I can confirm, the single
preg_replace()with/ +/iis faster than/\s{2,}/. Onepreg_replace()would be also faster, instead of the three (or more)str_replace().Patch in #24 is a working and acceptable solution IMO.
Comment #54
gregarios commentedOk, so in the case of normal emails going out, considering each email is going to have maybe 1/100th of those spaces, each email would save around .001 seconds? lol
Comment #55
sgabe commentedI think, we discussed this issue and agreed to stick with #24. Complex workarounds should be discussed in a new issue, after we have a new stable release.
Comment #56
gregarios commentedOk, I'm having issues with the reformatted CSS now in the emails. SOME email clients do NOT like that the CSS does not have a line break after each CSS entry. Removing the stock line breaks then adding line breaks at designated intervals is definitely NOT the way to go. The module needs to leave the CSS alone, so a person's mail.css file remains intact. Place a line-break after each semicolon in the CSS if you must have line breaks.
We need to go back to the way it was done in the patch from comment #24.
Comment #57
sgabe commentedI am using this for a while and the line breaks are intact in the CSS. Can you give us particular details how to reproduce? I would dig deeper into this.
Comment #58
miro_dietikergregarios,
you've simply confirmed that the patch #24 we've considered RTBC was right.
There's no objection and even no need to reactivate the topic.
So, once again:
Back to #24, back to unchanged newlines, back to RTBC.
however, gregarios, can you please provide more details and exact examples about which clients don't accept multiple css statements on one single line? What we really should start is a data collection with information about mail clients and compatibility information. Sending quality mime mails is heavily related to best practice and incompatibility workarounds. Understanding the standards and strictly implementing them is by far not enough to be mass compatible...
Comment #59
gregarios commentedI wish I could give examples, but ever since the change I've gotten emails back from some people stating that their emails look wrong, borderline unreadable now since the change, and the only changes I see in the source are the CSS text. About 1 in 500 people complain that they can no longer read the emails because the text is black on black background, which can only happen because of my CSS changes.
Yes... I see... back to #24 for sure fixes it.
Comment #60
jerdavisTalked with sgabe about this a bit and I'd like to see us keep some basic optimization of css. The patch in #24 is simple enough, but parts of #46 seem like a more correct approach. This is a proposed compromise between the two.
This still performs some css optimization, based on the example provided by Drupal core's css aggregation functions:
http://api.drupal.org/api/function/drupal_load_stylesheet/6
It then also calls the wordwrap function in case there are still long lines from a default css file.
I'd like to commit this ASAP, so I'd appreciate any assistance with testing you can provide! This patch is rolled against HEAD.
Comment #61
sgabe commented#60 is confirmed to be working as advertised. Descendant selectors, child selectors and properties with multiple values are intact. Long lines wrapped at the given width. I can't see any problem.
Comment #62
gregarios commentedI am confirming that some email clients, like some older versions of Outlook (there are about 10) do NOT like having missing line breaks before each CSS selector.
(As an example — width of code shortened for emphasis)
Some clients do not like:
While the same clients will show the CSS correctly if this is how the same CSS looks:
I say leave all line-breaks in. Break it further if you want the "RFC821" compliance, but please don't remove line breaks that are put into the "mail.css" manually.
Comment #63
sgabe commentedAttaching a new variant of #60 reffering to gregarios's request with 2 modifications:
Comment #64
jerdavisCommitted to HEAD, thanks sgabe, gregarios and everyone else!
Comment #65
gregarios commentedThe patch in #63 removes all linebreaks other than the 700-character limit ones.
Here is the actual CSS from my email:
Do not commit.
Comment #66
jerdavisgregarios,
Can you please re-verify this? I'm not seeing this in testing. Are you using a mail.css?
Jeremiah
Comment #67
gregarios commentedYes. I took 6.x-1.0-alpha2 and applied the patch from #63, which broke the line-breaks from my mail.css file, and added only its own in 700 character lines as seen in the sample in comment #65.
Apparently the patch is stripping all line endings? Maybe you are only allowing Windows line endings? I'm running FreeBSD 6.3 on the server.
Comment #68
gregarios commentedAttached is my mail.css file. (with the .css extension replaced with .txt so you could read it from this thread as text.)
This is most definitely not fixed. :-\
Can't we just leave the CSS alone? Its not like the admin isn't the one who created it -- it shouldn't be 90000 characters long anyway!
Comment #69
sgabe commented@gregarios: I think we will not find a solution that can be satisfying in every cases, especially Outlook what is an infernal nightmare anyway, the patch in #63 could be the best scenario and I think will work in most cases.
If the user don't use custom mail.css and the CSS aggregation is on (hopefully is), Drupal core will do this for us with the site's stylesheets. So we are talking about only custom mail.css. If you provide a well-formatted stylesheet the line breaks will be intact. Hence the custom mail.css will be made manually by the users (not included with themes etc.) I assume that everybody can do this.
IMHO we should stick with #63 if there is no other issue, and note it in the documentation that mail.css should be well-formatted, otherwise there can be problems with some mail clients. This can be done in #614782: Update README.txt and additional documentation.
Now I can only attach a formatted version of your mail.css made by an online CSS formatter, but I think it will be enough to do the job. Please test it then report back with your opinion.
Comment #70
gregarios commentedI don't see how this will change anything. "Well Formatted" is a matter of opinion, too. I always do my CSS the way you see mine, and I can read it very well that way. So, you're saying that my CSS is to blame for this module stripping out all the line breaks?! Bah.
I cannot use the aggregation, because I only need the 20+ lines of CSS that are in my emails, not the entire CSS file from my website. (over 700 lines, formatted the way I DO it -- it would be over 10,000 lines the way your "well formatted" idea is)
Ok, how about this: Have the function strip out ALL line-breaks of any kind. Then have it insert a line-break after every "}" in the CSS. Then have it put in a line-break after every 700 or so characters as a safety as well. Wouldn't that work?
The way I do my CSS is perfectly acceptable and is actually "well formatted" in some circles, too. Wouldn't you rather have compatibility, instead of forcing people to code CSS exactly like you want them too?
Comment #71
sgabe commentedI am trying to be a little more constructive here. If we use the CSS as it is the site's stylesheets can break the message without aggregation. So if we don't do anything here, we should make at least the site stylesheet embedding optional: #685574: Optional site's css embedding However, for me the well-line-breaked :D stylesheet seems to be fine. I get your point, of course.
Comment #72
gregarios commentedI guess I'm missing something here... isn't CSS aggregation the job of the "mimemail_compress" module? I don't use it, and my site's CSS are not included in the emails at all. I just want my mail.css file to be included with as little destruction of it as possible in the main module's emails. :-)
Comment #73
sgabe commented@gregarios: Attaching new patch against current HEAD. I have tested it with your mail.css and it looked correct. Please give it a try.
Comment #74
gregarios commentedThe patch to HEAD in comment #73 works perfectly. Thank you.
Here is what my original CSS looks like with it:
Comment #75
gregarios commentedComment #76
gregarios commentedComment #77
sutharsan commentedIt's only fixed if it has been committed.
Comment #78
sgabe commentedIt has been committed.
Comment #80
drupalfan2 commentedhi,
where can I find a version of mimemail, where this patch or solution is included?
The only version I can find is
6.x-1.0-alpha2 Download (19.6 KB) 2010-Mar-24
and the patch above is from May 2010.
Comment #81
sgabe commentedCheckout the HEAD from CVS.
Comment #82
sgabe commented@DrupalFan2: There is a new alpha3 release - which includes this patch - listed on the project page.
Comment #83
gregarios commentedNice job. Alpha3 seems to work nicely. :-)
Comment #84
sgabe commentedHallelujah! Thanks for your help!
Comment #86
mdinc_org commentedI use this code
// Perform some safe CSS optimizations. (derived from core CSS aggregation)
// Regexp to match comment blocks.
$comment = '/\*[^*]*\*+(?:[^/*][^*]*\*+)*/';
// Regexp to match double quoted strings.
$double_quot = '"[^"\\\\]*(?:\\\\.[^"\\\\]*)*"';
// Regexp to match single quoted strings.
$single_quot = "'[^'\\\\]*(?:\\\\.[^'\\\\]*)*'";
$css = preg_replace(
"<($double_quot|$single_quot)|$comment>Sus", // Strip all comment blocks
"$1", // but keep double/single
$css); // quoted strings.
$css = preg_replace(
'<\s*([@{}:;,]|\)\s|\s\()\s*>S', // Remove whitespace around separators,
'\1', $css); // but keep space around parentheses.
// End the file with a new line.
$css .= "\n";
and my CSS code will have a clean view