SMTP callback blues: offering $60 to have this installed so that *all* the emails get sent out properly

Well, I've narrowed it down (with some help from the exim list and Bill Hacker)...

He tells me that part of the problem is here:

Content-Type: text/html; charset="utf-8"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit

## Content-Type and Content-transfer encoding do not match

If anyone has any ideas about how to fix this with og2list, that'd be great!
Thanks

Albert
www.ithou.org

Hi killes, :-) and thanks for your thoughts here...

I don't think the difficulty is with incoming mail directed *to* og2list. [Currently, incoming mail gets posted to the group forum, just fine on my system, AFAIK.]

I think it is with mail that is being sent *by* og2list that is problematic.

So, heres' the actual breakdown of my actions and the subsequent behavior of og2list.

_Sample Action_

(1) Go to the website [for example www.ithou.org].
(2) Go to the organic group [for example "test group"]
(3) Click "create forum topic"
(4) Enter the Title, Body, Text of the message. Click "Submit".

_Resulting Behavior_

(1) After about 2 minutes, the og2list cronjob sends the new forum message to all of the members of the organic group ["test group"].
(2) Approximately 80% of the messages sent are successfully sent.
(3) However, 20% of the messages are bounced, with various "rejected RCPT" messages as shown in the original posting above.
(4****) An analysis of the "Message Headers" from all the emails sent *out* by the og2list shows that the mail that is being sent *out* by og2list to have "charset: utf-8", but with encoding usually set such that "content transfer encoding: 7bit" or occasionally "content transfer encoding: quoted printable".

_WHAT I THINK IS THE BUG_
I'm told that 7bit encoding with a utf-8 charset creates a "red flag" for some spam filtering agents? And that this might be a contributing factor in the "rejected RCPT" errors messages in Behavior item (3) above.

_WHAT I THINK SHOULD BE THE FIX_
I've tried to track this down and I think it might be involved in the mail-out.pl file, which has the section:

    my $msg = MIME::Entity->build
      ("From"       => "$from_name <$from_address>",
       "To"         => "$to_name <$to_address\@$mail_domain>",
       "Subject"    => encode('MIME-Header', "$subject"),
       "Type"       => "$content_type",
       "Encoding"   => "-SUGGEST",

I think the line about "Encoding" => "-SUGGEST" might be the culprit. It defaults to "7bit" when in fact this might be incompatible with a "utf-8" charset. However, I've tried to change this to what I think should be a fix, but the Encoding in the mail messages that og2list sends out doesn't seem to change.

[In particular, I've experimented with changing mail-out.pl as follows:]

    my $msg = MIME::Entity->build
      ("From"       => "$from_name <$from_address>",
       "To"         => "$to_name <$to_address\@$mail_domain>",
       "Subject"    => encode('MIME-Header', "$subject"),
       "Type"       => "$content_type",
#       "Encoding"   => "-SUGGEST",
       "Encoding"   => "base64",

Note: even with this change in the mail-out.pl file, the Encoding stays at mostly "7bit" or occasionally "quoted printable" -- and "base64" is no where to be found. [I've attempted to clear cache both with drupal and mozilla. Additionally, I've tried a graceful restart of apache.] [And I'm only trying "base64" because I think it's compatible with utf-8, if I'm not mistaken.]

Thanks for any ideas, and I hope this is helpful to some else too! :-)

All the best,

Albert
www.ithou.org

Thanks, bengen! :-) Awesome!

That makes a lot of sense....

I'll look into that and see if I can figure out how to get my install of Exim to accept Verp's! :-)

Thanks, again...

For those reading at home, this is a description of what might be going on... :-)

http://forums.serverwatch.com/showthread.php?t=18009

How to fix it? Well... I'm workin' on it... :-) It has something to do with changing the ACL's in exim so that "bounce messages" are received, I think.

I guess there might be some variation of joshk's ACL command [see http://drupal.org/node/74736] that allows bounce messages to be received. [I believe his ACL command only allows the non-VERP version of the emails to be received.]

 # og2list
accept condition = \
${run{OG2LIST_HOME/valid-recipient.pl $local_part}{1}{0}}
endpass

Or possibly, there's something in valid-recipient.pl which could help those recipients be seen as valid.

All ideas are welcome! :-)

Thanks,
Albert
www.ithou.org

Hi Amazon, :-)

I'd be absolutely glad to do that... once I figure out what the correct debugging steps are :-) !

BTW, there's a small patch I wrote for the EXIM install instructions for some small typographical stuff located in the last comment here:

http://drupal.org/node/74736

I think this patch probably should be added to the exim.install file at some point in time.

Thanks for all your help :-)

Albert
Esalen Alumni Group

OK, two days straight.... whew... and little bits of progress.... [it's tough in those salt mines! :-) ]....

Tough to find much info on the web re: how to handle VERP bounces. Anyhow, for those of you watching at home :-)... you can check how your exim box would respond if a "verify sender" request was sent to it, by using the command:

exim -bvs checkthisperson@domain.com

For example, in my case,

exim -bvs improv+f.kahane1=verizon.net@ithou.org

Anyhow, looks like adding an additional router to the ACL at the very end solves *at least part* of the problem.... :

Add this to the end of your ACL router in the exim.configure file. [This is a total hack and I'm not sure really if / how it works... so ymmv. I think it also causes some troubles with the transport, but at least the mail of those people who are doing an SMTP call on you is getting sent out, now!]

og2list_VERP:
            debug_print = "R: og2list from $return_path \
              for $local_part@$domain"
            driver = accept
            transport = og2list_transport
            condition = ${lookup mysql{SELECT 1 FROM og2list_groups \
                                       WHERE LOCATE(recipient,'$local_part')>0}\
                                 {$value} \
                                 fail}

Good luck, all...

Albert
www.ithou.org/Esalen

Hi everyone...

I've been away from this for a while... just 'cause I had to make some other pieces for my site... but I was wondering if anyone might be available to help me install this (properly! :-)... and what it might cost... I'm running a small personal site that's is trying to help support others who need emotional support, so I don't have much in the way of funds... but could probably pay $60 dollars or so for this to get done right... [I know that's not that much, it's just what i can afford]....

I'd prefer to give this money to one of the great civicspace people who've been helping me, like killes or bengen, just kinda as a way of thanking them for their help so far, in case they wanted to take this on.

Anyhow, if anyone's gotten this working right let me know.... I've gotten *almost* everything working except there's just one or two email address that strangely bounce rather than receive the mail properly. Most of the problems are documented here in this thread. I kinda set up one of the VERP bounce permissions ok, but the other email address, just doesn't seem to want to take.... I don't think it has to do with the og2list setup, so much as my EXIM or my DNS or some other kind of thing I don't really understand. It's a little beyond me, so thanks for considering...

Thanks for your time and help...

Best,
Albert