Extend mail2web support for mail clients (Yahoo mail) and some html cleaning up

The patch looks ok.

However its big and needs some more testing with some different e-mail clients (Help wanted).

Thanks, I'll look into this soon. What is the "html corrector filter"?

I'm occasionally getting a PHP error as follows:

Cannot modify header information - headers already sent by ([...]/notifications/notifications_mail2web/notifications_mail2web.module:257) in /homepages/41/d173730136/htdocs/realrational/includes/common.inc on line 314.

But the message does now seem to be making it into the DB.

Moved the module to a different project

Ryan, I'm weighing the costs/benefits of putting these message-cleaners (for Yahoo, Mail.app, Outlook, Gmail, Lotus, etc., etc.) into a separate directory and including them. The aim would be to provide cleaners for several mail clients w/ mail2web, but then if someone wants support for a client that's not been considered, they can write the small plugin and stick it in this directory. This gets around each person having to hack the module directly if they find they want to support X mail client. I need to take a closer look, but I believe it could be as straight-forward as looping through any plugins in the /cleaners directory and setting a flag upon first match.

Thoughts?

Ryan, I'm going to take a look at this this week. In regards to embedding the message id which is the crux of your patch, I'll try to use what you have here. I'll then break out the message cleaning into includes. Please let me know if you've already broken up the patch. Otherwise, I'll proceed later in the week.

Ryan, I've committed fixes for returning the node, using parts of your patch, on the dev version of Drupal 5. Will do Drupal 6 soon. As for the part of the patch that adds the headers to the message, how can I get this issue to repeat? You say it happens in Yahoo Mail?

thanks,
Ian

I suspect that the message id is also being stripped when using the iPhone's mail.app.

Ian,
I can help port this to D6 as I am in the same boat as @Veggieryan in that I am moving all my projects to mailhandler/mail2web for mailing-list-like features.

R,
Coby

[Edited for clarity]

I changed the backup message id function a bit for d6. it should work for d5 as well but note the leading underscore to indicate a private function.

/**
 * Find the message id in the body if mail client stripped it out of the headers.
 *  
 * @param $body
 *   raw email body
 *
 */
function _mail2web_get_backup_messageid($body) {
  if (preg_match("/([0-9]+\.){4}+[a-z0-9]+@([a-z0-9-]+\.[a-z]{2,5})/i",$body,$matches)) {
  	//  Check to make sure the match has our mail2web domain
    $domain = variable_get('mail2web_server_string', 'example.com');
    if ($matches[2] == $domain) {
      $backup_messageid = $matches[0];
      // We have a match for the messageid and the mail2web domain string so we are good
      return $backup_messageid;   	
    }
  }	
  else {
	// No matches found so assume we have no messageid and node->threading is not set
    return false;
  }  
}  

Might be a matter of semantics but I had to rebuild it to understand it and ended up with a bit different answer; I think this is a bit more efficient:

1) We need only run preg_match with my regex as it looks for an exact match. If we have more than one match we might have bigger issues.
2) Using parentheses in the pattern will allow us to use the matches array to do a few more checks.
3) Value returned for the backup id is set explicitly

r,
coby

OK, here is some code that works for iphone and other clients that may add stuff to the to header when replying. This is working on the D6 version but should also work for D5? The first and last if statements are just for context.

 if (!$node->threading) { 
     $node->threading = _mail2web_get_backup_messageid($node->body);
  }
  //  Some mail clients (iPhone) change the toaddress from example@example.com to 
  //  "example@example.com"<example@example.com>
  //  @TODO This may need to be addressed in mailhandler?
  //  @TODO Should we be using strpos or stripos - I think we already have PHP 5 dependency?
  //  Check to see if it starts with something like a double quote and if the mailbox name is in the string
  if (strpos($header->toaddress, $mbox_name) != 0 && strpos($header->toaddress, $mbox_name) !== FALSE ) {
    //  Just set it to the mailbox
        $header->toaddress = $mbox_name;
  }
  if ($header->toaddress == $mbox_name && $node->threading && ....

There should be a better way to do that but it works for now. Thoughts?

I do apologize for not putting up patches. I need to go back and build some more incremental stuff for the D6 version. In the meantime I do not want to hijack this thread for the D6 version, so I will upload my working D6 version here #351260: Proposed Changes For D6 Version and you can track my public repository here: http://github.com/cglusky/metanomy/tree/mail2web_mid - I am learning git so sniff around the different branches for the latest commits, but for now that's it.

The problem with the iPhone was not dropping the messageid it was that it changed the toaddress and mail2web uses that string to make sure we are working against the mail2web mailbox (per your patch I believe:-). My "fix" in #13 just checks to see if the mail2web mailbox string is in the toadress if they are not initially an exact match. Will not argue that it's a crap fix - my string manipulation skills are not Kung Fu Enabled.

For example:

mail2web mailbox = example@example.com and should be set as the toaddress

iPhone toaddress = "example@example.com"<example@example.com>

That causes this check to fail when it should not:

...
 // The In-reply-to header is cleaned and passed in $node->threading
  if ($header->toaddress == $mbox_name && ...

[EDIT] My impression is that this may need to be handled in MailHandler. There is code there to clean In Reply To and we may need to clean the toaddress?

Sorry a bunch of edits to correct my semantics - need more coffee:)

This is fixed and will be available in the next release. If you want it now for further testing/review, it is necessary that you run the dev versions of messaging/notifications. This will be released for 5 and 6 in the next release of mail2web at the end of this week.

Thanks for the patches and testing!

Ian

Switch to mailcomment project. Mail2web is now known as "mailcomment" due to a trademark issue with the name mail2web.