Post comments via email reply to notification

Take a look at #1348912: Reply to comment in email for info on mailhandler and d6

After doing some research on mailhandler and og_mailinglist it appears mailhandler will be the easiest implimentation we can add for replying to comments in emails.

og_mailinglist isn't bad, but has a few issues for us:

• integrated solution, includes hard coded assumptions which mostly works for us, but could be problematic for custom solutions
• requires the phpmailer library. This is an external library that could be troublesome for us to ship with commons
• has rudimentary security checking, not too hard to spoof an email as someone else.
• but... doesn't require us to use feeds and create dummy nodes.

mailhandler on the other hand, uses the feeds module, which requires a new CT. It'd be nice if there was a way we could hide that.
Also, while mailhandler has much finer control over how nodes and comments are posted, we would need to secure it so people cannot add their own keypairs (ie change node type, text format, etc) to change where email messages get posted in drupal.

For both methods, we need to add a way to authenticate the email. This probably is best done as a token embedded inside the mail header, as well as a line in the email. It needs to be user (maybe node?) specific so emails cannot be spoofed.

I'm leaning towards mailhandler right now, since requiring another 3rd party library feels DOA for me. We will need to implement a feature that sets up a feed and a default mailbox, probably during install.

Here is a screenshot of the main configuration page. This sets a default mailbox to recieve messages:

https://www.evernote.com/shard/s12/sh/c599ca07-f4b8-48bf-9a1f-fa2fa3a9fe...

We can change the default mailhandler feed so that it doesn't create a new content type to import mail. I think in our case, with comments, we only need one node -- therefore the standalone form will work fine:

https://www.evernote.com/shard/s12/sh/ae42108b-32df-42e0-b0c8-d725623cc9...

We will need to re-export a mailhandler feature, similar to mailhandler_default, found in mailhandler/modules

For authentication, the tokenauth module does some of what we want, but we'd have to dive into it to figure out if it does it all. What it does is provides tokens for users at a url path, and authenticates for that path, for that user only. We can adapt this module for mailhandler to provide a token for a user and node id to be sent out with an email using the built in tokens from tokenauth via the message notify feature. (since it uses tokens)

Since this feature was in Commons 2.x, I've added this to the Meta issue: #1930406: [Meta] - As a site Administrator I would like to upgrade from 6.x-2.x to 7.x-3.? without losing functionality

If I forwarded one of my notification's by email to another person, wouldn't that person then have all the data they needed to masquerade as me?

I may be missing something obvious here ;-)

Yes. Email is not a secure method of communication. Most mailing lists only use simple address verification, so if you can spoof someones email (easy), then you can send to the mailing list.

Looking at the tokenauth module, we can embed that in the 'from' header to the user, and so if they reply, it'll include the token. However, that user can also forward that token to someone else, and that person could masquerade as said user and post a comment on their behalf. Luckily, using tokenauth, damage is limited to one thread, and we suggest that people don't forward emails ;-)

In regards to our functionality, I think we want to only use it to respond to comments. I think there might be a way eventually to post nodes, but since we're looking at a 'tokenauth' based on a node, this wouldn't work for node creation (because the token doesn't exist yet)

Some clarification on tokenauth, since it generates a token per user per message/node created, we just need to embed the token inside the message and re-export the feature.

On the receiving side, per the mailhandler_tokenauth module:

 * Authenticate message based on token from tokenauth module.
 *
 * If the user's token is found somewhere in the "to" field, assign that user's
 * uid and name to the node object. A rough search for the token somewhere in
 * the "toaddress" is performed instead of an exact, ordered match in order to
 * allow some freedom in the format of allowed "toaddress". For example, if
 * using a catchall email address, the toaddress could be:
 *
 * f93ksj35dx@example.com - where f93ksj35dx is the user's token or
 * alternatively:
 * f93ksj35dx-foo@example.com - where f93ksj35dx is the user's token and foo is
 * the name of an Organic Group to which the message should be assigned.
 *
 * A rough search allows for different approaches to use this single
 * authentication method.

This should provide us the basic authentication needed to secure comment replies from random address spoofing, but doesn't really fix the problem of people forwarding emails to others, and then others malicously masquerading as the originator. Just have trustworthy friends ;-)

This should provide us the basic authentication needed to secure comment replies from random address spoofing, but doesn't really fix the problem of people forwarding emails to others, and then others malicously masquerading as the originator.

Yes can see this being both the most useful and the most risky with the same groups of people in certain cases. Say a head of service posts their service strategy asking for comment, a valid criticism is received and notification sent, head of service forwards notification to larger workforce, and disgruntled technologically-savvy employee sees opportunity and posts 'my strategy is ****' from a near impossible to trace mta masquerading as the originator.

But so long as said head is well-briefed in these messages and not to forward them, it's actually really useful, because they might be more likely to interact.

So I'm on the fence I suppose, I'd probably disable it for all but those whom I'm confident properly understood the whole thing, but at the same time glad those who did understand could interact without having to take additional steps to do so.

In terms of security, replying to comments via E-mail is no different than a mailing list, which has inherent security issues. There is really no way to use email in a full-proof secure fashion, since the from address can be spoofed and the token (password) can be mistakenly shared.

The ways to mitigate this is by having the token in the drupal reply-to, and requiring this token when you reply to an email. For most people, this allows you to hit reply to keep talking in the comment thread.

But either way, reply via email can be disabled if admins don't want to use it like a mailing list.

So most of the components exist to make this feature work. The major work we will need to do is alter the message functionality inside commons_notify to change the from message to use tokens.

Probably not more than 4 or 5 days with testing.

Been playing with this most of the weekend. I have something that works pretty well with my limited testing. It uses mailhandler, mailcomment & mailhandler_singlemailbox. I have included a make file that details all of the modules and patches that are required.

After enabling, you'll need to enter your mailhander mailbox credentials, and your mailbox will need to be should be set up as a catch all. This allows you to, for example, send to groupname@example.com to post content directly into the group called Groupname. It also enables replying to notifications via email to have those added as comments on the relevant post.

Thanks @mrfelton, I'm working on integrating part of #14 (Mail Comment / Message integration) into the Mail Comment module at #1730706: Basic integration with Message Notify. Also, as the maintainer of Mailhandler / Mail Comment, let me know if you have any questions. I don't have time to actively monitor discussions but will respond if you contact me directly.

Hi. Thanks everyone for your work -- I am very interested in this.

I guess from the title of this thread that we're integrating email with comments, which is great for every commons content type except Q&A.

Does anyone have a sense of the likelihood or ETA for integrating email with Commons Q&A? I mean that when you are notified of a question, you can reply by email and that creates an answer to the question on the site.

My users really want to respond by email, and the thing they most want to respond to are Commons Questions...

Thanks

For Commons, Mailhandler is one possibility but I would personally like this as a pluggable architecture so if you want to use Mailhandler, do it but you can use other tools such as Mailgun.

For a MVP, I would choose Mailgun.net as a solution since Mailhandler and parsing email is just a pain while with Mailgun we could outsource parsing of emails, their receival and everything else and then receive all the emails via an API.

Trying to use mail comment with commons with no luck when I found this threat. Any update on this?

@robertgarrigos: I'm not working on integration with Commons, only with Message Notify, which theoretically should allow Mail Comment to work with Commons. I'm afraid I can't provide support on integration, as I've never even installed Commons before.

Thanks Dane.

In any case, this is a commons issue, so can I make it work? The problem now is that some of the commons message types cannot be used with mailcomment because their form_id's name is not "message_type_form". Cloning the message type doesn't work either. Is there a way to have those message types work with mailcomment?

right, found a way, although I had to hack the code of mailcomment. Hope Dane could implement it someway (and better). This is what I've done (sorry for not posting a patch, I really think this can be very improved):

On mailcommnet_message_notify.module file:

change this three lines in mailcomment_message_notify_form_alter() function:

  if ($form_id != 'message_type_form') {
    return;
  }

with this:

  $string = $form_id;
  $pattern = "/^message_type_.*form/";
  preg_match($pattern, $string, $output_array);
  if (!isset($output_array[0]) || !isset($form['name']['#default_value'])) {
    return;
  }

Then the module needs to read the message entity to insert the "reply above..." line. This is done by looking for a "field_comment_ref" or "field_node_ref" fields. Although I created those fields, for some reason the module doesn't find them. So, I changed the module's code to use the reference fields created by the commons feature for the message types it uses to send messages for new node and comments (commons_notify_comment_created and commons_notify_node_created). I've done this by cloning and changing the code within the mailcomment_message_notify_mail_alter() function in the same file.

The code I clone is:

if (isset($message['params']['message_entity']->field_comment_ref)) {
    // This is a comment.
    $cid = $message['params']['message_entity']->field_comment_ref[LANGUAGE_NONE][0]['target_id'];
    $comment = comment_load($cid);
    $messageid_params['uid'] = $comment->uid;
    $messageid_params['cid'] = $comment->cid;
    $messageid_params['nid'] = $comment->nid;
    $messageid_params['time'] = $comment->created;
    $ancestor_msg_id = mailcomment_mail_comment_ancestor_message_id($messageid_params['nid'], $messageid_params['cid']);
    if (variable_get('mailcomment_alter_subjects', 1)) {
      $message['subject'] = t('Re: %subject', array('%subject' => $message['subject']));
    }
  }
  elseif (isset($message['params']['message_entity']->field_node_ref)) {
    // This is a node.
    $nid = $message['params']['message_entity']->field_node_ref[LANGUAGE_NONE][0]['target_id'];
    $node = node_load($nid);
    $messageid_params['uid'] = $node->uid;
    $messageid_params['nid'] = $node->nid;
    $messageid_params['cid'] = 0;
    $messageid_params['time'] = $node->created;
    if (variable_get('mailcomment_alter_subjects', 1)) {
      $subject = $message['subject'];
      $subject = variable_get('site_name', '') ? '[' . variable_get('site_name', '') . '] ' . $node->title : $subject;
      $message['subject'] = $subject;
    }
  }

Then I added the elseif clauses to match the right commons reference field, which are field_target_comments and field_target_nodes. So, this is how it goes:

  if (isset($message['params']['message_entity']->field_comment_ref)) {
    // This is a comment.
    $cid = $message['params']['message_entity']->field_comment_ref['und'][0]['target_id'];
    $comment = comment_load($cid);
    $messageid_params['uid'] = $comment->uid;
    $messageid_params['cid'] = $comment->cid;
    $messageid_params['nid'] = $comment->nid;
    $messageid_params['time'] = $comment->created;
    $ancestor_msg_id = mailcomment_mail_comment_ancestor_message_id($messageid_params['nid'], $messageid_params['cid']);
    if (variable_get('mailcomment_alter_subjects', 1)) {
      $subject = $message['subject'];
      $subject = t('Re:') . ' ' . $subject;
      $message['subject'] = $subject;
    }
  }
  elseif (isset($message['params']['message_entity']->field_target_comments)) {
    // This is a comment.
    $cid = $message['params']['message_entity']->field_target_comments['und'][0]['target_id'];
    $comment = comment_load($cid);
    $messageid_params['uid'] = $comment->uid;
    $messageid_params['cid'] = $comment->cid;
    $messageid_params['nid'] = $comment->nid;
    $messageid_params['time'] = $comment->created;
    $ancestor_msg_id = mailcomment_mail_comment_ancestor_message_id($messageid_params['nid'], $messageid_params['cid']);
    if (variable_get('mailcomment_alter_subjects', 1)) {
      $subject = $message['subject'];
      $subject = t('Re:') . ' ' . $subject;
      $message['subject'] = $subject;
    }
  }
  elseif (isset($message['params']['message_entity']->field_node_ref)) {
    // This is a node.
    $nid = $message['params']['message_entity']->field_node_ref['und'][0]['target_id'];
    $node = node_load($nid);
    $messageid_params['uid'] = $node->uid;
    $messageid_params['nid'] = $node->nid;
    $messageid_params['cid'] = 0;
    $messageid_params['time'] = $node->created;
    if (variable_get('mailcomment_alter_subjects', 1)) {
      $subject = $message['subject'];
      $subject = variable_get('site_name', '') ? '[' . variable_get('site_name', '') . '] ' . $node->title : $subject;
      $message['subject'] = $subject;
    }
  }
  elseif (isset($message['params']['message_entity']->field_target_nodes)) {
    // This is a node.
    $nid = $message['params']['message_entity']->field_target_nodes['und'][0]['target_id'];
    $node = node_load($nid);
    $messageid_params['uid'] = $node->uid;
    $messageid_params['nid'] = $node->nid;
    $messageid_params['cid'] = 0;
    $messageid_params['time'] = $node->created;
    if (variable_get('mailcomment_alter_subjects', 1)) {
      $subject = $message['subject'];
      $subject = variable_get('site_name', '') ? '[' . variable_get('site_name', '') . '] ' . $node->title : $subject;
      $message['subject'] = $subject;
    }
  }

As I said, I'm sure Dane could find a more elegant solution to adapt the code to the commons message types, but this works right now for me.

I opened up an issue in the Mail Comment queue: #2156655: Improve Message Notify integration

However, I'm afraid I don't have time to work on this at the moment.