Virtual Email Passphrase Authentication

i recognize the problem that we are trying to solve, and agree that it is worth solving. but the proposed solution adds quite a bit of complexity for the admin and for the user. lets try to find a simpler solution. if we can't then the proposed solution is acceptable.

What is the likelihood that an Admin's mail priovider already supports this email address capability. For example, if I host with pair.com or rackspace or phpwebhosting or dreamhost, do I already have this feature. It seems unlikely that I could convince one of these hosting companies to provide these special features. I don't think many Drupal admins control their mail server.

I suppose that emailing a confirmation message back to the author is too cumbersome for moblog posting? This is standard for mailman lists and so forth.

Authentication via email sucks.

Hmm. I didn't mean to sound so pessimistic in my last post. I want this feature, and will likely commit it if a patch appears.

Javanaut - perhaps you could writeup some docs about how to configure a mail server for this feature? also helpful might be a list of email providers known to support this.

Javanaut - you are welcome to become Maintainer of this module. Let me know.

I didn't think of the solution where an admin just uses a catchall email address. Sounds good. You gonna write some code?

I am a bit surprised that there are still people left who are brave enough to use wildcard addresses. I thought all of them had been drowned in spam...

For more comments see:
http://lists.drupal.org/archives/drupal-devel/2004-10/msg00518.html

i don't think spam and wildcard addresses is applicable here. the entity using a wildcard address is the mailhandler inbox, not a person. further, if this address gets spammed the messages will simply be rejected as unauthenticated.

After some more thought, i agree that a catch_all mailbox is the way to go.

I noticed solves this problem without forcing the user to create a passphrase. they simply assign an email address to each user. the security is in the obscurity. if the secret email address is ever compromised, flickr provides areset button which isues you a new address. its a bit simpler than forcing the user to pick a passphrase.

I think this is pretty distinct fromthe 'report errors by email' issue.

i like how flickr handles this - http://flickr.com/account/uploadbyemail/. they issue you a secret alias email address like @mobile.flickr.com. anything sent to that address is assumed from you. you can regenerate a new alias at any time from your user profile. that protects you when you gave your address away to a mean person.

so, now this need seems solvable with just mailalias.module and a little UI help on the profile pages. i am considering removing the security feature of mailhandler and request that users use this alias method instead. any thoughts?

tokenuth module provides a secret hash per user so i'd like mailhandler to just integrate with tokenauth

I realize I'm responding to a rather old discussion, but since this issue is still open, I thought this might be useful. I like the idea of using a secret email address per user, but it may not be feasible for most people to create that many aliases. Instead, you could use "plus" addressing so that only one email address is needed but still allows a unique key per user. E.g.,

myemail+1234@example.com
myemail+abcd@example.com

These should both be delivered to myemail@example.com, but mailhandler could then look at the "1234" or "abcd" and see if that secret key matches with the From address on the email.

The biggest issue with this is that not all mail providers support it. Some support a minus sign instead of the plus sign, some support both, and some support neither (see this Wikipedia entry). A solution might be to allow configuration of which symbol to use; alternately just support plus addressing, and if someone's email hosting doesn't support it, they can create an address on a free service like Gmail that supports POP. According to Wikipedia, there is also the issue of "some systems ... refusing to send mail addressed to a user on another system merely because the local-part of the address contains the plus sign (+)." I don't know how common a problem this is, however.

subscribing

I need this feature as well.

Has this been done for Mailhandler yet? I noticed Mailalias module, but it's still stuck in the D5 Era. I would argue that this should be in Mailhandler by default as an optional security feature.

I'd like to know what people are doing to provide this extra layer of authentication, because as with the original poster, I realize it's easy to spoof email addresses and thus hijack accounts.

As a "proof of concept" i just wrote this "exploit" to test out how easy it was to spoof. Turns out, pretty friggin simple. I would have hoped that gmail (which i'm using for my mailhandler drop) would have spam filtered it out, but this was not the case.

Here's the "code"

echo "I just posted under your name." | mail -a "From: person-to-hijack@theiremail.com" -s "Subject is I spam you" mailhandler@yourmailhandleraddress.com

For all those who don't think this is a problem, please provide me with your email address and your mailhandler email address and I'll make some money off viagra affiliates.

I'm going to need to code this for my project, so if anyone wants feature requests right now I would suggest asking for them.

Right now, I'm going to create unique unique random auto-generated keys which when users submit their emails to the mailhandler address, it will get verified againsnt that and there from address. If it fails, I'll skip it and mark the email as read or delete it depending on the settings.

I'll display this key privately to the users profile page in a tab or secondary menu or what ever drupal calls it.

I'll add this as an optional configuration Mailhandler config under "Use Password".

I'll assume that most people will be using a catch-all, but I'll also provide a hook which people who do not want catch-alls, can create the email alias/address for that user (and perhaps also change the auto-generated key, which is to be used).

Anything else people want? or things I'm not thinking of.

I'll also be creating this as a patch for mailhandler of HEAD of the D6 version, as I believe it's an integral security "feature" which should be added by default. What do you guys think?

User tokens is a feature of tokenauth module.

currently token auth does not provide this functionality, but it appears the author would like to integrate it with mail handler as it's in the TODO.

j0rd, did you move forward on this ? I need similar functionnality for a mobile site too.
I was planning on doing the same as you except that I don't want users to have to type their auth key everytime. I want to setup up a generic mailhandler account mailhandler@example.com and then assign addresses to users such as mailhandler-SDSDIZENFSD@example.com and mailhander-2348FSN23@example.com. They can then send their emails to those two addresses and I will be able to match the account. This requires a simple catch-all account and, if you're using exim a simple suffix configuration.

I'd be willing to code into your patch/module my way of doing it if you design it so it can support both, there is so much in common in our solutions.

Thanks

I've yet to start the work, but still require this feature. I have a project that will require this which I'll finish in the next month or so.

I'll post my solution when it's available.

Ok, I'll take that as a "Thanks, but I'm not interested".

@z.stolar: going along with my other refactoring post. I'd suggest we slightly refactor the module so that
- mailhandler_authenticate is in fact a hook (of which there can be only one implementation enabled at a time)
- mailhandler offers a module which implements the standard way of doing it
- mailalias_user would be changed so that the code from lines 371 to 379 are implemented in that hook

Then I can provide a custom module back to the community and it will support future versions of mailhandler. And anyone else who comes up with their own need can handle it.

Would this be a good plan ?

@PGiro: I'll happily examine any improvements to mailhandler, as patches or stand-alone modules (preferable in your case), and if needed, introduce a new hook.

@z.stolar: ok, will do then.

Here is my tentative patch to open up the discussion. I have yet to get mailsave_imagefield to work so I can fully test it.

I have made the process_message method call a private _authenticate method. This one checks that there exists an implementation of mailhandler_authenticate somewhere in the PHP code. If there is, it calls it.
I have provided 2 modules
- mailhandler_fromauthentication.module which corresponds to the current code in mailhandler_authenticate() when mailalias is not installed
- mailhandler_fromaliasauthentication.module which corresponds to the current code in mailhandler_authenticate()

So really it's now easy to plug in any authentication scheme anyone would like and make them separate modules.

Why didn't I use the typical drupal pattern of "module_implements" : because I never want to rely on From: to authenticate...

@z.stolar : can you find time to check my patch ?

@PGiro: I'll do my best. I'm in a rather busy time now.
I appreciate your efforts. If others in this thread are willing to review it, it'll be a great help!

I applied PGiro's patch (#26) and have implemented to "from address + to alias" authentication method using the framework this patch provides. See attached file if you require this. It uses Token Auth as moshe has recommended (#14).

While I see PGiro's approach abstracting the authentication for Mailhandler is great, I think the method of how it currently operates could be improved using hooks. The way PGiro has done it (and I understand why he's done it this way) involves abstracting the authentication function out into a module and then enabling a single module of authentication. While this works, I've never seen another module do this and it feels unnatural to a seasoned Drupal dude.

Perhaps it could be done, by leaving the default authentication method in the mailhandler module and allowing other modules to provide authentication methods....then provide Mailhandler with an entry point, in which you can choose a different authentication method on a configuration page. Perhaps it's best if someone takes a look and improves how this is done before more authentication modules are written.

DESCRIPTION:
This module uses authenticates off the from address, much like the default mailhandler module.
It also provides another check which is based off the email address the user sends his email to.
This will require that your mailserver is setup to accept "catch-all" emails on the domain.

EXAMPLE:

1. In mailhandler you set up a mailbox for foo@gmail.com
2. You've enabled Mailhandler To Alias Authentication module and disabled all other auth modules
3. You login as "myuser" and visit his My Account -> Token Authentication page and grab the To Authentication addresss. This module uses the format of foo-$TOKEN@gmail.com for aliases.
4. You email this "to alias" email address using "myuser" email address.
5. Run cron to pull in the mailhandler emails
6. Mailhandler will check that the email came from "myusers" email address and then check to see if it was also sent to "myusers" unique "to alias" address provided from his Token Authentication page.
7. If both these conditions are true, we will create the post for this user.
8. Otherwise, we will try to submit the node as anonymous (default by mailhandler auth).

@j0rd: you are absolutly right, I didn't do it the usual drupal way by using hooks. There reason being that hooks allow multiple modules to responde to an event but in our case, we only want one module doing the authenticating which, I felt, was different

So your patch requires my patch ? Is that it ?
Because I'd like to reuse your form_alter stuff which I haven't gotten around to writing yet!

@z.stolar : can we get some feedback so j0rd and I can move on to what next be done ?

@PGiro: I didn't go over *all* lines of the patch, but have gone deep enough to notice quite a few issues with it. For example, I don't like the fact that your module uses mailhandler's name space for its hook. Also, various syntax and coding standards errors were observed.

I acknowledge the importance of this feature, but I think that if you already took the separate module path, you should try and implement as much as possible in that module. As I said above, if you need the addition of a hook in mailhandler, or any other changes for that matter, please submit a patch for these changes.

@z.stolar: needless to say I am disappointed in your response.

I have submitted said patch and have explained why the method is in the mailhandler namespace. I would rather have discussed the merits of that versus a real drupal hook rather than just a "you're using a method in my namespace" which I believe makes sense in the context I described (we don't want multiple modules implementing the hook, unlike all other drupal hooks I know of). Like j0rd said, this is non-standard and maybe there's a better way. I was hoping for a discussion.

I don't know what you mean by "took the separate module path". In fact, I have not : I provided an explanation of what I thought we could do to provide the requried functionality and I asked for some feedback from you. Since I did not get a response on that and you asked for a patch, I submitted the patch.
Now, I get a response about the coding standards and syntax which brings nothing to the discussion except give me the feeling I am wasting my time trying to coordinate with you and contribute.

Do you want to discuss this or not ? If so, please review my patch on the merits of the functionality it provides and please don't just list a catalog of coding standard errors, that's beside the point at this stage of the discussion.
If not, just say you don't care/don't have the time/don't want to so I stop wasting my time trying to plug into your project.

@PGiro: Allow me to apologize for my brief answer.

Let's understand what it's all about. First and most, the discussion IS open, and was never shut down. My comments about the syntax/standards, wasn't the most important one, but it is important to understand that it's easier to /read/ a patch, when it follows the standards. Such a patch has a higher chance of being processed. Certainly it is not the issue itself though.

The separate module path: What you did was to create a new (sub)module of mailhandler, which in my opinion is the right path. What I meant to say, is that you should not implement mailhandler's (new) hook in your module, as if it was mailhandler's (talking about mailhandler_authenticate). You should rather use your new module's name space for that.

As stated above - the discussion is open, and anyway - who am I to close it?
I don't mean to act as a single moderator, and if there will be a collective understanding that your patch is ready and necessary, I won't stand in your way.
However, I don't promise to accept every solution, even if it's a good one. Again - this decision should be collective (more than you and me ;-) ), and the patch should be tested and reviewed by a larger public, before it is applied.

I hope this satisfies you, and again - sorry for being too short of words earlier.

I've just read through this issue. While there should only ever be one authentication method on a mailbox, do people see the need to allow for multiple authentication methods in order that one mailbox may use one method, while another uses a different method? It appears that the patch is one authentication method module-wide, as opposed to per-mailbox, but maybe I am missing something. If it were per-mailbox, you could have mailhandler's mailhandler_authenticate() look at the mailbox it is acting on. When you setup a mailbox, you could invoke some mailhandler_authenticate() hook to find modules that offer authentication means, and then when you save the mailbox, you save the authentication method w/ that mailbox to the database. What do you think about this?

@z.stolar: I'll answer your comment in the upcoming days. I've been working on something else at the moment

I like the idea of selecting an authentication method per mailbox. I don't think this is all that important, but it's easily coded.

I like the idea of "contrib" modules supplying mailhandler with extra methods of authencation via hooks.

I like the idea of having the authentication methods being selectable on the configuration page, again because this is easily done.

I like the idea of having it default to the current authentication method available in the module.

--
It wouldn't be much work from the patch PGiro has already supplied.

We have implemented this feature without even needing a patch - mailhandler allows you to specify a header used for looking up From address. We have simply set it to be "To" header. Then used mailalias.module to give every user some alias (like "andy-random123123@drupal.org"). User sends email to andy-random123123@drupal.org, which is a wildcard for some mailbox. Mailhandler gets the email, finds the "From" in "To" header (that's the tricky part :-) and assigns a proper user. Done.

Is there a catch in our approach? :)

Attaching a proof of concept patch (will be perfected later) to implement a generic authentication framework for mailhandler. What it does/allows:

hook_mailhandler_authenticate

Any module can implement hook_mailhandler_authenticate and will receive email headers and body to match users. Example: Match a special X-MyModule-From: XYZ header to own database table.
If at least one (first) module returns a valid user email, authentication succeeds.

Authentication fallback

Because there needs to be a fallback and currently, there needs to be a way to disable this fallback, I have implemented new setting for every mailbox, called "Authenticate using From header". It defaults to true and all current mailboxes will be updated to have it enabled (= it's backwards compatible).

Why? If you implement the hook and then still allow matching to a From address, everybody can still fake users by knowing their from. You need to be able to disable this behavior by disallowing matching the From: in authenticate at all

Possible change: Automatically disallow From matching if there is at least one authenticating module?

Compatibility

This aims to be almost backwards compatible. If you are not using mailalias module, the behavior shouldn't change. If you are using mailalias module, you need to upgrade it too (patch will follow into the mailalias queue - simply implement the new hook)

@meba - I am thinking each mailbox should store the name of the callback for its authentication method so that it may be called explicitly instead of the idea that as long as one of the authentication methods succeeds, proceed. This would still use a hook for declaration of authentication methods and on the form you would select a default for each mailbox. A hook should be able to implement more than one authentication method which would allow the mailhandler module itself to ship with the current method, virtual passphrase, and "no authentication", for example.

What do you think of the following (untested code):

Implements info and execution hook

/**
 * Defines and executes message authentication methods
 * 
 * @param $op
 *   String info or execute
 * @param $name
 *   String identifier for authentication method
 * @param $args
 *   Array of arguments to pass in to the authentication method callback
 */
function mailhandler_mailhandler_authenticate($op, $name = NULL, $args = array()) {
  switch ($op) {
    case 'info':
      $methods = array();
      foreach (module_list() as $name) {
        if (module_hook($name, 'mailhandler_authenticate_info')) {
          $function = $name .'_mailhandler_authenticate_info';
          $methods[] = $function();
        }
      }
      return $methods;
    case 'execute':
      $method = array();
      foreach (module_list() as $name) {
        if (module_hook($name, 'mailhandler_authenticate_info')) {
          $function = $name .'_mailhandler_authenticate_info';
          $methods = $function('info');
          // TODO - May not be found, like if providing module is disabled.  Must fail gracefully
          if (array_key_exists($name)) {
            if ($method[$name]['extension'] && $method[$name]['basename']) {
              module_load_include($method[$name]['extension'], $method[$name]['module'], $method[$name]['basename']);
              $method[$name]['callback']($args);
            }
            else {
              drupal_load('module', $method[$name]['module']);
              $method[$name]['callback']($args);
            }
            break;
          }
        }
      }
      break;
  }
}

Modules define their authentication methods

/**
 * Implementation of hook_mailhandler_authenticate_info()
 * 
 */
function mailhandler_mailhandler_authenticate_info() {
  return array(
    'default' => array(
      'title' => 'Default',
      'description' => 'Checks whether the sender matches a valid user in the database',
      'callback' => 'mailhandler_authenticate_default',
      'module' => 'mailhandler',
      'extension' => NULL, // as in $type in module_load_include
      'basename' => NULL, // as in $name in module_load_include
    ),
  //'virtual' => array(
  //  // todo
  //),
  );
}

The default authentication method currently used in mailhandler

function mailhandler_authenticate_default($node, $header, $origbody, $mailbox) {
  // $fromaddress really refers to the mail header which is authoritative for authentication
  list($fromaddress, $fromname) = mailhandler_get_fromaddress($header, $mailbox);
  if ($from_user = mailhandler_user_load($fromaddress, $node->pass, $mailbox)) {
    $node->uid = $from_user->uid; // success!
    $node->name = $from_user->name;
  }
  else if (function_exists('mailalias_user')) { // since $fromaddress failed, try e-mail aliases
    $result = db_query("SELECT mail FROM {users} WHERE data LIKE '%%%s%%'", $fromaddress);
    while ($alias = db_result($result)) {
      if ($from_user = mailhandler_user_load($alias, $node->pass, $mailbox)) {
        $node->uid = $from_user->uid; // success!
        $node->name = $from_user->name;
        break;
      }
    }
  }
  if (!$from_user) {
    // failed authentication. we will still try to submit anonymously.
    $node->uid = 0;
    $node->name = $fromname; // use the name supplied in email headers
  }
  return $node;
}

Calling code, which was (until today) in mailhandler_retrieve_message

Used to be:

$node = mailhandler_authenticate($node, $header, $origbody, $mailbox);

and would now be:

$node = mailhandler_mailhandler_authenticate ('execute', $mailbox['authentication'], array($node, $header, $origbody, $mailbox);

The info hook would be called on the settings page for when making/editing a new mailbox in order to retrieve/list the available authentication methods, which should also handle when an authentication method disappears for which a mailbox is using.

Here is working code and update path of the above concept with some fixes.

Attached is a new patch that, in addition to the default email address-based authentication, implements token auth authentication along the lines of @j0rd's #29

I committed the patch from #42 along with additional documentation. If you apply this patch note you must run update.php. Additional rough edges to clean up include:

* support for mail aliases when using the tokenauth authentication plugin
* more documentation about the authentication plugin system
* better fail-over when a plugin is missing

The three tasks in #43 are complete. I will leave this open for additional feedback until the next release of mailhandler is made.

Fixed

I just downloaded/installed the 6.x-1.10 code and have found that the Message authentication method always reverts to "Mailhandler Default" even if I set it to Mailcomment Default. Debugging now, but I wanted to notify you/others about the bug.

I'm not sure exactly what was happening, but I think it was caused by a problem on my site. The form was actually throwing an "invalid submission" error because no input format was selected but there was no input format only because someone "accidentally" deleted it.

We could potentially add a condition to the hook_validate to give a better error message about the input format, but that's for a different issue.

subscribing just to have this excellent discussion in the list of my posts.

I have another question on this issue. This is the tokenauth module for mailhandler?

I want to receive emails to my server at an address like foo@myserver.com.
I would like all mail received at that address to be processed by mailhandler.
I would also like to use the token auth, so really mail is sent to foo+token@myserver.com.

I DO NOT want to check the from addresses at all. I want the authentication just based off the token in the To: address. If the token is in the To address, then authentication based on whatever user has that token, would be ideal, but otherwise just making sure the token matches a particular existing user is fine.

This way, someone can send from ANY address, and still post via tokens.

Is this possible with the current configuration? If not, can someone point to where in the php code and what file so I can comment this out?

Thanks