Return-Path overwritten by the PHP mail() function

• Please set the status of your issues with patches to "patch (code needs review)". This will make it show up in the patch queue, so others can find it and look at it.
• For security reasons, you should use mime_header_encode() on the return path when passing it to mail(). That way, modules don't need to worry about header injection exploits.
• You should use CVS to roll your patches. It is not only much faster, but it's also much easier for us to apply and test your patch locally. You should also roll your patches from the root of a Drupal tree and with the right arguments. Check the handbook for cvs diff instructions.

All right. This should look better.
There are comments along with the code I added.

1. the default envelope Return-Path address is set: site-mail or ini_get('sendmail_from'). It will be overwritten by $from if it is present.
2. the return path is passed through mime_header_encode(), and then in the additional_parameters of mail().

Attachment	Size
Return-Path2.patch	1.4 KB

What happens if you're not using sendmail?

Nothing different than before. The patch makes sure that $return_path is not empty before applying it to the mail function.

If no sendmail value is found through ini_get('sendmail_from'), then the default site-mail value is used.
I tried with a fresh install of drupal, and if no sendmail address is found (no from, no sendmail_from and no site-mail), then the emails are sent with an empty from field.

scor, why not just set the 'Return-Path' as part of the initial assignment of the $defaults array?

This is already done, but the point of this patch is to add the -f option in the mail function, otherwise PHP will overwrite our return-path by the one set on the server. See the first post for more details.

From the PHP Mail() function page:
- The additional_parameters parameter was added in PHP 4.0.5
- The additional_parameters parameter is disabled in safe_mode and the mail() function will expose a warning message and return FALSE when used (PHP 4.2.3)

That means they could be some issue on some configurations older than PHP 4.0.5.
There is a module here : http://drupal.org/project/returnpath
Don't you think it would be good to have this basic feature in the core instead of having to use an extra module?

we only support php 4.3.3 and higher

(personally i think it's silly some created a module for a one line patch)

re The additional_parameters parameter is disabled in safe_mode and the mail() function will expose a warning message and return FALSE when used (PHP 4.2.3)
-> I added the safe_mode compatibility to the patch.

What happens if you're not using sendmail?

Steven, I got it working fine on 3 different servers: postfix, Exim 4.63 and qmail (all with safe_mode OFF).

before: Return-Path: <httpd@web9.hostingcompany.com>
after: Return-Path: <myemail@mysite.com>

is there any chance to get this committed before next month?

Attachment	Size
Return-Path3.patch	1.26 KB

Does that patch need more testing before being commited?
I've been using it for a while now without any problem.

it's a pain in the ass getting enough reviews to get something committed. but it needs testing because if it gets committed and breaks a bunch of people's sites it's a pain in everyone's butt.

I do not think this patch avoids or gracefully handles this:

The user that the webserver runs as should be added as a trusted user to the sendmail configuration to prevent a 'X-Warning' header from being added to the message when the envelope sender (-f) is set using this method. For sendmail users, this file is /etc/mail/trusted-users.

http://us3.php.net/manual/en/function.mail.php

This patch especially needs testing on a Windows server.

#252502: Sent emails rejected by mail servers when using PHP mail() was a duplicate.

Also bumping release because that "issue" (I would say "inconvenience", because drupal_mail_wrapper() exists just for this purpose) is present in all supported versions.

subscribing

I think this is a pretty important issue which I duplicated. But I also don't know of any portable way to achieve this (except not relying on PHP's mail()). The -f parameter is a sendmail option. Fortunately, all popular mail packages always come with a bin compatible with /usr/bin/sendmail. So -f should be fine across different *nixes.

Windows remains. Anyone on windows?

One key item to note here is the mail header. Unless the user that apache is running as is added to the /etc/mail/trusted-users file Sendmail will append a warning in the email header like this:

X-Authentication-Warning: webserver.hostingcompany.com: user set sender to address@somedomainname.com using -f

This is caught by most/all spam filtering agents (or at least the good ones). Make sure your provider has this configured, I have been with several that did not but after a brief call with thier support most agreed to make the change (one even changed thier server build SOP to add the apache user as a trusted user on every build!)

Subscribing. Updated patch for d5.

Attachment	Size
return_path.patch	1.05 KB

marking http://drupal.org/node/165938 as duplicate.

Here is the working patch for D7 from the duplicate issue...

Attachment	Size
do-the-return-path-correctly_9.patch	2.19 KB

The last submitted patch failed testing.

@Damien in #13:
No. This is not just an incovenience. We're passing the header wrongly to PHP's mail() function - that's nothing less than a bug in core.

We can't expect admins to install a contrib module, only to achieve Drupal's core functionality works correctly.

Plus: contrib can not even solve it correctly. Return path module (which currently fixes it) is not compatible with other mail backends (see #98412: Module interferes with smtp.module), because they all use drupal_mail_wrapper(), and it can be used only by one module at the same time.

So please let's go on testing and improving this, so this enerving bug finally gets fixed.

@Pancho: in that case, you will need to do your homework. Will this "-f" flag works with non-sendmail backends? With Postfix? With Exim? With QMail? Under Windows? In safe mode? On shared hosting?

Also note that the "we can't have several mail backend" issue is being solved for D7 in #331180: fix pluggable smtp/mail framework.

Sure. We can do all these tests on different backends, platforms and configurations. And surely I'm ready to take a major part of that work on me.

But since when is it the patch writer's duty to do all the testing work, "my homework" as you call it? I didn't say the patch were rtbc, did I? Sorry, but the way you put it is not okay.

Now, I'm working on another issue atm. But as soon as I'm done with that, I'll come back to this. Then I will check #331180, possibly rework this patch here and write a testing plan.

@22: #9 says this can work with postfix, Exim 4.63 and qmail. It won't work with safe mode (would be best not to try -f in this case). In Windows there is the possibility of ini_set()-ting sendmail_from. I have -f working very nicely on cPanel shared hosting (PHP running as Apache module) - without any X-Warning header. See also http://drupal.org/node/165938#comment-1153548). I think cPanel automatically sorts out the question of "trusted" users.

The X-Warning header may be the killer though - see #12 and #16. Even if that is not a problem, this is all a bit of a mess :P

BTW is there a use case for using returnpath and smtp contrib modules at the same time? SMTP servers seem to set the Return-Path: for you. (OK, returnpath and a mime email backend is a realistic but incompatible combination in D6/D5).

[slightly updated]

The popular PHPMailer library just does the following (simplified):

I.e. the additional flag(s) are set if defined safe_mode is disabled, and 'sendmail_from' is overidden for Windows.

The same is done for the sendmail method.

Yes...

... and if it doesn't work on your particular server (i.e. you get X-Warning headers) then you can try another sending method, e.g. sendmail (as you mention), or smtp.

FWIW I think Joomla bundles PHPMailer, but from a quick glance at the code I couldn't work out which sending method it uses or how it handles this problem.

Subscribe.