Here's a recipe for Qmail that I used many years ago when experimenting with using FUDFORUM and MAILMAN to get something like the functionality that OGM will provide. In the comments are also references to other recipes including POP mailboxes and a method I devised for using Mailman's external archiver functionality to effectively use Fudforum as the archive instead of Mhonarc.

These ideas should be adaptable for OGM.

http://fudforum.org/forum/index.php?t=msg&th=4519&start=0&

Thanks for the great work you are doing here.

I'm not a programmer but have tinkered with qmail quite a bit over the years so could contribute in this area. I'd be happy to help work on this once I get some breathing space in August. Ping me (anyone) if you want to collaborate on getting OGM working with qmail, which comes standard on all Plesk servers and is widely used.

Cheers,

Tobias

CommentFileSizeAuthor
#7 mailwrapper.pl_.txt1 KBtobias

Comments

tobias’s picture

tobias commented

Hi Kyle -

I am looking at the EXIM files and it seems to me that we should be able to do this quite easily with qmail on Plesk servers. Please correct me if I am wrong! :)

Here's how I would do it - can you help get the scripts to work?

1) create a dedicated domain for incoming mail for the mailing lists and have all mail for the domain routed to a single mailbox (I have set this up already through the Plesk control panel, eg foo@groups.bar.com)

2) in the qmail home directory for the mailbox (eg /var/qmail/mailnames/group.bar.org/foo)
create a file called .qmail-default containing a line like this:

|preline /usr/bin/php /var/path/to/og_mailinglist_qmail_transport.php

This, as you can imagine, pipes in the message and sends it raw to the transport script.

Cheers,

Tobias

tobias’s picture

tobias commented

Kyle -

In the transport script, what are these variables:

// Set command line arguments (sent by the exim4 transport) to variables.
$mail_username = $argv[1];
$mail_domain = $argv[2];

is it the destination username@domain? On qmail, these variables are not passed at all. The script would have to parse the email to get the destination address and thus the group name - or bounce mail to unknown username.

Cheers,

Tobias

tobias’s picture

tobias commented

Hi Kyle,

I've been doing some more troubleshooting and am not succeeding in even running the drush command. Can you shed some light? See drush output and messages below. First copy/paste is how it looks with your code. Second copy/paste is after I changed function names as advised by drush.

Meanwhile, my qmail setup is generating the same errors as below on the commandline so I think I am on the road, unless I am missing something.. here's the line in my .qmail file for the default maildrop I am using for ogm. First line includes a log record for each mailing, the second line also saves received mail to a maildir for troubleshooting. 

|preline cat | drush -v -d -r /var/www/vhosts/ml.kb2.org/httpdocs -l http://ml.kb2.org ogm-post-email >> ./qmail.log
./Maildir/

Cheers,

Tobias

-- CUT HERE ONE -- 

[root@kabissa httpdocs]# cat sites/all/modules/contrib/og_mailinglist/testing/applemail | drush -v -d ogm-post-email
Drush bootstrap phase : _drush_bootstrap_drupal_root() [0.05 sec, 2.09 MB]                                                                                    [bootstrap]
Initialized Drupal 6.17 root directory at /var/www/vhosts/ml.kb2.org/httpdocs [0.06 sec, 2.59 MB]                                                                [notice]
Drush bootstrap phase : _drush_bootstrap_drupal_site() [0.08 sec, 2.65 MB]                                                                                    [bootstrap]
Initialized Drupal site default at sites/default [0.17 sec, 2.8 MB]                                                                                              [notice]
Drush bootstrap phase : _drush_bootstrap_drupal_configuration() [0.19 sec, 2.92 MB]                                                                           [bootstrap]
Drush bootstrap phase : _drush_bootstrap_drupal_database() [0.2 sec, 2.92 MB]                                                                                 [bootstrap]
Successfully connected to the Drupal database. [0.2 sec, 2.92 MB]                                                                                             [bootstrap]
Drush bootstrap phase : _drush_bootstrap_drupal_full() [0.22 sec, 3.14 MB]                                                                                    [bootstrap]
Drush bootstrap phase : _drush_bootstrap_drupal_login() [0.56 sec, 15.47 MB]                                                                                  [bootstrap]
Found command: ogm-post-email (commandfile=og_mailinglist) [0.57 sec, 15.47 MB]                                                                               [bootstrap]
Warning: The command callback function _og_mailinglist_post_email has a deprecated name.  It must begin with drush_og_mailinglist_.  Skipping hook functions.    [notice]
[0.57 sec, 15.47 MB]
WD php: Invalid argument supplied for foreach() in /var/www/vhosts/ml.kb2.org/httpdocs/sites/all/modules/contrib/og_mailinglist/og_mailinglist_transport.inc  [error]
on line 330. [0.58 sec, 15.9 MB]
Error: Could not parse message body from the text/plain portion of the email.
Drush command could not be completed. [0.61 sec, 15.9 MB]

-- CUT HERE TWO AFTER FIXING FUNCTION NAME IN og_mailinglist.drush.inc -- 

[root@kabissa httpdocs]# cat sites/all/modules/contrib/og_mailinglist/testing/applemail | drush -v -d ogm-post-email                   
Drush bootstrap phase : _drush_bootstrap_drupal_root() [0.06 sec, 2.09 MB]                                                                                    [bootstrap]
Initialized Drupal 6.17 root directory at /var/www/vhosts/ml.kb2.org/httpdocs [0.07 sec, 2.59 MB]                                                                [notice]
Drush bootstrap phase : _drush_bootstrap_drupal_site() [0.08 sec, 2.65 MB]                                                                                    [bootstrap]
Initialized Drupal site default at sites/default [0.15 sec, 2.8 MB]                                                                                              [notice]
Drush bootstrap phase : _drush_bootstrap_drupal_configuration() [0.16 sec, 2.92 MB]                                                                           [bootstrap]
Drush bootstrap phase : _drush_bootstrap_drupal_database() [0.18 sec, 2.92 MB]                                                                                [bootstrap]
Successfully connected to the Drupal database. [0.18 sec, 2.92 MB]                                                                                            [bootstrap]
Drush bootstrap phase : _drush_bootstrap_drupal_full() [0.19 sec, 3.14 MB]                                                                                    [bootstrap]
Drush bootstrap phase : _drush_bootstrap_drupal_login() [0.52 sec, 15.47 MB]                                                                                  [bootstrap]
Found command: ogm-post-email (commandfile=og_mailinglist) [0.54 sec, 15.47 MB]                                                                               [bootstrap]
WD php: Invalid argument supplied for foreach() in /var/www/vhosts/ml.kb2.org/httpdocs/sites/all/modules/contrib/og_mailinglist/og_mailinglist_transport.inc  [error]
on line 330. [0.55 sec, 15.9 MB]
Error: Could not parse message body from the text/plain portion of the email.
Drush command could not be completed. [0.57 sec, 15.9 MB]
kyle_mathews’s picture

kyle_mathews commented

You're missing some parameters for the drush command.

Look at the og_mailinglist_exim4_transport.php file and how it is calling the drush command.


    // Save the email to file. We would just pass the email directly through
    // bash but as it turns out, there's a size limit to how big bash arguments
    // can be and emails with large attachment's exceed that limit.
    $rand_filename = "/tmp/" . rand(1000, 10000000);
    file_put_contents($rand_filename, $raw_email . "\n");
    exec("drush " . $drush_alias . " ogm-post-email " . $rand_filename . " " . $mail_username,
         $result);

The executed drush command looks something like drush @mysite /tmp/121233 test-group. That's what your qmail script will have to do as well.

BTW, are you using the latest code? I hadn't done a push of code of d.o. for quite awhile and just barely pushed the last 3-4 months of changes from github to d.o. this morning and rolled a 3rd alpha.

Also, if this isn't urgent, I'm going to be doing away with the drush system soon anyways so it might not be worth your while to get this working. The new system will use the mimemail module and will work by having a script do an HTTP POST of new emails to your drupal site. The drush thing works for the most part but it's way way too hard for most people to set up and is rather fragile as a number of files have to have the right file permissions set. See #782360: Replace the current Drush system for handling incoming mail with a mimemail-inspired POSTing system for more info.

tobias’s picture

tobias commented

Hi Kyle -

Thanks for the quick reply. I am using the latest code from github, not d.o.

What is drush @mysite /tmp/121233 test-group? Doesn't seem to match what I think drush would do. I just tried the following with no success:

drush -v -d ml.kb2.org sites/all/modules/contrib/og_mailinglist/testing/applemail test

The error:

The drush command 'ml.kb2.org ogm-post-email sites/all/modules/contrib/og_mailinglist/testing/applemail test' could not be found. [0.59 sec, 15.47 MB]

I'm happy to stop trying until you roll the next release without drush, but I am eager to help and get involved in the project. The sooner I can get it working on qmail the better, as far as I am concerned - any help from any coder to get this done would be much appreciated.

Thanks!

Tobias

kyle_mathews’s picture

kyle_mathews commented

Woops, my bad. I mis-typed the drush command.

It should be drush @mysite ogm-post-email /tmp/121233 test-group

But yeah, it's probably not worth your effort until I can roll out the new code. Hopefully I'll get to it later this week.

tobias’s picture

tobias commented
StatusFileSize
new mailwrapper.pl_.txt1 KB

Hi Kyle,

Thanks for the reply.

I'm pleased to report that this morning, with a fresh head, I was able to get this working on my qmail system with my mailwrapper.pl script (see attached) placed in the qmail default mailbox home dir eg /var/qmail/mailnames/foo/bar calling e.g. drush -r /path/to/httpdocs -l http://foo.com ogm-post-email $filename test - messages to test@foo.com are piped through and posted to the test group. I am having the problem though that it is recursively sending messages forever until I disable the dot-qmail file - presumably something related to your Exim instructions about allowed site domains which I still need to get my head around.

I am also getting this error even when it successfully posts: 

WD php: Duplicate entry '4-1' for key 1                                                                                                                       [error]
query: INSERT INTO og_mailinglist_thread
             VALUES (4, 1) in /path/to/httpdocs/sites/all/modules/contrib/og_mailinglist/og_mailinglist_api.inc on line 93.
Posted discussion for  to group  with nid=4.Drush command could not be completed.

My mailwrapper.pl script is not currently able to extract the first part of the email address of the list and send it through to drush as a variable as required, though it is able to extract the full email address. A programmer may be able to change the script to get the first part of the email address -- or could you adjust your script to accept the full email address and parse out the first part?

Would welcome input from others interested in qmail at this stage. Thanks!

Cheers,

Tobias

ps - and for future reference, mailwrapper.pl requires the Mail::Internet module from CPAN, which I was able to install easily on my system using the "cpan install Mail::Internet" command.

http://search.cpan.org/~markov/MailTools-2.06/lib/Mail/Internet.pod

kyle_mathews’s picture

kyle_mathews commented

I've removed the drush integration in favor of a much simpler method for POSTing emails. See #782360: Replace the current Drush system for handling incoming mail with a mimemail-inspired POSTing system, the function og_mailinglist_post() in og_mailinglist_transport.inc and the new installation instructions for exim. I think getting qmail working should be a lot simpler now. I'll be rolling a new alpha next week but for now, grab it from github.

tobias’s picture

tobias commented

Hi all,

I think with the new changes, I will still run into the same problem I have with my qmail wrapper not being able to extract the username portion of the email address so that the transport script can direct the message to the appropriate organic group. Would welcome a contribution from a programmer to either hack my wrapper to get this working (see above) or to write a new one that can replicate EXIM's functionality.

Alternatively, maybe Kyle could adjust his script to extract the destination username from the piped email so that it is more MTA agnostic?

Cheers,

Tobias

kyle_mathews’s picture

kyle_mathews commented

Pulling the destination username aka group name from the email w/ code sounds like a reasonable solution. Since Exim gives it to you nice and easy I'd thought other MTAs would as well.

I'll figure that out tomorrow or Tuesday and commit it.

tobias’s picture

tobias commented

that sounds great - thanks, kyle! :)

tobias’s picture

tobias commented

Hi Kyle,

I've now got OGM working on my qmail system using your new, improved and drush-free method. All that remains to get full qmail compatibility (unless I am missing something) is to get the OGM script to parse out the group username and domain name which my recipe does not extract and pass to the OGM script.

Here's the recipe so far:

1) create a domain for the email side of OGM with a default mailbox for all mail to the domain (eg foo@group.bar.org)

2) copy the og_mailinglist_exim4_transport.php file into the home dir /var/qmail/mailnames/group.bar.org/foo for the mailbox and set the ownership to be popuser:popuser

3) in the default mailbox, create a .qmail file containing the following lines: 

|preline ./og_mailinglist_exim4_transport.php foo group.bar.org >> ogmtransport.log
./Maildir/

The first line pipes the incoming email into the test@group.bar.org group and appends the ogmtransport.log file.

The second line also archives the email for troubleshooting/archival purposes.

That's it! I also had to make a tweak to the og_mailinglist_api.inc file to change the path to PEAR on my Plesk system to /usr/share/pear/Mail/mimeDecode.php

Cheers,

Tobias

kyle_mathews’s picture

kyle_mathews commented
Version: 6.x-1.0-alpha2 » 6.x-1.x-dev

Awesome! Glad it's finally working :) Your recipe looks solid and pretty easy to follow. One thing that could be improved perhaps for a noob qmail user is Step #1. I'd have no idea about how to go about setting up a domain mailbox for qmail. So if you could write a step-by-step guide for that, I think the rest can just be pushed into a qmail_og_mailinglist/INSTALL.txt file.

I added a few other options for grabbing the mimeDecode library including yours http://github.com/KyleAMathews/og_mailinglist/commit/1f90f8c05271952a9ab...

kyle_mathews’s picture

kyle_mathews commented

OGM doesn't require $mail_username anymore.

http://github.com/KyleAMathews/og_mailinglist/commit/51590faf9ab83acb86f...

tobias’s picture

tobias commented

Lovely! I have pulled your latest version from git and tested it on two lists. Your changes appear to be working perfectly, also for me on my qmail system.

I've pasted in a draft INSTALL.txt file with qmail instructions below. Feel free to adjust and use in a new qmail install folder as you like.

As regards step 1: I would hope that anybody running qmail would be familiar with the lifewithqmail.org site (http://www.lifewithqmail.org/lwq.html) which explains how to set up catchall addresses (what I called "default addresses" above). Also, if you are using plesk (as I am) then you can create the catchall using the plesk control panel. I found some video tutorials for various versions of plesk, of which this seems a typical one, for plesk version 8: http://www.webhostingresourcekit.com/flash/plesk-8-linux/plesk8linux_cat...

This is major progress - thank you! Finally I feel like I can see the results of the work you are doing and start helping in a concrete way. Here's looking forward to the beta! :)

Cheers,

Tobias

These files are for integrating OG Mailinglist with the QMAIL MTA.

# Installation Instructions (Only tested on Plesk 8.6.0 system, documentation for other systems is welcome). These instructions assume root access. 

1. Create a domain for the email side of OGM with a "catchall mailbox" for all mail to the domain (eg groups@group.yoursite.com). For Plesk 8.6.0 tutorial see http://www.webhostingresourcekit.com/flash/plesk-8-linux/plesk8linux_catchall.html - if you are not using Plesk you can also refer to http://www.lifewithqmail.org/lwq.html 

2. Copy the og_mailinglist_exim4_transport.php file from ~/exim_og_mailinglist into the home dir for the mailbox and set the ownership to be popuser:popuser using the chown command.

eg /var/qmail/mailnames/group.yoursite.com/groups  

3. In the home dir of the  mailbox, create a .qmail file containing the following lines:

|preline ./og_mailinglist_exim4_transport.php foo group.bar.org >> ogmtransport.log
./Maildir/

The first line pipes the incoming email into the test@group.yoursite.com group and appends script output to the ogmtransport.log file. (You may want to remove this after you get your script running correctly, or change >> to > to only save the output from the last time it is run)

The second line also archives the email for troubleshooting/archival purposes. (You may also want to remove this line after you get your script running correctly)

4. Create a file called sites_info.php in the mailbox home dir containing the following, tailored for your site(s). 

<?php

/*
 * @return array of sites w/ associated POST urls.
 */
function og_mailinglist_site_info() {
  return array (
    'eduglu.com' => array(
            'post_url' => 'http://yoursite.com/og_mailinglist',
            'validation_string' => 'get this from /admin/settings/og_mailinglist and paste here',
           ),
    'anothersite.com' => array(
            'post_url' => 'http://anothersite.com/og_mailinglist',
            'validation_string' => 'get this from /admin/settings/og_mailinglist and paste here',
           ),
  );
}

5. Test setup by sending email to a test group's email on your site e.g. test-group@group.yoursite.com

Questions? Better ideas? Please contribute to the QMAIL issue in the OGM issue tracker at http://drupal/node/849608 - thanks!
kyle_mathews’s picture

kyle_mathews commented

Looks lovely! Instructions commited with minor edits: http://github.com/KyleAMathews/og_mailinglist/commit/cd34b50cc43aa3392a3...

tobias’s picture

tobias commented

Great - thanks Kyle!

Please update the URL near the top, which got munged by drupal.org. It should be:

http://www.webhostingresourcekit.com/flash/plesk-8-linux/plesk8linux_cat...

Cheers,

Tobias

kyle_mathews’s picture

kyle_mathews commented

Fixed: http://github.com/KyleAMathews/og_mailinglist/commit/5d4de1a07a68cb4e90e...

kyle_mathews’s picture

kyle_mathews commented
Status: Active » Fixed

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.