Listhandler for 5.x
Shiny - March 9, 2007 - 04:01
| Project: | Listhandler |
| Version: | 5.x-1.x-dev |
| Component: | Code |
| Category: | task |
| Priority: | critical |
| Assigned: | philipnet |
| Status: | patch (code needs review) |
Description
have you started making this 5 compatible?
If not, i volunteer.
#1
Just another user that would love to see how this would work for my site running Drupal 5.
I'm running a site trying to consolidate a lot of different sources. Both blogs, site rss feed, and more than one mailing list. Being able to feed the mailing lists into individual forums would get great.
#2
Shiny; I'd love to review some patches.
I currently don't use the module myself, so I don't have plans for an update.
#3
code underway.
#4
#5
Shiny, this may or may not be useful.. I spent a few hours going through the upgrading from 4.7 to 5.1 documentation and took a stab at getting this to work.. but then found out the server doesn't have the php imap module..so mailhandler isn't able to pull in messages - which I think is required.
Also..what I thought I would find but didn't -
I couldn't figure out how to tell it which "forum" to look for "mailhandler messages" that it would catch and send the web-forum responses back to the list..or perhaps by the sql query it does it "knows" which are? Not sure quite yet..
When saving options in the admin menu, it says they save, but they don't. I'm sure this has to do with my attempt at modifying the forms to upgrade them to 5.1
It was an attempt (that isn't working yet), so it may or may not assist you!
<?php
function listhandlher_help($section) {
$output = '';
switch($section) {
case 'admin/help#listhandler':
$output = '<p>'. t('The listhandler module allows you to connect mailing lists to forums and vice versa. It works in conjunction with the mailhandler module. Mailhandler receives an email and then asks listhandler if the received email is part of a list. If the email is from a mailing list associated with a forum on your site, then listhandler adds the recieved email to the forum.') .'</p>';
$output .= '<p>'. t('Listhandler administration allows you to set the email address of the list administrator. The email address is used as the From: field to check the address of messages sent by anonymous users. You can also enable and disable the listhandler for a list.') .'</p>';
$output .= t('<p>You can:</p>
<ul>
<li>administer listhandler <a href="!admin-listhandler">administer >> listhandler</a>.</li>
<li>administer mailhandler <a href="!admin-mailhandler">administer >> mailhandler</a> to create a mailbox which list handler can work with.</li>
<li>administer forum <a href="!admin-forum">administer >> forum</a>.</li>
', array('!admin-listhandler' => url('admin/content/listhandler'), '!admin-mailhandler' =>
url('admin/content/mailhandler'), '!admin-forum' => url('admin/content/forum'))) .'</ul>';
$output .= '<p>'. t('For more information please read the configuration and customization handbook <a href="!listhandler">Listhandler page</a>.', array('!listhandler' => 'http://www.drupal.org/handbook/modules/listhandler/')) .'</p>';
return $output;
case 'admin/modules#description':
$output = t('Connect your mailing lists to your drupal site and your drupal site to your mailing lists.');
break;
case 'admin/content/listhandler':
$output = t("First go to the <a href='!url'>mailhandler configuration screen</a> and add some mailboxes. A mailbox becomes recognised as a mailing list if the 'Second E-mail address' is set. This is not optional for mailing lists.", array('!url' => url('admin/mailhandler')));
break;
}
return $output;
}
/**
* Permissions for this module.
*/
function listhandler_perm() {
return array('administer listhandler');
}
function listhandler_menu($may_cache) {
$items = array();
if ($may_cache) {
$items[] = array('path' => 'admin/content/listhandler',
'title' => t('Listhandler'),
'callback' => 'listhandler_admin',
'access' => user_access('administer listhandler'));
}
return $items;
}
/**
* Configuration options for this module
*/
function listhandler_admin_form($form) {
$form['listhandler_from'] = array(
'#type' => 'textfield',
'#title' => t('Admin address'),
'#default_value' => variable_get('listhandler_from', ''),
'#size' => 50,
'#maxlength' => 100,
'#description' => t('The email address of the list administrator. Used as From: address of messages sent by anonymous users.'));
$form['listhandler_strip_title'] = array(
'#type' => 'textfield',
'#title' => t('Strip title'),
'#default_value' => variable_get('listhandler_strip_title', 'Re: , AW:,[my list]'),
'#size' => 50,
'#maxlength' => 100,
'#description' => t("A comma delimited sequence of strings that should be stripped from the titles of topics and comments posted to the forums. You can use it to remove certain tags like 'Re: ' or '[My Mailinglist]'."));
$form['listhandler_accountstatus'] = array(
'#type' => 'select',
'#title' => t('Account status'),
'#default_value' => variable_get('listhandler_accountstatus', 0),
'#options' => array(0 => t('blocked'), 1 => t('active')),
'#description' => t('The status of accounts created by listhandler.'));
$form['listhandler_list']['#tree'] = TRUE;
$form['listhandler_prefix']['#tree'] = TRUE;
$result = db_query("SELECT * FROM {mailhandler} WHERE mailto != '' ORDER BY mailto");
while ($mailbox = db_fetch_object($result)) {
$form['listhandler_list'][$mailbox->mid] = array(
'#value' => "<div><a href=\"mailto:$mailbox->mailto\">$mailbox->mailto</a></div>");
$form['listhandler_prefix'][$mailbox->mid] = array(
'#type' => 'textfield',
'#title' => '',
'#default_value' => $mailbox->prefix,
'#size' => 50,
'#maxlength' => 100);
}
$form[] = array(
'#type' => 'submit',
'#value' => t('Submit'),
);
return $form;
//OLD return drupal_get_form('listhandler_settings', $form);
}
/**
* Theme call to render the tabular form of mailing lists and prefixes
*/
function theme_listhandler_settings($form) {
$output = drupal_render($form['listhandler_from']);
$output .= drupal_render($form['listhandler_strip_title']);
$output .= drupal_render($form['listhandler_accountstatus']);
if (count($form['listhandler_list'])) {
$rows = array();
foreach(element_children($form['listhandler_list']) as $mid) {
$row = array();
$row[] = drupal_render($form['listhandler_list'][$mid]);
$row[] = drupal_render($form['listhandler_prefix'][$mid]);
$rows[] = $row;
}
$header = array(t('mailing list'), t('prefix'));
$output .= theme('table', $header, $rows);
$output .= '<div>'. t("The prefix well be prepended to all outgoing mail subjects for the specified list. It is a good idea to put the prefix in the 'strip title' field.") .'</div>';
}
else {
$output .= '<div>'. t("No mailing lists have been defined. Please go to the <a href='!url'>mailhandler configuration screen</a> and add some mailboxes. A mailbox becomes recognised as a mailing list if the 'Second E-mail address' is set.", array('!url' => 'admin/mailhandler')) .'</div>';
}
$output .= drupal_render($form);
return $output;
}
function listhandler_admin($form) {
$op = $_POST['op'];
$edit = $_POST['edit'];
if ($op == t('Submit')) {
variable_set('listhandler_from', $edit['listhandler_from']);
variable_set('listhandler_strip_title', $edit['listhandler_strip_title']);
variable_set('listhandler_accountstatus', $edit['listhandler_accountstatus']);
if (isset($edit['listhandler_prefix'])) {
foreach ($edit['listhandler_prefix'] as $key => $value) {
db_query("UPDATE {mailhandler} SET prefix = '%s' WHERE mid = %d", $value, $key);
}
}
drupal_set_message('The configuration options have been saved.');
}
$output = drupal_get_form('listhandler_admin_form', $form);
return $output;
}
/**
* nodeapi hook
*
*/
function listhandler_nodeapi(&$node, $op, $arg = 0) {
switch ($op) {
case 'update': // what to do with updates?
break;
case 'insert': // mail new forum topics to mailing list.
if ($node->type == 'forum') {
if($node->sentbylisthandler) { // avoid running an infinite loop.
// save ids in listhandler table.
db_query("INSERT INTO {listhandler} (msgid, nid, cid, pid, uid, mid, tid) VALUES ('%s','%s','%s','%s','%s','%s','%s')", $node->message_id, $node->nid, 0, 0, $node->uid, $node->mid, $node->tid);
// call watchdog
watchdog('listhandler', "Forum '". $node->title ."' inserted.");
}
else {
$node->subject = $node->title;
$node->comment = $node->body;
listhandler_send_mail((array) $node);
}
}
break;
}
}
/**
* Called by comment.module after submitting a comment.
* Sends comments to specified addresses.
*
* $type: type of call, here insert is expected.
*
* $edit: Array containing the comment.
* Important fields:
* $edit["subject"]: Oh well.
* $edit["comment"]: The text
* $edit["nid"]: commented node id
* $edit["cid"]: comment id
* $edit["pid"]: cid of parent
* $edit["uid"]: user id
* $edit["mid"]: id of mailbox
* $edit["tid"]: taxo id of the forum
*/
function listhandler_comment($edit = array(), $type) {
if($type == "insert") { // only accept calls by the comment insert hook.
if($edit["sentbylisthandler"]) { // avoid running an infinite loop.
// save comment ids in listhandler table.
db_query("INSERT INTO {listhandler} (msgid, nid, cid, pid, uid, mid, tid) VALUES ('%s','%s','%s','%s','%s','%s','%s')", $edit["message_id"], $edit["nid"], $edit["cid"], $edit["pid"], $edit["uid"], $edit["mid"], $edit["tid"]);
// call watchdog
watchdog('listhandler', 'Comment '. $edit["subject"] ." inserted.");
}
else {
if(filter_access($edit['format']) && $edit['subject'] == truncate_utf8(decode_entities(strip_tags(check_markup($edit['comment'], $edit['format']))), 29, TRUE)) {
// subject was autogenerated, get it from parent instead
if ($edit['pid']) {
$parent = db_fetch_object(db_query('SELECT * FROM {comments} WHERE cid = %d', $edit['pid']));
$edit['subject'] = truncate_utf8(decode_entities(check_plain(t('Re: ') . $parent->subject)), 29, TRUE);
}
// no parent, get node title
else {
$parent = db_fetch_object(db_query('SELECT title FROM {node} WHERE nid = %d', $edit['nid']));
$parent->subject = truncate_utf8(decode_entities(strip_tags(check_markup($parent->title, $edit['format']))), 29, TRUE);
$edit['subject'] = t('Re: ') . $parent->subject;
}
db_query("UPDATE {comments} SET subject = '%s' WHERE cid = %d", $parent->subject, $edit['cid']);
}
listhandler_send_mail($edit);
}
}
}
function listhandler_send_mail($edit) {
global $user, $base_url;
if(!$user->uid) {
$edit["name"] = variable_get("anonymous", "Anonymous");
$edit["mail"] = variable_get("listhandler_from", "");
}
else {
$edit["name"] = $user->name;
$edit["mail"] = $user->mail;
}
// Find the list to send to.
$mboxen = array();
// taxo terms of the node.
if (count($edit['taxonomy'])) {
foreach ($edit['taxonomy'] as $taxoterm) {
// Now for the mailbox.
switch ($GLOBALS['db_type']) {
case 'mysql':
case 'mysqli':
$result2 = db_query("SELECT * FROM {mailhandler} m WHERE m.commands REGEXP '%s'", 'tid:[[:blank:]]*'. $taxoterm .'[[:>:]]');
break;
case 'pgsql':
$result2 = db_query("SELECT * FROM {mailhandler} m WHERE m.commands ~ '%s'", 'tid:[[:blank:]]*'. $taxoterm .'[[:>:]]');
}
while ($mbox = db_fetch_object($result2)) {
$mboxen[] = array($mbox->mid, $mbox->mailto, $taxoarr->tid);
}
}
}
else {
$result = db_query("SELECT t.tid FROM {term_node} t WHERE t.nid = %d", $edit["nid"]);
while ($taxoarr = db_fetch_object($result)) {
// Now for the mailbox.
switch ($GLOBALS['db_type']) {
case 'mysql':
case 'mysqli':
$result2 = db_query("SELECT * FROM {mailhandler} m WHERE m.commands REGEXP '%s'", 'tid:[[:blank:]]*'. $taxoarr->tid .'[[:>:]]');
break;
case 'pgsql':
$result2 = db_query("SELECT * FROM {mailhandler} m WHERE m.commands ~ '%s'", 'tid:[[:blank:]]*'. $taxoarr->tid .'[[:>:]]');
}
while ($mbox = db_fetch_object($result2)) {
$mboxen[] = array($mbox->mid, $mbox->mailto, $taxoarr->tid);
}
}
}
// Find parent's MsgID if received by listhandler.
if ($edit["pid"]){
// We should get only one or no result.
$result = db_fetch_object(db_query("SELECT * FROM {listhandler} WHERE cid = %d", $edit['pid']));
}
else {
$result = db_fetch_object(db_query("SELECT * FROM {listhandler} WHERE nid = %d", $edit['nid']));
$edit['pid']=0;
}
if($result) {// Parent was sent or received by listhandler
$mid = "References: ". $result->msgid ."\n";
$mid .= "In-Reply-To: ". $result->msgid ."\n";
}
//common headers
$dir = str_replace("/", ".", substr(strchr(str_replace("http://", "", $base_url), "/"), 1));
foreach ($mboxen as $mbox) {
$edit["mid"] = $mbox[0];
$address = $mbox[1];
$edit["tid"] = $mbox[2];
// We add tokens to the MID. Replies to comments sent by listhandler can thus be classified easier.
$msgid = "<listhandler=". $edit["mid"] ."&site=". $dir . $_SERVER["SERVER_NAME"] ."&nid=". $edit["nid"] ."&pid=". $edit["pid"] ."&cid=". $edit["cid"] ."&uid=". $user->uid ."&tid=". $edit["tid"] ."&". md5($edit["nid"] . $edit["cid"]) ."@". $dir . strtolower($_SERVER["SERVER_NAME"]) .">";
$mid .= "Message-Id: ". $msgid ."\n";
// send mail
$file = drupal_get_path('module', 'listhandler') .'/mail.inc';
if (file_exists($file)) {
include_once($file);
}
if (function_exists('mail_html_to_text')) {
$subject = mail_html_to_text($edit['subject']);
$body = mail_html_to_text($edit['comment']);
}
else {
$subject = strip_tags($edit['subject']);
$body = strip_tags($edit['comment']);
}
// OLD $err = user_mail($address, $subject, $body, "From: ". $edit["name"] ." <". $edit["mail"] .">\n". $mid);
$err = drupal_mail($mid, $address, $subject, $body, $edit["mail"]);
// save comment ids in listhandler table.
// Not strictly needed, the same info is stored in the msgid.
db_query("INSERT INTO {listhandler} (msgid, nid, cid, pid, uid, mid, tid) VALUES ('%s','%s','%s','%s','%s','%s','%s')", $msgid, $edit["nid"], $edit["cid"], $edit["pid"], $user->uid, $edit["mid"], $edit["tid"]);
// call watchdog
watchdog('listhandler', t("'!s' sent to '!a'.", array("!s" => $edit["subject"], "!a" => $address)));
}
}
/**
* Called by mailhandler after a new mail has been received.
*
* Only the first and last argument are used.
*
* $node: The new node or comment
* $stream: The number of the opened imap stream.
* $msg_number: The number of the current message in the mailbox.
* $mailbox: Array containing info about the mailbox the message is from.
* $header: Some, but not all headers (From, To, message_id, ...) of the current message.
*/
function listhandler_mailhandler($node, $result, $msg_number, $header, $mailbox){
global $base_url;
if($node->tid) { // If no tid is assigned, then the mail is not comming through a mailhandler enabled list. Do nothing.
$node->cid = 0; // Might have a cid from the msg id. This will confuse comment_post. Updating comments via mail does not make sense anyway. GK
// Override value from msgid. Might be crossposted.
$node->mid = $mailbox['mid'];
// We need to ensure that no messages that have been sent by listhandler on this site (!)
// get processed. We would get an infinite loop on a mailing list.
$dir = str_replace("/", ".", substr(strchr(str_replace("http://", "", $base_url), "/"), 1));
$regex = "<listhandler=[[:print:]]+@". $dir . strtolower($_SERVER["SERVER_NAME"]) .">";
if(ereg($regex, $header->message_id)){
// Sent by listhandler on this site. Destroy the node.
watchdog('listhandler', t("Discarded message '!s' to avoid loop.", array("!s" => $node->title)));
unset($node);
return;
}
else {
// Debug
//if($node->site != $dir . $_SERVER["SERVER_NAME"]) { // parent (!) not sent by listhandler on this site.
if ($node->uid == 0) { // mailhandler did not find the author, create a new account
$node = listhandler_create_author($node, $header);
}
//}
// Strip stuff from title
$replace = explode(',', variable_get('listhandler_strip_title', ''));
$replacements = array();
foreach ($replace as $v) {
$replacements[trim($v)] = '';
}
unset($replacements['']);
if (count($replacements)) {
$node->title = trim(strtr($node->title, $replacements));
}
if (trim($node->title) == '') {
$node->title = truncate_utf8(decode_entities(strip_tags(check_markup($node->body, 1))), 29, TRUE);
}
$node = listhandler_find_parent($node, $header);
$node->message_id = $header->message_id;
$node->sentbylisthandler = true;
return $node;
}
}
else { // allow other modules to have that node unaltered.
return $node;
}
}
/**
* Create account for unknown authors of a new mailsubmitted node or comment.
* Admin is able to choose if it will be
* a real account or a blocked one (see conf).
* if no account can be created (email or name already taken) the name and address
* are prepended to the node body and posted as anonymous.
*/
function listhandler_create_author($node, $header){
$from = $header->from;
$from_address = strtolower(sprintf("%s@%s", $from[0]->mailbox, $from[0]->host));
// decode encoded author name
$from_name = '';
$from_arr = imap_mime_header_decode($header->from[0]->personal);
for($i = 0; $i < count($from_arr); $i++) {
if ($from_arr[$i]->charset != 'default') {
$from_name .= drupal_convert_to_utf8($from_arr[$i]->text, $from_arr[$i]->charset);
}
else {
$from_name .= $from_arr[$i]->text;
}
}
$from_name = trim($from_name, '"');
if ($from_name == '') {
$from_name = $from_address;
}
// check if name is available
if (db_num_rows(db_query("SELECT name FROM {users} WHERE LOWER(name) = LOWER('%s')", $from_name)) > 0) {
$node->body = t('Message from !n !a', array('!n' => $from_name, '!a' => $from_address)) ."\n\n". $node->body;
watchdog('listhandler', t("Cannot create account: The name '!s' is already taken.", array("!s" => $from_name)));
}
// check if address is valid
elseif (!valid_email_address($from_address)) {
$node->body = t('Message from !n !a', array('!n' => $from_name, '!a' => $from_address)) ."\n\n". $node->body;
watchdog('listhandler', t("Listhandler: Cannot create account: The email address '!s' is not valid.", array("!s" => $from_address)));
}
// create account
else {
$empty_account = new StdClass();
$from_user = user_save($empty_account, array('name' => $from_name, 'pass' => user_password(), 'init' => $from_address, 'mail' => $from_address, 'roles' => array(DRUPAL_AUTHENTICATED_RID), 'status' => variable_get('listhandler_accountstatus', 0)), 'account');
watchdog('listhandler', t('Created account for !fa, with roles !rid', array ('!fa' => $from_address, '!rid' => implode(', ', $from_user->roles))));
mailhandler_switch_user($from_user->uid);
}
$node->uid = $from_user->uid;
$node->name = $from_user->name;
return $node;
}
/**
* Find the parent of a new mailsubmitted comment. First use some heuristics
* and assumptions about Message-ID, References, and In-Reply-To headers.
* Then try to find a parent based on Subject line.
* If all else fails, start a new forum topic.
*/
function listhandler_find_parent($node, $header){
if($node->threading) {
$parent = db_fetch_object(db_query("SELECT * FROM {listhandler} WHERE msgid = '<%s>'", $node->threading));
// Debug
// watchdog('listhandler', "Parent seek <pre>'". $node->threading ."\np.nid:".$parent->nid."\n p.cid".$parent->cid."\nSubject: ".$header->subject."'</pre>");
if($parent) { // parent (!) seen by listhandler on this site.
return listhandler_parent_thread($node, $parent);
}
}
// still no parent. now find the most recent matching subject in the current forum
$title = trim(substr($node->title, (strrpos($node->title, ':') + 1)));
$parent = db_fetch_object(db_query("SELECT * FROM {listhandler} l INNER JOIN {node} n ON l.nid = n.nid WHERE n.title LIKE '%%%s' AND mid = %d ORDER by n.nid DESC", $title, $node->mid));
if ($parent) {
return listhandler_parent_thread($node, $parent);
}
else { // no parent found, new thread.
return listhandler_new_thread($node);
}
}
function listhandler_parent_thread($node, $parent) {
// Debug
// watchdog('listhandler', "Parent found <pre>'". $node->threading ."\np.nid:".$parent->nid."\np.cid".$parent->cid."\nSubject:".$node->title."'</pre>");
$node->pid = $parent->cid;
if($node->cid) unset($node->cid);
$node->nid = $parent->nid;
$node->type = 'comment';
return $node;
}
function listhandler_new_thread($node) {
$node->type = 'forum';
$node->nid = 0;
$node->taxonomy[] = $node->tid;
$node->pid = 0;
$node->comment = 2;
return $node;
}
#6
Maybe a patch file is a better way of going about this.. note: I changed listhandler_admin($form) from above to listhandler_admin() because it didn't belong there
#7
thanks jriddiough.
My employer has given me all next week to work on any FOSS project i like -- i'll knock this task off my list then, including merrging testing what you've provided here.
#8
This update:
-Includes the .info file
-Replaces deprecated $_POST['op']
-Completes a clean run through coder.module
#9
I took a glance at the patch, hopefully I will have some time to test it out and see what more needs to be done. Is there anything you're aware of that needs work, Dave? I think I'll check out that coder.module :)
#10
i've merged this with what i've got
(it's all in my darcs repo, http://darcs.coffee.geek.nz/contributions/modules)
attaching my .install
fixes bug with missing {} around tables and adds postgresql defs.
(should be merged to drupal 4.7 probably)
#11
I'm just waiting for this too.
#12
Subscribing.
#13
Subscribing
#14
darcs repo
Either a directory was forgotten or the site was rearranged a little.
http://darcs.coffee.geek.nz/drupal/contributions/modules/listhandler/
#15
Subscribing. We are waiting for this as well.
#16
Please help all the 5.x people by making this available..this is desperately needed for the Forums...it will improve Drupal's image a lot...pls do this someone.
#17
Bump, we need this realy !!
#18
I took some time to test out this set of patch, using the code available from the darcs repository. Main bug : the listhandler_admin_form function needs more work, to print description/help, or at least labels, for every field, which is necessary to put in the right values.
Here's a patch incorporating partches submitted earlier in this thread. It might be easier as a starting point, for people who can't use darcs, along with a few path corrections.
#19
It would be much appreciated if the owner could step up now and take a look to get this resolved. Listhandler is a very important module, and lot of people are waiting for this.
#20
Humm.. after testing, it seems like the last patch needs to be improved a lot. Mailhandler does it's thing, but the message is not imported/published in the forum.
@shiny : can you confirm that your code does import messages successfully?
#21
When attempting to install Listhandler on Drupal 5.1, using the Darcs version, I can't even get Listhandler to show up in the module page so I can install it. Is there something else I need to be doing to test this here?
#22
Here's a new version of the patch .. this is still incomplete, but it's working better. This patch applies against HEAD (14/02/2007) and still needs more work.
#23
I've reviewed the patch and applied it to CVS so can create more patches from there.
#24
I was able to patch against HEAD fine (couldn't wait for the CVS to update). However, there's a test done on $node->tid in listhandler_mailhandler(); which is not being set by Mailhandler or Listhandler. I post this here as I didn't know if this was a change in Mailhandler that has happened since its Drupal 5.x version, that might in turn require some changes in Listhandler.
#25
A small documentation patch that might help a lot of people test this out .. even though it's only about getting mailhandler to work.
#26
Tried the latest HEAD version -- IT WORKS -- except that the first forum post originated in a forum comes back to the forum and is not filtered out, as it used to be in 4.6.7.
Weird thing is, the comments/replies are NOT duplicated as well. It only seems to be the initial forum post.
#27
Oops, sorry... An update to my previous post: ANYTHING that originates in the forum IS duplicated when it returns via Mailhandler. The message is NOT filtered out.
#28
i hope this comes for Drupal 5 soon :)
anxiously await it
#29
Attached is a patch that corrects the display on the Listhandler configuration page.
1: Displays " | " in tabulated form
2: Saves updates to mailing list prefixes
3: Doesn't display mid values
4: Doesn't display duplicated "The configuration options have been saved" message
#30
Correction,
1: Displays "[mailing list] | [prefix]" in tabulated form
#31
This patch works nicely for me, thanks!
#32
subscribing, I'd love to implement this.
Joris
#33
subscribing... would like to see this working for 5.x (and soon 6.x)
#34
Any updates on this?
#35
For me it works perfectly in 5.3 when receiving from mailing list.
The thing i'm not sure of is that it sends correctly the mail from forum to mailing list.
From a quick look at the code it seems to me that:
$err = drupal_mail($mid, $address, $subject, $body, $edit["mail"]);should be changed in:
$err = drupal_mail('listhandler-mail', $address, $subject, $body, $edit["mail"],$mid);#36
hi
cld u kindly make an official release or a dev release for 5.1
thanks
#37
My patch should solve a bug related to the headers of mails sent from drupal when user posts a message in the forum. It also provide an attachment feature
1) Now listhandler should dected loop mails (mails generated from forum posts) and duplicates mails should be discarded.
2) Client mail programs should display correctly mails in hierarchy mode.
3) Forums attachments are converted into file links at the botom of mail body.
The attachment feature should be easy extended in order to convert them in real mime attachment (listhandler option to choose between the two methods?).
It applies against listhandler HEAD release and it contains all the previous patches of this thread.
It needs work, so any testing and feedback is well appreciated.
#38
I applied this patch to a new copy of HEAD from CVS, but it stumbled so I fixed it by hand. Here is a patch that works for me.
HTH
#39
Hi! Looks not bad.
The update to the mailhandler table does not happen, I had to run it from Mysql. Tomorrow I'll test it fully and report.
#40
I dnloaded from CVS and applied listhandler-head-to-5.patch on top of it.
Basically nothing happens, apart that email subjects do get stripped of what should be removed. When one mails in content from the mailing list it gets posted (as with mailhandler alone), yet if I make comments on the forum nothing gets to the list. Any suggestions? Maybe I should have first patched with mailfromforum.patch and THEN applied listhandler-head-to-5.patch? It's not very clear from the previous postings.
#41
One more problem.... when user answers to a topic from the mailing list, his answer does not become a comment. It becomes a new topic. Let alone breaking threads, this has heavier consequences, as it means that if you want to accept content from people who are subscribed to the list you either:
1) manually add accounts for each of them and give them a role for them to be enabled to create these "topics" (safe but heavy) OR
2) allow anonymous users to create the nodes on their own...
As I write this I am starting to understand that I haven't seen listhandler running at all. This is what MailHandler does. So when is ListHandler supposed to run, and how? based on cron?
#42
I think you patched it correctly, listhandler-head-to-5.patch should be applied to HEAD. It includes the attachments code from mailforum,patch
If the comment is not getting to the list then this needs to be fixed, obviously.
Which listmailer are you using?
Did any messages get through to watchdog? In particular for listhandler, starting with 'Error sending.....'
Have you got tid: xxx in the commands field in the mysql mailhandler table?
I have managed to get it to work with ezmlm but I had to add some code:
<?php
$mid['From'] = $edit['mail'];
$mid['Return-Path'] = $edit['mail'];
$midstr = "";
foreach ($mid as $k=>$v){
$midstr .= "$k: $v\r\n";
}
// using mail() because drupal_mail() strips headers
$ok = mail(trim($address), $subject, $body, $midstr);
?>
This goes in place of
<?php$ok = drupal_mail('listhandler-mail', $address, $subject, $body, $edit["mail"], $mid);
?>
on line 313
It's the 'Return-Path' that drupal_mail() appears to strip out if the request comes from a module, but I'm not sure about this. 'Return-Path' is used by ezmlm to establish where the mail comes from, not 'From' as you might expect.
If you have access to your mail logs you might see a refusal, then you will at least know that an email was sent but not accepted.
#43
You said:
Listhandler does not use cron, when a comment is submitted, the listhandler function 'listhandler_comment()' or 'listhandler_nodeapi()' should be run and they run 'listhandler_send_mail()'. They also put a message into watchdog.
I am using the following function to help in debugging:
<?phpfunction _listhandler_debugfunc($message) {
$log = "/path/to/your/files/test.log";
$hh = fopen($log, 'a');
fwrite($hh, $message);
fclose($hh);
}
?>
edit the path to your log and use it with lines like this:
<?php_listhandler_debugfunc("in listhandler_send_mail 'stuff you want to check' \n");
?>
Your other points are I think more mailhandler related and are working on my test rig so I can't really say why it's not working for you. Check that you have 'type: forum' in the commands field in mailhandler.
#44
So... basically we need a better set of docs and a nicer UI... as it turns out, the current docs for mailhandler ask you to write commands for mailhandler in terms of:
taxonomy: ['catname1', 'catename2',...]
This is very nice but infact it disables listhandler.
If you are to use this module you should write in the form:
tid: 22
type: forum
promote: 1
status: 1
comment: 2
It would be much handier if we really could write catnames as strings, although this indeed can lead to trouble, as there is no unique index on catnames... Maybe we can put more tids on different lines, can we? I haven't checked but I'd say yes, given the way REGEXP works in the select.
There seems to be something going wrong on function listhandler_mailhandler, though. I keep receiving the number '2', instead of the text in the comments sent from the mailing list... will have to trace this later on.
#45
I'm using mailman on debian etch, with mailhandler and mailman_manager. It seems to work fine so far, I only have to trace it and find out how the text of the mailing list comment gets to become a number... but okay, this evening I'll give it a couple of hours and find it out.
I really think it should getter a better explanation and we should update the mailhandler docs to explain the potential conflict in command style.
One more thought: I'm positive I can add more fixed taxonomy marks on automated input by putting it in sequence, I wish I could have a similar command for audience: (working with organic groups), so I can drive a dedicated forum into a given group. Yet this is quite an overlapping solution, as OGs do a lot af mailing themselves, which should definitely NOT happen when people are already subscribed to mailman... which means patching the OG module, too. To a fairly wide extent this kind of forums make OGs redundant, so probably I'll simply get rid of OGs altoghether... less stuff being executed is always good news :)
Hmmm... one thing we really may want is a follow-up to the comment/topic creation forms, that requests to people who submit new content but are NOT subscribed to the list whether they want to subscribe. It would save a lot of admin trouble and make better marketing for the lists. There is a lot of returning traffic from OGs because people get mailed about new stuff, if I am to get rid of OGs I'd rather have an equivalent traffic driver in place. I guess that would work.
Do you think it's possible to make it?
#46
Listhandler only uses tid: nn
it ignores the rest. It uses mysql REGEXP to pull the right record from mailhandler table. See line 235 for an example:
<?php$result2 = db_query("SELECT * FROM {mailhandler} m WHERE m.commands REGEXP '%s'", 'tid:[[:blank:]]*'. $taxoterm .'[[:>:]]');
?>
Any other entries in mailhandler.commands are used by mailhandler itself.
I have just tested listhandler with taxonomy: ['this','that'] in commands and it runs ok
#47
I have to say that i'm using now my patch on a production site and it does not work as it seemed to work on a fresh drupal installation.
Posts in the forum still gets duplicate.
Test site uses php5 whereas production site uses php4. Don't know if it could be the reasons, i don't think, but I'm going to investigate it.
Listhandler does not discarded duplicated msgid, even my patch seems to correctly generate them.
#48
Just a quick addition. Duplicate comments are correctly discarded, but not open thread posts.
Hutch, does the patch correctly work for you?
#49
Comments are working, I *think* new posts work too, I will test tonight and come back with results.
.........
The result is that I have it working both ways for both new topics and comments, and there are no duplicates, the Drupal log shows them being discarded. The code for this is in function listhandler_mailhandler(). It compares the Message-Id in the incoming mail to check if it has already passed through the system.
I suspect that the problems people are having are to do with the sending (listhandler) and receiving mail (mailhandler) The rest of the code is working.
It is possible that Message-Id is being over-written somewhere, which would have the effect of duplication.
#50
I think i've found the reason with the new topic problem.
I run the cron.php with the php command line and in that enviroment the $_SERVER["SERVER_NAME"] is empty.
This leads the ereg function in listhandler_mailhandler() to check for matching, which obviously returns false.
For who needs it and can't change how the cron.php has to run, the quick workaround should be to override $_SERVER["SERVER_NAME"] with the basename of your site (i.e:www.mysite.com)
#51
I would imagine running cron.php from the command line might have other ramifications for your site - why don't you just hit the webpage from cron instead of running it from the command line?
I think the command I use is just "wget example.com/cron.php" which will also return any errors cron pops up with such as missing db connection or whatever.
#52
Long story. However, yes, you're right. And now it runs with wget :-)
#53
As soon as i'll be free, i'll try to write a patch for handling mime attachment generated from forum messages. In the meanwhile, i'd suggest to change in the patched listhandler.module the line:
$attachments .= "\n".$href;with
$attachments .= "\n".htmlentities($href);in order to avoid broken links in mails generated from posts where attachment name contains spaces.
#54
This should be corrected in the install:
db_query("CREATE TABLE {listhandler} (lid int(10) unsigned not null auto_increment primary key,
msgid varchar(255) not null,
nid int(10) NOT NULL default '0',
cid int(10) NOT NULL default '0',
pid int(10) NOT NULL default '0',
uid int(10) NOT NULL default '0',
mid int(10) NOT NULL default '0',
tid int(10) NOT NULL default '0',
prefix varchar(255) NOT NULL DEFAULT'',
key (lid)
) /*!40100 DEFAULT CHARACTER SET utf8 */;");
break;
key (lid) is redundant, since you already have a primary on it.
#55
Dear All,
Gerhard has kindly passed the development of ListHandler to myself and Ser.
Currently there is a 5.x-1.x-dev branch (http://drupal.org/node/202136) that is tracking the work for Drupal 5 compatibility. Everything in that branch will be ported into Head as well.
So far I've committed my patch (http://drupal.org/node/126126#comment-524188) and all subsequent patches are under review.
I plan to release a beta version in the New Year - end of January at the latest.
Please continue to post patches to this task; please diff them against the DRUPAL-5 branch.
Thanks.
Phil L.
#56
Welcome Phil :-).
I'm glad that this module is still developped.
This patch solves the space bug with the attachment feature (upload module) from my previuos patch and adds support also for comment_upload module.
It does not support uploadpath module or any other module that changes file attachments path after listhandler processing (link in the email could be different from the real one).
If we want support more types of upload modules,i think that the attachments link implementation should be modular, with a plugin for every module.
It also adds, at the end of mails with attachments,a notice message with the link of the post/comment that generated it.
It contains all previous 5.x patches and it applies against DRUPAL-5.
#57
This is good news ;-)
I am using ezmlm and have found it necessary to alter things a bit to get it to work. I have done this by expanding on the 'include' feature, which optionally allows a file to be included. It uses the mailhandler commands by adding 'list_type: ezmlm'. Listhandler looks for this and if found prepends the value to the filename to be included (if it exists), eg ezmlm_mail.inc which contains the following:
<?php$mid['From'] = $edit['mail'];
$mid['Return-Path'] = $edit['mail'];
$midstr = "";
foreach ($mid as $k=>$v) {
$midstr .= "$k: $v\r\n";
}
// using mail() because drupal_mail() strips headers
$ok = mail(trim($address), $subject, $body, $midstr);
?>
I have done this because I cannot get drupal_mail() to work, the mail is sent but has had some headers stripped out and ezmlm refuses the message.
It does not of course interfere with existing usage and provides greater flexibility for dealing with different listmailer software.
The attached patch contains all of mailfromforum2.patch plus new code to read in the mailhandler commands and handle the include.
#58
Hutch, just to know, what's the MTA used by your drupal?
Looking at drupal_mail:
// Note: if you are having problems with sending mail, or mails look wrong
// when they are received you may have to modify the str_replace to suit
// your systems.
// - \r\n will work under dos and windows.
// - \n will work for linux, unix and BSDs.
// - \r will work for macs.
If you can find that this why drupal_mail does not work for you, i think that your patch should refer to a general solution for every Windows MTA servers instead of for a given listmailer.
#59
The server running Drupal is linux Debian sarge running a qmail MTA.
I will experiment some more with drupal_mail() and see if I can get it to work.
The ezmlm listmailer requires the Return-Path header to be set the same as From:
I don't think the problem is linefeeds, the reason for using \r\n to separate header lines is so that Windows based MTAs can handle it. Qmail handles it either way. The mail arrives but is rejected by ezmlm, I can see this in the qmail send logs.
If the .inc file does not exist and/or no list type is defined the default drupal_mail will be run instead so there shouldn't be any problem there.
Anyway I will experiment some more and try to figure out why drupal_mail is mangling the headers, the drupal_mail code suggests I should be able to get the Return-Path header set the way I want it.
#60
Hi!
now working perfectly. My lost comments came from having set a:
comment: 2
command in mailhandler, which was supposed to state that newly imported messages could be commented.
What I am using is:
Debian etch
Postfix with mysql based virtal domains
Mailman
Great module, btw.
#61
Added full support for mail attachements in mails generated by forum posts.
Listhandler configuration page has now the option to enabling the convertion of forum attachments in links or send them as mail attachments (the default).
Again, it contains all previuos patches, it applies on DRUPAL-5 tag and it's reviewed with code review module.
Hutch, it does not contain your patch because i can test it and i fear to loose something in appling it, so i leave you the task (consider to remove your debug function).
#62
Your patch applied perfectly, I will try to persuade drupal_mail() to work with ezmlm as soon as I get a moment.
I presume that this does work with Mailman.
#63
Yes, I've tested it with mailman.
I was in hurry in my previous post so, in addition to my english grammar errors, i attached an incomplete patch.
This is the corrected one with the missed attachment configuration option.
#64
Patch mailfromforum4.patch is working fine and it is now working with drupal_mail()
I had to add the following though:
$mid['Return-Path'] = $edit['mail'];Immediately above the drupal_mail line.
Without it drupal_mail sets the Return-Path to the default mail for the site and ezmlm rejects the post as that email address is not a member of the list.
So far so good ;-)
#65
Hey Guys,
I've been watching this thread for awhile, and I'm really impressed with the work being done to bring this module up to snuff for 5.x. Thank you!
I know its not quite finished yet, but I'm just curious how hard it will and whether there will an effort to upgrade this to 6.x once the efforts here are finished. I'm only asking because of the 6.0 release candidate announcement. If its a "not any time soon" or "it'll be hard" that's fine since i probably won't be upgrading to 6.x soon anyways, but just wanted to ask. Can also put this into a new issue/thread if necessary.
#66
Hey rcross,
Ser has taken on development for Drupal 6, however he's currently waiting on me to finish my work on a Drupal 5 version.
I expect a 1.0 stable release to take a few months as the beta release will need a bit of time for 'being out there and available' for adequate testing.
Please wait until 5.x-1.0 stable is available before creating a new task for a Drupal 6 version.
Thanks.
#67
Thanks for that hutch.
I've put that in CVS and an updated .tar.gz will be along shortly.
#68
Phil, Hutch patch is modelled for my patch, where $mid is an array, and i don't think it will be useful without mine.
You should probably write it as:
$mid .= "Return-Path: ". $edit['mail'] ."\n";but in this case it'll not work anyway because the wrong drupal_mail usage does not process mail headers.
#69
Just to confirm that, $mid needs to be an array to work with drupal_mail().
'References', 'In-Reply-To', 'Message-Id' and 'Return-Path' are the 4 keys.
$mid['Return-Path'] = $edit['mail'];could go just below
$mid['Message-Id'] = $msgid;so that it would get picked up by the _listhandler_mail_attachments() function. Not tested by me but I don't see any reason why it would not work.
#70
Thanks for pointing out my schoolboy error!
I'll work on getting samuelet's patch in next.
#71
Hi Samuelet,
Can you re-roll your patch as a unified diff format please?
The line numbers are now out of sync and your patch now puts code in the wrong places :( .
Thanks.
P.
#72
I have attached two patches, one for ezmlm and one for samuelet's attachments. They should be applied in that order. They were built from listhandler.module-5.x-1.x-dev of 2008-Jan-05
I have tested it and it has sent mail, not tested the attachments feature though.
#73
Just curious - would it make sense to write this module to utilize the functions from mailhandler.module? It seems like there is already quite a bit of duplication efforts going on. (Note - i haven't looked at the code, just following this thread with interest, so apologies for any ignorance)
#74
Hi rcross,
I'm not sure where the idea of duplication of effort comes from. Mailhandler handles the incoming email, Listhandler send outgoing email. Decoding of email and creation of email are different processes.
However, feel free to look at the code of both Listhandler and Mailhander and suggest any benefits or improvements you can see. Please create a new task (for a 2.0 version) if that is the case as this task is specifically for providing a Drupal 5.x-1.x version of Listhandler.
Thanks.
P.
#75
I agree that mailhandler is aimed at handling incoming email - but my understanding is that listhandler also does this by creating new nodes based on email from the mailing list (and filtering out those messages it has sent). I just remember seeing a post or two about making another module "compatible with mailhandler", so I figured it might be worth investigating. Apologies for possibly hijacking this thread - just didn't seem to make sense to post a new issue if it was pointless, and there isn't a forum for this module or anything. I'll try taking a look at the code when I get a chance (probably won't be soon though)
#76
Listhandler-5.x-1.0-beta1 is now out.
Please download and test.
P.
#77
I've installed Listhandler-5.x-1.0-beta1 and I have found one issue. When I hit reply in an email no matter what I put in, what shows in the post is a number 2 and that's it. Just 2.
I'm not a programmer so I really can't tell you anymore than this. Any ideas on this?