Netnews Synchronization

The netnews module enables synchronization of a forum with a netnews (NNTP) newsgroup. A netnews link is the configuration that defines the relation (connection) between a forum in the Drupal website and a newsgroup.

A netnews link contains:

Name
A symbolic name for the netnews link. This name should only contain alphanumeric chars and underscores.
Link Status
Specifies whether the Netnews Link is active. If inactive, no attempts will be made to synchronize the forum and the newsgroup
Newsgroup
The name of the newsgroup that the forum should be synchronized with. A typical example of a newsgroup name is comp.lang.php
Forums
Specifies the forum for which the Netnews Link is defined.
News Server
Name (DNS) or IP number of the news server where the specified newsgroup can be accessed
Port
The tcp port number under which the News Server is accessible. The default (119) is usually right.
Authentication method
Specifies the authentication method used for logging in to the news server. Supported values are none and NNTP authentication. When the latter is selected, the fields NNTP user and NNTP password must have a value.
NNTP user
User name to be used when NNTP authentication is selected.
NNTP password
Password to be used when NNTP authentication is selected.
Last messages synched
This field contains the message number of the last message that was read from the newsserver. This field is automatically updated after every netnews -> drupal synchronization. Normally, it is not necessary or useful to change this number. For a new netnews link, it should be set to 0 (the default).
Max. messages per synch
Specifies the maximum number of messages that will be imported from the news server in a single netnews -> drupal synchronization
Old messages
Maximum number of existing messages that will be imported from the news server the first time a netnews -> drupal synchronization is performed for this Netnews Link.
User for netnews messages
This drupal user will own all messages that are imported from the netnews server for this Netnews Link. This user should already be defined in drupal.

Runtime configuration

There are a number of options that can be used to change certain behaviour of the netnews module, and to get tracing output from the netnews module and the NNTP interface.

Synchronize often
Normally, netnews -> drupal synchronization only happens through the netnews_cron hook. When this option is set, netnews -> drupal synchronization occurs whenever a node or comment is added to the specified forum. Note that if the newsgroup has a high traffic volume, this could have a negative impact on the performance of the drupal site.
Synchronize on exit
Starts a synchronize for every active Netnews Link with this option enabled.
Netnews tracing
If set, a trace file will be generated for this Netnews Link. The file will be written to modules/netnews/.trace .
Trace NNTP server directly
Normally, if any tracing from the NNTP interface is to be included in the trace file, the trace output for the interface is included blockwise after an NNTP interface call has completed. When this option is selected, all tracing from the NNTP interface is written to the trace file directly. This increases the I/O load of the tracing significantly, and should only be used when necessary (e.g. when there is reason to believe that the NNTP interface crashes).
Tracer function
Name of callback function to be used for tracing by the NNTP interface if Trace NNTP server directly is selected. The default, "_netnews_trace", is usually the right value. Only change it if you are sure that you know what you are doing
NNTP Logging
A number of options exist to configure the amount of tracing from the NNTP interface:
Errors
If enabled, errors are written to the trace file.
Info
If enabled, informational messages are written to the trace file
Protocol
If enabled, the messages exchanged between the NNTP interface and the news server are written to the trace file. In case of NNTP authentication, the user name and password are masked.
'); case 'admin/netnews' : return t('The netnews module can be used to synchronize a forum on the website with a newsgroup on a netnews server. Each Netnews Link defines a relationship between a forum and a newsgroup.

Below is an overview of the defined Netnews Links on your system.'); case 'admin/modules#description' : return t('Synchronization of nodes and comments with a netnews group.'); } } /** * Implementation of netnews_perm() */ function netnews_perm() { return array ( 'administer netnews' ); } /** * Implementation of netnews_menu() */ function netnews_menu($may_cache) { $items = array (); if ($may_cache) { $items[] = array ( 'path' => "admin/netnews", 'title' => t("netnews" ), 'callback' => 'netnews_admin', 'access' => user_access('administer netnews'), 'weight' => 5); $items[] = array ( 'path' => "admin/netnews/list", 'title' => t("list" ), 'type' => MENU_DEFAULT_LOCAL_TASK, 'weight' => -10); $items[] = array ( 'path' => 'admin/netnews/add', 'title' => "add netnews link", 'callback' => '_netnews_admin_edit', 'access' => user_access('administer netnews' ), 'type' => MENU_LOCAL_TASK); $items[] = array ( 'path' => 'admin/netnews/edit', 'title' => "update netnews link", 'callback' => '_netnews_admin_edit', 'access' => user_access('administer netnews' ), 'type' => MENU_CALLBACK); $items[] = array ( 'path' => 'admin/netnews/delete', 'title' => "delete netnews link", 'callback' => '_netnews_admin_delete', 'access' => user_access('administer netnews' ), 'type' => MENU_CALLBACK); } // attachment support additions : else { // nocache if (arg(0) == 'node' && is_numeric(arg(1))) { $nid = arg(1); $node = node_load($nid); // print_r($node); if ($node->nid ) // && ($node->name=='netnews') // node->name ?? node->type='image' sometimes, but can I depend on 'name' { // When viewing forum nodes, add a context refresh button $items[] = array ( 'path' => 'node/' . $nid . '/refresh', 'title' => t('refresh'), 'callback' => '_netnews_admin_refresh', 'access' => node_access('update', $node), 'weight' => 1, 'type' => MENU_LOCAL_TASK ); $items[] = array ( 'path' => 'node/' . arg(1) . '/delete', 'title' => t('delete'), 'callback arguments' => array ('confirm' => '1'), 'access' => node_access('update', $node), 'weight' => 1, 'type' => MENU_LOCAL_TASK ); } } } return $items; } /** * Implementation of netnews_nodeapi(). */ function netnews_nodeapi(& $node, $op, $arg) { switch ($op) { case 'insert' : $drupalid = _netnews_drupalid($node->nid); if (!_netnews_insert_lock() && _netnews_drupal_message_is_new($node->nid) && _netnews_node_has_newsgroup($node->nid)) { if ($servers = _netnews_get_links($node->nid)) { foreach ($servers as $srv) { _netnews_store_drupal_message($srv['nngid'], $drupalid, $node->nid); } } } break; /* attachment support additions : */ case 'view' : // Force the image and upload apis to be called - this will initialize extra properties // As if this node were an image, and/or an upload if (function_exists('image_load')) { image_load($node); } if (function_exists('upload_nodeapi')) { // now do I really have to do this by hand? // Is it working upload_nodeapi($node, $op, $arg); } if ($node->images) { // drupal_set_message("Viewing netnews as image"); image_view($node, FALSE, TRUE); // This initializes node->body in its special way } else if ($node->files) { // drupal_set_message("Viewing netnews as upload"); upload_nodeapi(& $node, $op, $arg); // this adds the attachemnts to the end of the body } break; case 'delete' : // clean up our independant table db_query('DELETE FROM {netnews_messages} WHERE nid = %d', $node->nid); _netnews_trace("Deleted drupal-usenet link in database for node " . $node->nid); // Clean out any attachments we may have created if ($node->files) { upload_delete($node); } break; /* : /attachment support additions */ } } /** * implementation of netnews_admin() */ function netnews_admin() { print theme('page', _netnews_overview()); } /** * Implementation of netnews_comment() */ function netnews_comment($op, $comment) { switch ($op) { case 'insert' : if (!_netnews_insert_lock() && _netnews_drupal_message_is_new($comment['nid'], $comment['cid']) && _netnews_node_has_newsgroup($comment['nid'])) { if ($servers = _netnews_get_links($comment['nid'])) { foreach ($servers as $srv) { _netnews_store_drupal_message($srv['nngid'], _netnews_drupalid($comment['nid'], $comment['cid']), $comment['nid'], $comment['cid'], $comment['pid']); } } } break; } } /** * implementation of netnews_exit() */ function netnews_exit() { _netnews_synchronize_drupal(); $runtime = _netnews_current_runtime(); if ($runtime['synch_on_exit']) { _netnews_synchronize_link($runtime['nngid']); } /* * run "on exit" synch for every netnews link that has been configured * for it */ $oldname = $runtime['name']; $query = 'SELECT name FROM {netnews_links} WHERE active = %d AND synchmethod <> %d'; $result = db_query($query, NETNEWS_ACTIVE, NETNEWS_SYNCH_TO_NNTP); while ($name = db_fetch_object($result)) { if ($name != $oldname) { $runtime = _netnews_current_runtime($name); if ($runtime['synch_on_exit']) { _netnews_synchronize_link($runtime['nngid']); } } } /* * leave the runtime behind that you found on entry of the exit hook */ _netnews_current_runtime($oldname); } /** * implementation of netnews_cron() */ function netnews_cron() { _netnews_synchronize_all_links(); } /** * generates an overview of defined netnews links * * @return * overview page */ function _netnews_overview() { $sql = 'SELECT * FROM {netnews_links}'; $header = array ( array ( 'data' => "Name", 'field' => 'name', 'sort' => 'asc' ), array ( 'data' => "Group", 'field' => 'groupname' ), array ( 'data' => "Active", 'field' => 'active' ) ); $sql .= tablesort_sql($header); $result = pager_query($sql, 25); $destination = drupal_get_destination(); while ($data = db_fetch_object($result)) { $rows[] = array ( $data->name, $data->groupname, $data->active, l("edit",'admin/netnews/edit/' . $data->nngid, array (),$destination), l(t('delete'), 'admin/netnews/delete/' . $data->nngid, array (), $destination) ); } if ($pager = theme('pager', NULL, 25, 0)) { $rows[] = array ( array ( 'data' => $pager, 'colspan' => '4' ) ); } if (!$rows) { $rows[] = array ( array ( 'data' => "No newsgroup links available", 'colspan' => '4' ) ); } return theme('table', $header, $rows); } /** * deletes a netnews link configuration from the database and drupal variables * * @param $nngid id of the netnews link to be deleted */ function _netnews_admin_delete($nngid = 0) { $name = db_result(db_query('SELECT name FROM {netnews_links} WHERE nngid = %d', $nngid)); variable_del('netnews_' . $name); db_query('DELETE FROM {netnews_links} WHERE nngid = %d', $nngid); drupal_set_message(t('The netnews link has been deleted')); drupal_goto('admin/netnews'); } /** * Menu callback; handles pages for creating and editing netnews links */ function _netnews_admin_edit($nngid = 0) { if (isset ($edit['name'])) { _netnews_current_runtime($edit['name']); } if ($_POST['op'] == t('Create netnews link') || $_POST['op'] == t('Update netnews link')) { if (_netnews_admin_validate($_POST['edit'])) { $output = _netnews_admin_save($_POST['edit']); } else { $output = _netnews_admin_form($_POST['edit']); } } elseif ($nngid) { $netnews = _netnews_admin_load($nngid); drupal_set_title($netnews['name']); $output = _netnews_admin_form($netnews); } else { $output = _netnews_admin_form(); } print theme('page', $output); } /** * validates admin input for a netnews link config * * @param $edit array from edit form * * @return * boolean */ function _netnews_admin_validate($edit) { /* * name should only contain [a-zA-Z0-9_] */ if (!preg_match('/^\w+$/', $edit['name'])) { _netnews_trace('name rejected: ' . $edit['name']); form_set_error('name', t('The name should only contain a-z, A-Z, 0-9 and underscore (_) characters')); } /* * name should not already be present in netnews_links for a new link */ if ($edit['nngid'] == 0 && db_result(db_query("SELECT COUNT(name) FROM {netnews_links} WHERE name = '%s'", $edit['name']))) { form_set_error('name', t('The name %name is already in use.', array ( '%name' => theme('placeholder', $edit['name'])))); } /* * if auth method is set to come from the link config, user and password * must be specified (\TODO: is requiring a password too strict?) */ if ($edit['authmethod'] == NETNEWS_AUTH_LINK && (!isset ($edit['authuser']) || $edit['authuser'] == '')) { form_set_error('authuser', t('Link config authorization set but no NNTP User specified')); } if ($edit['authmethod'] == NETNEWS_AUTH_LINK && (!isset ($edit['authpasswd']) || $edit['authpasswd'] == '')) { form_set_error('authpasswd', t('Link config authorization set but no NNTP Password specified')); } /* * check the user method */ if ($edit['usermethod'] == NETNEWS_USER_DEFINED) { if (!isset ($edit['username']) || $edit['username'] == '' || db_result(db_query("SELECT COUNT(*) from {users} WHERE name = '%s'", $edit['username'])) == 0) { _netnews_trace('username rejected: ' . $edit['username']); form_set_error('username', t('The name of an existing user must be specified')); } } /* * check the taxonomy link, if applicable */ if ($edit['linktype'] == NETNEWS_LINK_CAT) { if (function_exists('forum_validate_term')) { forum_validate_term($edit['tid'][0]); } else { _netnews_validate_category_link_as_term($edit['tid'][0]); } } /* * Enable a new vocabulary for autotaxonomy, if applicable */ if ($edit['autotaxonomy']) { $vid = variable_get('netnews_autotaxonomy_vid', 10); if (!taxonomy_get_vocabulary($vid)) { $vocab = array ( 'name' => 'Keyphrases extracted from NetNews', 'description' => 'Terms here have been automatically inferred from message titles. Nome may not make sense, If they are non-informative, feel free to delete them', 'multiple' => TRUE, 'nodes' => array ('forum'), ); $vocab = taxonomy_save_vocabulary($vocab); variable_set('netnews_autotaxonomy_vid', $vocab['vid']); } } if (function_exists('forum_validate_term')) { forum_validate_term($edit['tid'][0]); } else { _netnews_validate_category_link_as_term($edit['tid'][0]); } if (form_get_errors()) { return FALSE; } if ($edit['fixtax']) { _netnews_fix_taxonomy($edit['nngid']); } return TRUE; } /** * stores a netnews link configuration to the database and drupal variables * * @param $edit validated input fields for the link * */ function _netnews_admin_save($edit) { $columns = '(name, groupname, servername, serverport, authmethod, authuser, authpasswd, linktype, tid, active, oldmsgs, usermethod, username, lastmsg, synchmethod, autotaxonomy, isbinary, maxmsgs)'; $values = "('%s', '%s', '%s', %d, '%s', '%s', '%s', %d, %d, %d, %d, %d, '%s', %d, %d, %d, %d, %d)"; /* $edit['name'], $edit['groupname'], $edit['servername'], $edit['serverport'], $edit['authmethod'], $edit['authuser'], $edit['authpasswd'], $edit['linktype'], $edit['tid'][0], $edit['active'], $edit['oldmsgs'], $edit['usermethod'], $edit['username'], $edit['lastmsg'], $edit['maxmsgs'] */ $sets = "name = '%s', groupname = '%s', servername = '%s', serverport = %d, authmethod = '%s', authuser = '%s', authpasswd = '%s', linktype = %d, tid = %d, active = %d, oldmsgs = %d, usermethod = %d, username = '%s', lastmsg = %d, synchmethod = %d, autotaxonomy = %d, isbinary = %d, maxmsgs = %d "; if ($edit['nngid'] != 0) { db_query('UPDATE {netnews_links} SET ' . $sets . ' WHERE nngid = %d', $edit['name'], $edit['groupname'], $edit['servername'], $edit['serverport'], $edit['authmethod'], $edit['authuser'], $edit['authpasswd'], $edit['linktype'], $edit['tid'][0], $edit['active'], $edit['oldmsgs'], $edit['usermethod'], $edit['username'], $edit['lastmsg'], $edit['synchmethod'], $edit['autotaxonomy'], $edit['maxmsgs'], $edit['isbinary'], $edit['nngid']); } else { db_query('INSERT INTO {netnews_links} ' . $columns . ' VALUES ' . $values, $edit['name'], $edit['groupname'], $edit['servername'], $edit['serverport'], $edit['authmethod'], $edit['authuser'], $edit['authpasswd'], $edit['linktype'], $edit['tid'][0], $edit['active'], $edit['oldmsgs'], $edit['usermethod'], $edit['username'], $edit['lastmsg'], $edit['synchmethod'], $edit['autotaxonomy'], $edit['maxmsgs'], $edit['isbinary']); } variable_set('netnews_' . $edit['name'], _netnews_runtime_info($edit)); drupal_set_message(t('The netnews link has been saved.')); drupal_goto('admin/netnews'); } /** * checks if a term is a valid forum container term * sets a form error if it is not * * @param $tid id of the term * NOTE: this code duplicates part of forum_validate in forum.module * TODO: remove when patch for forum.module is done */ function _netnews_validate_category_link_as_term($tid) { // Extract the node's proper topic ID. $vocabulary = variable_get('forum_nav_vocabulary', ''); $containers = variable_get('forum_containers', array ()); if (db_result(db_query('SELECT COUNT(*) FROM {term_data} WHERE tid = %d AND vid = %d', $tid, $vocabulary))) { if (in_array($tid, $containers)) { $term = taxonomy_get_term($tid); form_set_error('tid', t('The item %forum is only a container for forums. Please select one of the forums below it.', array ( '%forum' => theme('placeholder', $term->name )))); } } } /** * retrieves a netnews link configuration from the database * * @param $nngid id of the link config */ function _netnews_admin_load($nngid) { if ($srv = db_fetch_array(db_query('SELECT * FROM {netnews_links} WHERE nngid = %d', $nngid))) { $srv['runtime'] = _netnews_current_runtime($srv['name']); } return $srv; } /** * generates a form for editing a netnews link configuration * * @param $edit prefilled fields for an existing link * * @return * the form */ function _netnews_admin_form($edit = '') { // TODO: consider adding a "server trusted" option. if set, mails // posted through NNTP that match a user in drupal (name, email) // will be owned by that user $form['name'] = array ( '#type' => 'textfield', '#title' => "Name", '#default_value' => $edit['name'], '#size' => 50, '#maxlength' => 32, '#description' => "Specify a symbolic name for the netnews link (alphanumeric chars and underscores only)", ); $activeoptions = array ( NETNEWS_INACTIVE => 'inactive', NETNEWS_ACTIVE => 'active' ); $form['active'] = array ( '#type' => 'select', '#title' => "Link status", '#default_value' => ($edit == '') ? NETNEWS_ACTIVE : $edit['active'], '#options' => $activeoptions, '#description' => "Select if link is active or not", '#extra' => 0, '#multiple' => FALSE, '#required' => TRUE, ); $form['groupname'] = array ( '#type' => 'textfield', '#title' => "Newsgroup", '#default_value' => $edit['groupname'], '#size' => 50, '#maxlength' => 255, '#description' => "Specify the name of the newsgroup (e.g. comp.lang.php)", ); $form['isbinary'] = array ( '#type' => 'checkbox', '#title' => "Binary Newsgroup", '#return_value' => TRUE, '#default_value' => $edit['isbinary'], '#description' => "Is this newsgroup expected to have binary attachments? Digital processing is more accurate if this is enabled, but the text may come out looking a little unformatted.", ); if (function_exists('taxonomy_form')) { $forumselector = taxonomy_form( variable_get('forum_nav_vocabulary', ''), $edit['tid'], t('Select forum. (Forum module must be enabled)
Please note that there is little sense in changing the forum for a netnews link that is already in use.'), 'tid' ); // TODO: this is one ugly hack to get rid of an 's' //$forumselector = preg_replace('/>' . t('Forums:') . '' . //t('Forum:') . '<', $forumselector); $form['forumselector'] = $forumselector; } $form['servername'] = array ( '#type' => 'textfield', '#title' => "News Server", '#default_value' => ($edit == '' ) ? 'localhost' : $edit['servername'], '#size' => 50, '#maxlength' => 255, '#description' => t('Specify the domain name of the News Server'),); $form['serverport'] = array ( '#type' => 'textfield', '#title' => "Port", '#default_value' => ($edit == '' ) ? 119 : $edit['serverport'], '#size' => 20, '#maxlength' => 8, '#description' => t('Specify the port of the News Server (usually 119)'),); $authoptions = array ( NETNEWS_AUTH_NONE => 'none', NETNEWS_AUTH_LINK => 'NNTP authentication', /*NETNEWS_AUTH_USER => 'user'*/ ); $form['authmethod'] = array ( '#type' => 'select', '#title' => "Authentication Method", '#default_value' => intval($edit['authmethod'] ), '#options' => $authoptions, '#description' => t('Select authentication method'), '#extra' => 0, '#multiple' => FALSE, '#required' => TRUE,); $form['authuser'] = array ( '#type' => 'textfield', '#title' => "NNTP user", '#default_value' => $edit['authuser'], '#size' => 32, '#maxlength' => 64, '#description' => "Specify the NNTP authentication user", ); $form['authpasswd'] = array ( '#type' => 'password', '#title' => "NNTP password", '#default_value' => $edit['authpasswd'], '#size' => 32, '#maxlength' => 64, '#description' => "Specify the NNTP authentication password", ); $synchoptions = array ( NETNEWS_SYNCH_TWOWAY => 'two-way read/write from news server', NETNEWS_SYNCH_TO_DRUPAL => 'read-only from news server', NETNEWS_SYNCH_TO_NNTP => 'write-only to news server' ); $form['synchmethod'] = array ( '#type' => 'select', '#title' => "News Server Sync Direction", '#default_value' => intval($edit['synchmethod'] ), '#options' => $synchoptions, '#description' => t('Select whether to read, write or both when taking to the news server'),); $form['lastmsg'] = array ( '#type' => 'textfield', '#title' => "Last message synched", '#default_value' => $edit['lastmsg'], '#size' => 10, '#maxlength' => 10, '#description' => "Number identifying the last message that was synched from this newsgroup to the website. Generally, you DO NOT NEED to change this number", ); $form['maxmsgs'] = array ( '#type' => 'textfield', '#title' => "Max. messages per synch", '#default_value' => $edit['maxmsgs'], '#size' => 10, '#maxlength' => 10, '#description' => "Specify the maximum number of messages that should be read/imported per synchronize action", ); $form['oldmsgs'] = array ( '#type' => 'textfield', '#title' => "Old messages", '#default_value' => $edit['oldmsgs'], '#size' => 10, '#maxlength' => 10, '#description' => "Maximum number of old messages to be imported from the news server when it is synchronized for the first time", ); /* * TODO: implement the other options */ /* $linkoptions = array(NETNEWS_LINK_CAT => 'forum', NETNEWS_LINK_TYPE => 'nodetype', NETNEWS_LINK_TREE => 'category tree'); $form .= form_select(t('Link type'), 'linktype', intval($edit['linktype']), $linkoptions, t('Select link type'), 0, FALSE, TRUE); */ /* // TODO: set required // TODO: selector for existing/enables node types? $form .= form_textfield(t('Node type'), 'type', $edit['type'], 32, 16, t('Specify the node type for this link')); */ // TODO: implement the NETNEWS_USER_CREATE option /* $useroptions = array(NETNEWS_USER_CREATE => 'create new users', NETNEWS_USER_DEFINED => 'use a predefined user'); $form .= form_select(t('User method for netnews messages'), 'usermethod', $edit['usermethod'], $useroptions, t('Select which method to use for the user that should own the messages that originate from the news server')); */ $form['username'] = array ( '#type' => 'textfield', '#title' => "User for netnews messages", '#default_value' => $edit['username'], '#size' => 50, '#maxlength' => 60, '#description' => "This user will own all messages that originate from the news server", ); /* * runtime settings */ $runtime = variable_get('netnews_' . $edit['name'], _netnews_runtime_info()); $form['synch_often'] = array ( '#type' => 'checkbox', '#title' => "Synchronize often", '#return_value' => TRUE, '#default_value' => $runtime['synch_often'], '#description' => "Enables netnews -> drupal synchronization whenever a drupal -> netnews synchronization is performed", ); $form['synch_on_exit'] = array ( '#type' => 'checkbox', '#title' => "Synchronize on exit", '#return_value' => TRUE, '#default_value' => $runtime['synch_on_exit'], '#description' => "Enables netnews -> drupal synchronization whenever a page is requested, but limited to one time per minute.", ); $form['tracing'] = array ( '#type' => 'checkbox', '#title' => "Netnews tracing", '#return_value' => TRUE, '#default_value' => $runtime['tracing'], '#description' => "Enables tracing of the netnews module to modules/netnews/.trace", ); $form['tracenntp'] = array ( '#type' => 'checkbox', '#title' => "Trace nntpserver directly", '#return_value' => TRUE, '#default_value' => $runtime['tracenntp'], '#description' => "Enables direct tracing of the nntpserver object module to modules/netnews/.trace", ); $form['tracer'] = array ( '#type' => 'textfield', '#title' => "Tracer function", '#default_value' => $runtime['tracer'], '#size' => 50, '#maxlength' => 50, '#description' => "Specify the name of the tracer function", ); $on = array (); foreach (array ( 'logerrors', 'loginfo', 'logprotocol' ) as $label) { if ($runtime[$label]) { $on[] = $label; } } $form['logging'] = array ( '#type' => 'checkboxes', '#title' => "NNTP logging", '#default_value' => $on, '#options' => array ( 'logerrors' => 'Errors', 'loginfo' => 'Info', 'logprotocol' => 'Protocol' ), '#description' => "Select which messages from the NNTP interface should be included in the tracing", ); $synchoptions = array ( NETNEWS_AUTOTAXONOMY_OFF => 'No automatic keyphrases are detected', NETNEWS_AUTOTAXONOMY_TERSE => 'Some keyphrases are extracted from the message title', NETNEWS_AUTOTAXONOMY_VERBOSE => 'Advanced grouping keyphrases are automatically applied' ); $form['autotaxonomy'] = array ( '#type' => 'select', '#title' => "Automatis Keyphrase Detection", '#default_value' => intval($edit['autotaxonomy']), '#options' => $synchoptions, '#description' => t('When accessing a busy binary newsgroup, many postings have similar titles. Enabling automatic keyphrases will detect patterns in postings, and try to dynamically create logical groupings (using taxonomy). This method can rapidly create a lot of vague terms, depending on the group, but will also help ensure all similar posts can be grouped correctly'),); if ($edit['nngid']) { $form['fixtax'] = array ( '#type' => 'checkbox', '#title' => "Fix taxonomy", '#return_value' => TRUE, '#default_value' => FALSE, '#description' => "Enables fixing of taxonomy when this form is submitted", ); $form['nngid'] = array ( '#type' => 'hidden', '#value' => $edit['nngid'], ); $form['button'] = array ( '#type' => 'submit', '#value' => "Update netnews link", ); } else { $form['button'] = array ( '#type' => 'submit', '#value' => "Create netnews link", ); } return drupal_get_form('netnets_profile', $form); } /** * sends new drupal messages inserted by the current user to netnews servers * * @param $status SELECT value for netnews_messages.status * */ function _netnews_synchronize_drupal($status = NETNEWS_NEWMSG) { global $user; $query = 'SELECT * FROM {netnews_messages} WHERE uid = %d AND status = %d'; $result = db_query($query, $user->uid, $status); while ($msg = db_fetch_object($result)) { $srv = _netnews_admin_load($msg->nngid); if ($srv['synchmethod'] & NETNEWS_SYNCH_TO_NNTP) { if ($msg->cid == 0) { // a node $node = node_load(array ( 'nid' => $msg->nid )); _netnews_trace('about to post node ' . $msg->nid, $node); _netnews_post_node($srv, $node); } else { // a comment $comment = _netnews_load_comment($msg->cid); _netnews_trace('about to post comment ' . $msg->cid, $comment); _netnews_post_comment($srv, $comment); } } else { _netnews_trace('Live posting to nntp server disabled for this group. Not propogating node ' . $msg->nid); _netnews_set_message_status($msg->nid, NETNEWS_DONTSYNCH); } /* * if synch_often is set in the runtime for the current server config, * run a netnews -> drupal synch as well */ if (($srv['runtime']['synch_often']) && ($srv['synchmethod'] & NETNEWS_SYNCH_TO_DRUPAL)) { _netnews_trace("synch_often triggered for " . $srv['name']); _netnews_synchronize_on_lastmsg($srv); } } } /** * loads a comment from the database * * @param $cid comment id * * @return * comment as an array * * TODO: shouldn't there be a comment_load() in comment.module like * there is a node_load() in node.module? */ function _netnews_load_comment($cid) { return db_fetch_array(db_query('SELECT * FROM {comments} WHERE cid = %d', $cid)); } /** * finds all active netnews links for which new netnews messages should be * imported to drupal, and call the _netnews_synchronize() function for * each link */ function _netnews_synchronize_all_links() { _netnews_trace("_netnews_synchronize_all_links()"); $query = 'SELECT nngid FROM {netnews_links} WHERE active = %d AND synchmethod <> %d'; $result = db_query($query, NETNEWS_ACTIVE, NETNEWS_SYNCH_TO_NNTP); while ($res = db_fetch_object($result)) { _netnews_synchronize_link($res->nngid); } // let this happen silently, remove messages for the user drupal_get_messages(); } function _netnews_synchronize_link($nngid) { _netnews_trace("_netnews_synchronize_link($nngid)"); if ($srv = _netnews_admin_load($nngid)) { _netnews_synchronize_on_lastmsg($srv); } } /** * performs netnews -> drupal synchronization for a given netnews link * * @param $srvinfo netnews link configuration * * TODO: this function went out of use, to be removed? * for now I keep the code, if it turns out this module should be able * to work on a NEWNEWS approach as well */ function _netnews_synchronize_on_timestamp_NO_LONGER_WORKS($srvinfo) { global $base_url; _netnews_trace("synchronizing netnews for " . $srvinfo['name']); $server = _netnews_create_nntpserver($srvinfo); $timestamp = $srvinfo['lastsynch']; if ($timestamp == 0) { $timestamp = $srvinfo['synchfrom']; } if ($newnews = $server->getNewNews($srvinfo['groupname'], $timestamp)) { if ($headers = $server->getHeaders($newnews)) { $msgs = 0; foreach ($headers as $idx => $hdr) { if (((!array_key_exists(NETNEWS_DRUPALID, $hdr)) or _netnews_drupalid_is_foreign($hdr[NETNEWS_DRUPALID])) && _netnews_nntp_message_is_new($hdr['Message-ID'])) { _netnews_insert_lock('set', TRUE); _netnews_import_message($server, $srvinfo, $hdr); _netnews_insert_lock('set', FALSE); if ($msgs++ > $srvinfo['maxmsgs']) { break; // don't go on forever } } } } /* * only update lastsynch if we managed to process all new messages */ if ($msgs <= $srvinfo['maxmsgs']) { _netnews_update_lastsynch($srvinfo['nngid']); } } if (count($server->getLog())) { _netnews_trace('logging from NNTP session: ', $server->getLog()); } $server->done(); } /** * performs netnews -> drupal synchronization for a given netnews link * * @param $srvinfo netnews link configuration */ function _netnews_synchronize_on_lastmsg($srvinfo) { global $base_url; _netnews_trace("synchronizing netnews for " . $srvinfo['name'] . " on lastmsg"); $server = _netnews_create_nntpserver($srvinfo); if ($srvinfo['lastmsg'] == 0) { _netnews_trace("initializing last message number"); $last = $server->getLastMsgNum($srvinfo['groupname']); $lastmsg = $last - $srvinfo['oldmsgs']; if ($lastmsg <= 0) { $lastmsg = $srvinfo['lastmsg'] = 1; } } else { $lastmsg = $srvinfo['lastmsg']; } _netnews_trace("starting with message number $lastmsg"); $maxmsgs = $srvinfo['maxmsgs']; if ($newnews = $server->getNewNewsXover($srvinfo['groupname'], $lastmsg, $maxmsgs)) { if ($headers = $server->getHeaders($newnews)) { foreach ($headers as $idx => $hdr) { if (((!array_key_exists(NETNEWS_DRUPALID, $hdr)) or _netnews_drupalid_is_foreign($hdr[NETNEWS_DRUPALID])) && _netnews_nntp_message_is_new($hdr['Message-ID'])) { _netnews_insert_lock('set', TRUE); if (!_netnews_import_message($server, $srvinfo, $hdr)) { _netnews_trace("Message " . $hdr['Message-ID'] . " was not imported", array ( 'server' => $server, 'server config' => $srvinfo, 'header' => $hdr )); // TODO: add a watchdog error here } _netnews_insert_lock('set', FALSE); } $lastmsg = $hdr['X-local-msgnum']; } } if (intval($lastmsg) == $srvinfo['lastmsg']) { /* * got new headers, but did not add any messages * advance $lastmsg by the number of records returned by getNewNewsXover * TODO: consider if this construction risks to miss messages, and * how to prevent that (what if a message is missed because of an * occasional failure like a broken connection?) */ $lastmsg += count($newnews); } // update the last message retrieved _netnews_update_lastmsg($srvinfo['nngid'], $lastmsg); } if (count($server->getLog())) { _netnews_trace('logging from NNTP session: ', $server->getLog()); } $server->done(); } /** * updates the time when a netnews link was last synchronized * * @param $nngid id of the netnews link config */ function _netnews_update_lastsynch($nngid) { /* * update lastsynch time for netnews links. * perhaps paranoid, but do prevent that lastsynch goes back in time, * just in case we're in a wild loop */ $lastsynch = db_result(db_query("SELECT lastsynch FROM {netnews_links} WHERE nngid = %d", $nngid)); $now = time(); $lastsynch = $now - (($lastsynch >= $now) ? 0 : 1); db_query("UPDATE {netnews_links} SET lastsynch = %d WHERE nngid = %d", $lastsynch, $nngid); } /** * sets netnews_links.lastmsg * * @param $nngid id of netnews link config * @param $lastmsg new value for netnews_links.lastmsg */ function _netnews_update_lastmsg($nngid, $lastmsg) { db_query("UPDATE {netnews_links} SET lastmsg = %d WHERE nngid = %d", $lastmsg, $nngid); } /** * checks if a drupalid originates from another drupal site * * @param $drupalid drupal identifier for a message * * @return * boolean */ function _netnews_drupalid_is_foreign($drupalid) { global $base_url; if (strncasecmp($drupalid, $base_url, strlen($base_url)) == 0) { return TRUE; } return FALSE; } /** * checks is a netnews message is new to drupal * * @param $msgid NNTP message-ID * * @return * boolean */ function _netnews_nntp_message_is_new($msgid) { return db_result(db_query("SELECT COUNT(*) FROM {netnews_messages} WHERE msgid = '%s'", $msgid)) == 0; } function _netnews_nntp_message_has_thread($references) { if (!isset ($references) || $references = '') { /* * a message without an empty references header field is the start * of a thread and thus always "has" one */ return TRUE; } else { /* * a message with references has a thread if its last reference is in * the database already */ $refs = explode(' ', $references); $lastref = $refs[count($refs) - 1]; return db_result(db_query("SELECT COUNT(*) FROM {netnews_messages} WHERE msgid = '%s' and status = %d", $lastref, NETNEWS_SYNCHED)); } } /** * checks if a drupal message is new (i.e. not in netnews_messages yet) * * @param $nid node id * @param $cid comment id (default 0 for a node) * * @return * boolean if message is new (TRUE) or not (FALSE) */ function _netnews_drupal_message_is_new($nid, $cid = 0) { return db_result(db_query("SELECT COUNT(*) FROM {netnews_messages} WHERE nid = %d and cid = %d", $nid, $cid)) == 0; } /** * imports a netnews message into drupal * * @param $server nntpserver object * @param $srvinfo netnews link config * @param $hdr netnews message header * * @return * TRUE is import succeeds, otherwise FALSE */ function _netnews_import_message(& $server, & $srvinfo, & $hdr) { $msgid = $hdr['Message-ID']; $importstatus = FALSE; if ($msg = $server->getBody($msgid)) { _netnews_trace('retrieved body - ' . count($msg) . ' lines'); $userstr = $hdr['From']; $uid = _netnews_check_nntp_user($userstr, $srvinfo); _netnews_switch_user($uid); if (!$srvinfo['isbinary']) { $msg = $userstr . "\n
\n" . implode("
\n", $msg); } else { $msg = implode("\n", $msg); } _netnews_store_nntp_message($srvinfo[nngid], $msgid); $cid = 0; $pid = 0; if (!array_key_exists('References', $hdr) || $hdr['References'] == '') { /* * new topic, insert as a node */ if (($nid = _netnews_add_node($hdr, $msg, $srvinfo, $uid)) === FALSE) { _netnews_trace("could not store node", form_get_errors()); // TODO: add watchdog error log? } } else { /* * followup, insert as a comment */ list ($nid, $cid, $pid) = _netnews_add_comment($hdr, $msg, $srvinfo, $uid); } // Server may have found some attachments, which it's held on to for us if ($server->attachments) { _netnews_trace('adding attachments '.join(', ',array_keys($server->attachments)).' to node '.$nid ); $node = node_load(array ( 'nid' => $nid )); _netnews_add_attachments(&$node, $server->attachments); } else { _netnews_trace('no attachments were retained by the server'); } if ($nid || ($nid && $cid)) { _netnews_set_nntp_message_complete($msgid, $nid, $cid, $pid); $importstatus = TRUE; } else { _netnews_set_nntp_message_failed($msgid); } _netnews_switch_user(); } return $importstatus; } /** * adds a netnews message as a node to drupal * * @param $hdr netnews messages header * @param $text message body * @param srvinfo netnews link config * @uid id of the user that will own the message in drupal * * @return * from node_submit() in node.module: * If the node is successfully saved the node ID is returned. If the node * is not saved, 0 is returned. */ function _netnews_add_node($hdr, $text, $srvinfo, $uid = 0) { $node = new StdClass(); $node->type = $srvinfo['type']; $node->title = $hdr['Subject']; $node->uid = $uid; $node->tid = $srvinfo['tid']; $node->taxonomy = array ( $srvinfo['tid'] ); $node->comment = 2; if($i = strstr($text,'ybegin')){ $text = substr($text,0,$i+200) . "\n... binary content followed " ; } $node->body = $text; // .dan. attachment support // Try and sync the saved timestamp with the original post $node->changed = strtotime($hdr['Date']); $node->messageid = $hdr['Message-ID']; require_once (drupal_get_path('module', 'netnews') . '/attachment_support.inc'); if ($nid = _netnews_nntp_message_already_exists($hdr['Message-ID'])) { //Run in replace mode. Allow existing nodes to be overwritten to avoid duplicates (mostly useful just for debug refreshes) drupal_set_message("Running in replace mode. Refreshing node $nid (" . $node->title . ') '); $hdr[NETNEWS_DRUPALID] = $nid; } if ($hdr[NETNEWS_DRUPALID]) { $node->nid = $hdr[NETNEWS_DRUPALID]; } if (!node_access('create', $node)) { _netnews_trace('_netnews_add_node did not have access to create a new node as user ' . $GLOBALS['user']->name . ' . You will need to give this user edit/create rights on nodes in the access.'); watchdog('content', t('Node: unauthorized node submitted (%title).', array ( '%title' => theme('placeholder', $node->title ))), WATCHDOG_WARNING); return FALSE; } // Keep full news headers around, hidden, just for reference if (variable_get("netnews_embed_headers", TRUE)) { foreach ($hdr as $h => $v) { $preamble .= sprintf("[%-16s]: %s\n", $h, $v); } $node->teaser = node_teaser($node->body); $node->body = "\n\n\n" . $node->body; } // /dan. attachment support $node = node_submit($node); node_save($node); _netnews_trace("saved node is now: ", print_r($node,1)); $nid = $node->nid; if ($nid === FALSE) { _netnews_trace("node_submit failed: ", $node); } elseif ($errors = form_get_errors()) { _netnews_trace("node_submit generated form errors:", $errors); } _netnews_trace("node submitted, nid = $nid"); // Binary groups are most manageable when sorted. // Infer some keywords from this post if ($srvinfo['autotaxonomy']) { // Which vocab to fill up with our keyphrase-on-the-fly $vid = variable_get('netnews_autotaxonomy_vid', 10); $keyphrases = netnews_infer_keywords(& $node, $hdr); $term_ids = array (); foreach ($keyphrases as $term) { if (count($term_defs = taxonomy_get_term_by_name($term)) == 0) { $term_def = array ( 'vid' => $vid, 'name' => $term, 'description' => 'auto-generated phrase' ); // Insert it on-the-fly $term_def = taxonomy_save_term(& $term_def); // pass by ref should tell us what the new id is $tid = $term_def['tid']; // but as an array, not an obj... drupal_set_message("Defined keyphrase '" . $term_def['name'] . "' in vocab $vid. "); } else { //drupal_set_message("Recognised keyphrase as '".print_r($term_defs,1)."' in vocab $vid. "); $term_def = $term_defs[0]; $tid = $term_def->tid; // TODO - ensure we are in the right vocabulary } $node->taxonomy[] = $tid; // drupal_set_message("Associating this node with keyphrase '".$term."' in vocab $vid. "); } taxonomy_node_save($nid, $node->taxonomy); } return $nid; } /** * adds a netnews message as a comment to drupal * * @param $hdr netnews messages header * @param $text message body * @param srvinfo netnews link config * @uid id of the user that will own the message in drupal * * @return * array of nid, cid and pid of the added comment */ function _netnews_add_comment($hdr, $text, $srvinfo, $uid = 0) { $cid = 0; $pid = 0; // first, get a $nid and possibly a $pid $refs = explode(' ', $hdr['References']); /* * the record from netnews_messages with the last message-ID from the * References field contains the nid, and the pid if there is one */ $lastref = $refs[count($refs) - 1]; /* * retrieve nid and cid from last ref, they will be nid and pid for the new * message */ $refids = db_fetch_object(db_query("SELECT nid, cid FROM {netnews_messages} WHERE msgid = '%s'", $lastref)); $comment = array (); $comment['nid'] = $refids->nid; $comment['pid'] = $refids->cid; $comment['uid'] = $uid; $comment['subject'] = $hdr['Subject']; //TODO: strip if equal Re: node subject $comment['comment'] = $text; // only try to post comments if comment.module is available if (function_exists('comment_post_new')) { comment_post_new($comment); $cid = $comment['cid']; } elseif (function_exists('comment_save')) { // check the original $cid = comment_save($comment); } return array ( $refids->nid, $cid, $pid ); } /** * checks and/or adds the user that will own a netnews message * * @param $userstr on input, the 'From' field from an NNTP header * on output, string that should be inserted at the * top of the netnews message * @param $srvinfo netnews link config * * @return * id of user in drupal (uid) */ function _netnews_check_nntp_user(& $userstr, $srvinfo) { $uid = 0; if ($srvinfo['usermethod'] == NETNEWS_USER_DEFINED) { $uid = db_result(db_query("SELECT uid FROM {users} WHERE name = '%s'", $srvinfo['username'])); $userstr = t("Message from %u:", array ( "%u" => $userstr )); } if ($srvinfo['usermethod'] == NETNEWS_USER_CREATE) { // TODO, obviously // see listhandler_create_author() in litshandler.module /* * split the NNTP user field, check if you already have such a user, if * not, add and return uid * can we add a marker to the user data so that we know it's a user * inserted by netnews.module? */ } return $uid; } /** * determines if a node belongs to a type/term that is associated with * netnews synchronization * * @param $nid id of the node * * @return * TRUE if $nid has a netnews link, or FALSE */ function _netnews_node_has_newsgroup($nid) { if ($tid = db_result(db_query('SELECT tid FROM {forum} WHERE nid = %d', $nid))) { return db_result(db_query('SELECT COUNT(*) FROM {netnews_links} WHERE linktype = %d AND tid = %d AND active = %d AND synchmethod <> %d', NETNEWS_LINK_CAT, $tid, NETNEWS_ACTIVE, NETNEWS_SYNCH_TO_DRUPAL)); } return FALSE; } /** * returns an array of server info objects for $nid * * @param $nid node id * * @return * array of netnews link objects, or NULL */ function _netnews_get_links($nid) { if ($tid = db_result(db_query('SELECT tid FROM {forum} WHERE nid = %d', $nid))) { $result = db_query('SELECT nngid FROM {netnews_links} WHERE linktype = %d AND tid = %d AND active = %d AND synchmethod <> %d', NETNEWS_LINK_CAT, $tid, NETNEWS_ACTIVE, NETNEWS_SYNCH_TO_DRUPAL); $servers = array (); while ($res = db_fetch_object($result)) { $srv = _netnews_admin_load($res->nngid); _netnews_trace("found server for node $nid: " . $srv['name']); $servers[] = $srv; } return $servers; } return NULL; } /** * handles the netnews posting of a node * * @param $srv news server configuation * @param $node node to be posted */ function _netnews_post_node($srv, $node) { $header = array (); $header['Subject'] = $node->title; $header['From'] = _netnews_get_userinfo($node->name, $node->uid, $srv); _netnews_post_message($srv, $header, $node->body, $node->nid); } /** * handles the netnews posting of a comment * * @param $srv news server configuration * @param $comment comment to be posted */ function _netnews_post_comment($srv, $comment) { $header = array (); $subjectpreg = '/^' . $comment['subject'] . '/'; if (preg_match($subjectpreg, $comment['comment'])) { /* * subject was extracted from comment, replace it by the subject of * the node */ $subject = db_result(db_query("SELECT title FROM {node} WHERE nid = %d", $comment['nid'])); } /* * insert 'Re: ' (RFC XXX) */ $subject = 'Re: ' . $subject; $header['Subject'] = $subject; $header['From'] = _netnews_get_userinfo($comment['name'], $comment['uid'], $srv); _netnews_post_message($srv, $header, $comment['comment'], $comment['nid'], $comment['cid'], $comment['pid']); } /** * handles the actual posting of a message * * @param $srv news server configuration * @param &$header netnews header of the message * @param $body body of the message * @param $nid id of the node the message refers to directly or indirectly * @param $cid id of comment if applicable, default 0 * @param $pid id of comments parent comment if applicable, default 0 */ function _netnews_post_message($srv, & $header, $body, $nid, $cid = 0, $pid = 0) { /* * add a drupal message id to the header */ $header[NETNEWS_DRUPALID] = _netnews_drupalid($nid, $cid); /* * set status to failure so that this message will not be picked more than * once by the code that posts new messages */ _netnews_set_message_status($header[NETNEWS_DRUPALID], NETNEWS_SYNCHFAILED); /* * first store this message as new in the netnews node table, * and only then think about dealing with the news server */ // this is now handled at insert of node/comment //_netnews_store_drupal_message($srv['nngid'], $header[NETNEWS_DRUPALID], // $nid, $cid, $pid); $server = _netnews_create_nntpserver($srv); $header['Newsgroups'] = $srv['groupname']; if ($cid != 0) { $header['References'] = _netnews_get_references($nid, $pid, $server); } $msgid = ''; /* * get the current last message number of the newsgroup, the new message * will get a later one */ $last = $server->getLastMsgNum($srv['groupname']); if ($server->post($header, $body)) { _netnews_set_message_status($header[NETNEWS_DRUPALID], NETNEWS_SYNCHED); $msgid = _netnews_set_message_id($server, $srv['groupname'], $header[NETNEWS_DRUPALID], $last); } else { _netnews_trace('posting failed:', $server->getLog()); } $server->done(); } /** * inserts a record for a new drupal node or comment into netnews_messages * * @param $nngid id in netnews_links of the associated newsgroup * @param $drupalid id that identifies the message within the drupal site * @param $nid id of the node the message refers to directly or indirectly * @param $cid id of comment if applicable, default 0 * @param $pid id of comments parent comment if applicable, default 0 * * @return * TRUE if insert succeeded, otherwise FALSE */ function _netnews_store_drupal_message($nngid, $drupalid, $nid, $cid = 0, $pid = 0) { global $user; if (db_query("INSERT INTO {netnews_messages} (uid, nid, nngid, drupalid, status, origin, cid, pid) VALUES (%d, %d, %d, '%s', %d, %d, %d, %d)", $user->uid, $nid, $nngid, $drupalid, NETNEWS_NEWMSG, NETNEWS_ORIGIN_DRUPAL, $cid, $pid)) { return TRUE; } return FALSE; } /** * inserts a record into netnews_messages for a new netnews message * * @param $nngid id of netnews link config * @param $msgid netnews Message-ID * * @return * boolean: TRUE if insert succeeded, otherwise FALSE; */ function _netnews_store_nntp_message($nngid, $msgid) { return db_query("INSERT INTO {netnews_messages} (nngid, msgid, status, origin) VALUES (%d, '%s', %d, %d)", $nngid, $msgid, NETNEWS_NEWMSG, NETNEWS_ORIGIN_NETNEWS); } /** * retrieves the NNTP message-ID for a newly added message from the news server * and stores it in the netnews_messages table. * * @param $server news server object (class nntpserver) * @param $group name of the group on the news server * @param $drupalid id identifying the message on the drupal site * * @return * message-ID, or NULL if not found */ function _netnews_set_message_id($server, $group, $drupalid, $from) { $msgid = NULL; if ($hdr = $server->findHeaderWith($group, NETNEWS_DRUPALID, $drupalid, $from)) { _netnews_update_message_id($drupalid, $hdr['Message-ID']); $msgid = $hdr['Message-ID']; } else { _netnews_trace("could not find message with drupal-id $drupalid"); // TODO: add watchdog error log } return $msgid; } /** * sets the status of a netnews message in drupal to completed * * @param $msgid NNTP message-ID * @param $nid node id * @param $cid comment id * @param $pid parent comment id */ function _netnews_set_nntp_message_complete($msgid, $nid, $cid = 0, $pid = 0) { _netnews_trace("setting $msgid to SYNCHED"); db_query("UPDATE {netnews_messages} SET status = %d, nid = %d, cid = %d, pid = %d, drupalid = '%s' WHERE msgid = '%s'", NETNEWS_SYNCHED, $nid, $cid, $pid, _netnews_drupalid($nid, $cid), $msgid); } /** * sets the status of a netnews message in drupal to synchfailed * * @param $msgid NNTP message-ID * @param $nid node id * @param $cid comment id * @param $pid parent comment id */ function _netnews_set_nntp_message_failed($msgid) { _netnews_trace("setting $msgid to SYNCHFAILED"); db_query("UPDATE {netnews_messages} SET status = %d WHERE msgid = '%s'", NETNEWS_SYNCHFAILED, $msgid); } /** * updates the status of a message in the netnews_messages table * * @param $drupalid id that identifies the message on the drupal site * @param $status new value for the status column */ function _netnews_set_message_status($drupalid, $status) { db_query("UPDATE {netnews_messages} SET status = %d WHERE drupalid = '%s'", $status, $drupalid); } /** * updates the NNTP message-ID of a message in the netnews_messages table * * @param $drupalid id that identifies the message on the drupal site * @param $msgid the NNTP message-ID to be stored in the msgid column */ function _netnews_update_message_id($drupalid, $msgid) { db_query("UPDATE {netnews_messages} SET msgid = '%s' WHERE drupalid = '%s'", $msgid, $drupalid); } /** * creates a string of user info to be used in an NNTP header * * @param $name username as it appears in a node or comment * @param $uid of the user that created a node or comment * @param $server news server configuration * * @return * string identifying a Drupal user */ function _netnews_get_userinfo($name, $uid, $server) { $email = db_result(db_query("SELECT mail FROM {users} WHERE uid = %d", $uid)); return "$name <$email>"; } /** * creates a string that identifies a message (node or comment) on a drupal site * * @param $nid id of the node that is or "owns" the message * @param $cid id of the comment (should be 0 for a node) * * @return * unique (within the site) identifier for a message */ function _netnews_drupalid($nid, $cid = 0) { global $base_url; return "$base_url:$nid" . (($cid != 0) ? ".$cid" : ''); } /** * creates the 'References' field for the NNTP header of a comment * * @param $nid id of the node referenced * @param $pid id of parent comment, if applicable (should be 0 if not) * @param $server server configuration * * @return * string for NNTP 'References' field, or FALSE if failed */ function _netnews_get_references($nid, $pid, $server) { $parentmsgid = db_result(db_query("SELECT msgid FROM {netnews_messages} WHERE drupalid = '%s'", _netnews_drupalid($nid, $pid))); _netnews_trace("getting header for $parentmsgid (" . _netnews_drupalid($nid, $pid) . ")"); if (($header = $server->getHeader($parentmsgid)) !== NULL) { $references = ''; if ($pid != 0) { $references = $header['References']; } $references = $references . " $parentmsgid"; return $references; } else { _netnews_trace("cannot get header for $parentmsgid"); } return NULL; } /** * creates and initializes an nntpserver object * * @param srvinfo netnews link config * * @return * nntpserver object */ function _netnews_create_nntpserver($srvinfo) { require_once ('nntpserver.inc'); $server = new nntpserver(_netnews_get_nntp_server($srvinfo)); $server->setLogging($srvinfo['runtime']['logerrors'], $srvinfo['runtime']['loginfo'], $srvinfo['runtime']['logprotocol']); if ($srvinfo['runtime']['tracenntp']) { $server->setTracer($srvinfo['runtime']['tracer']); } return $server; } /** * retrieves the news server configuration from a netnews_links record * * @param $linkrec netnews_links record read through db_fetch_array() * * @return * server configuration array */ function _netnews_get_nntp_server($linkrec) { return array ( 'nntp_server' => $linkrec['servername'], 'nntp_port' => $linkrec['serverport'], 'nntp_user' => ($linkrec['authuser']) ? $linkrec['authuser'] : NULL, 'nntp_passwd' => $linkrec['authpasswd'], 'nntp_group' => $linkrec['groupname'] ); } /** * Switch from original user to message submission user and back. * * Note: You first need to run mailhandler_switch_user without * argument to store the current user. Call _netnews_switch_user * without argument to set the user back to the original user. * * @param $uid The user ID to switch to * * borrowed from mailhandler.module */ function _netnews_switch_user($uid = NULL) { global $user; static $orig_user = array (); if (!count($orig_user)) { // store the initial user $orig_user[] = $user; } if (isset ($uid)) { $user = user_load(array ( 'uid' => $uid )); } // retrieve the initial user, can be called multiple times else if (count($orig_user)) { $user = array_shift($orig_user); array_unshift($orig_user, $user); } } /** * logs error/debugging messages to a local file * * @param $msg message to be logged * @param $object object to print through print_r * * TODO: connect this to Drupals logging system? */ function _netnews_trace($msg, $object = NULL) { $runtime = _netnews_current_runtime(); if ($runtime['tracing']) { $fp = fopen( dirname(__FILE__).'/' . $runtime['name'] . '.trace', 'a'); if ($fp) { fwrite($fp, format_date(time(), 'small') . ": $msg\n"); if ($object !== NULL) { fwrite($fp, print_r($object, TRUE) . "\n"); } } fclose($fp); } } /** * initializes an array with runtime settings * * @param $edit form fields from the netnews admin add link form * * @return * array with runtime settings */ function _netnews_runtime_info($edit = array ()) { static $defaults = array ( 'synch_often' => FALSE, 'synch_on_exit' => FALSE, 'tracing' => FALSE, 'logerrors' => FALSE, 'loginfo' => FALSE, 'logprotocol' => FALSE, 'name' => '', 'nngid' => 0, 'tracenntp' => FALSE, 'tracer' => '_netnews_trace' ); if (TRUE) { //variable_get("netnews_trace_all", FALSE)) { $defaults['tracing'] = TRUE; $defaults['name'] = 'netnews_default'; } $runtime = array (); foreach ($defaults as $label => $default) { if (array_key_exists($label, $edit)) { $runtime[$label] = $edit[$label]; } elseif (array_key_exists('logging', $edit) && is_array($edit['logging'])) { if (array_search($label, $edit['logging']) !== FALSE) { $runtime[$label] = TRUE; } } else { $runtime[$label] = $default; } } return $runtime; } /** * sets/gets the current runtime config * * @param $name name of a netnews link config * * @return * the current runtime config */ function _netnews_current_runtime($name = NULL) { static $current = NULL; if ($name === NULL) { if ($current === NULL) { $current = _netnews_runtime_info(); } } else { $current = variable_get('netnews_' . $name, _netnews_runtime_info()); } return $current; } /** * sets or reads a static lock flag that can be used as a "semaphore" to prevent * netnews.module from trying to send a message from NNTP that is just inserted * back to NNTP * NOTE: only works within one execution run of this module * TODO: check if this is a valid way of solving the problem * * @param $set boolean indicating whether the flag should be set (TRUE) or * read (FALSE) * @param $newvalue new value for the flag if $set is TRUE * * @return * current flag value if $set is FALSE, old flag value if $set is TRUE */ function _netnews_insert_lock($set = 'read', $newvalue = FALSE) { static $locked; if ($set == 'set') { $oldvalue = $locked; $locked = $newvalue; return $oldvalue; } else { return $locked; } } /* * TODO, and to use */ function _netnews_error($msg) { // write error message to watchdog log } function _netnews_fix_taxonomy($nngid, $query = FALSE) { $name = db_query(db_result('SELECT name FROM {netnews_links} WHERE nngid = %d', $nngid)); _netnews_current_runtime($name); _netnews_trace("Fixing taxonomy"); $query = "SELECT m.nid, l.tid FROM {netnews_messages} m, {netnews_links} l WHERE m.nngid = l.nngid AND m.cid = 0 AND l.nngid = %s"; $broken = FALSE; $inserts = array (); if ($result = db_query($query, $nngid)) { while ($n = db_fetch_object($result)) { if (!db_result(db_query('SELECT COUNT(*) FROM {term_node} WHERE nid = %d AND tid = %d', $n->nid, $n->tid))) { $broken = TRUE; _netnews_trace("insert candidate (nid,tid): (" . $n->nid . ", " . $n->tid . ')'); $inserts[] = $n; } } if ($inserts) { foreach ($inserts as $n) { _netnews_trace("insert term_node(nid,tid): (" . $n->nid . ", " . $n->tid . ')'); db_query('INSERT INTO {term_node} (nid,tid) VALUES (%d,%d)', $n->nid, $n->tid); } } } return $broken; } /************************ * Attachment Support */ require_once('Mail/MimeDecode.php'); /** * Menu callback; Imports or re-imports the named message * Requires the local node ID. Must be an existing node. * @author dan */ function _netnews_admin_refresh($nid = 0) { if(! $nid && is_numeric(arg(1)) ){$nid = arg(1);} drupal_set_message("Starting refresh. Node last changed: ".date('r',node_last_changed($nid))) ; if ($msg_info = _netnews_nntp_message_of_node($nid) ) { $srvinfo = _netnews_admin_load($msg_info['nngid']); //drupal_set_title(t("Updating Message ".$msg_info['nid']." = Message ".htmlspecialchars($msg_info['msgid'])." in ").$srvinfo['name']); drupal_set_title(t("Updating Node %id : %title",array('%id'=>$msg_info['nid'] , '%title'=>htmlspecialchars($msg_info['msgid']) ) ) ) ; if(!$msg_info['msgid']){ drupal_set_message("Can't find record of this original message request ".print_r($msg_info,1),"error"); } require_once(drupal_get_path("module","netnews") .'/nntpserver.inc'); $server = _netnews_create_nntpserver($srvinfo); _netnews_trace("requesting message '".$msg_info['msgid']."'"); unset($message); if ($message = $server->getArticle($msg_info['msgid'], $msg_info['msgnum']) ) { $hdr = $message[0]; $text = implode($message[1],"\r\n"); _netnews_trace('Got Article from server. Text length is '.strlen($text)." chars",$message); $hdr['Date'] = date('r'); // Override date to allow node over-write - we are refreshing! // drupal_set_message("Inserting message, node dated ".date('r',node_last_changed($node->nid)) ); _netnews_trace('Adding node of type '.$srvinfo['type'].' when refreshing'); if (!( $nid = _netnews_add_node($hdr, $text, $srvinfo) ) ){ _netnews_trace(t("Failed to Add Node when refreshing")); $node = node_load(array('nid' => $nid)); $node->body="... ".strlen($node->body)." bytes"; _netnews_trace("Message ".$hdr['Message-ID']." was not added as a node.", array( 'header' => $hdr, 'old_node' => $node ) ); } else { _netnews_trace(t("Added Node OK")); $node = node_load(array('nid' => $nid)); } // node_load doesn't seem to remember to reload its taxonomy // better do so or all the current associations go missing on the next save $node->taxonomy = array_keys(taxonomy_node_get_terms($node->nid)); // Server may have found some attachments, which it's held on to for us if($server->attachments){ _netnews_add_attachments(&$node,$server->attachments); } else { _netnews_trace('no attachments retained by the server'); } _netnews_trace("Inserted message, node ".$node->nid." dated ".date('r',node_last_changed($node->nid)) ); drupal_set_message(l(t("Imported Message ".$hdr["Subject"]),"node/$nid")); // $node = node_load(array('nid' => $nid),NULL,TRUE); // reload it, clear cache $output .= node_view($node, 0, 1); } else { if($message === ''){ $msg = t("Failed to retrieve article message retrieved from server was totally blank" ); } else { $msg = t("Failed to retrieve article message '".htmlspecialchars($msg_info['msgid'])."' from server" ); } drupal_set_message($msg,'error' ); _netnews_trace($msg, array('server' => $server,'server config' => $srvinfo, 'header' => $hdr)); } if (count($server->getLog())) { _netnews_trace('logging from NNTP session: ', $server->getLog()); } $server->done(); } else { drupal_set_message( t("Cannot refresh Node $nid as a netnews node, It has no original messageid recorded to sync with."),'error'); } print theme('page', $output); } /** * * End of callbacks * */ /** * Attach a given attachment to a node * The attachment is handed as a filepath * to be saved somewhere else. * Files are saved in the configured directory netnews_attachment_storage_path * which defaults to files/netnews * By default, the given filename is used, this can * be modified in the netnews settings. * * If the attachment is an image, the image.module (if present) * is used to thumbnail and associate the file with the node. * If not, the file is just hung on as an attachment. * * Attachments may not show up by default in all node view screens. * * * Some of this code based on * image_import_do_node() * @public * @param $node Handle on the node to add the attachment to * @param $path Filepath of the file to be attached. Its name will be retained */ function netnews_add_attachment(&$node, $path) { _netnews_trace("Adding attachment to node ".$node->nid); if(! isset($node->images)){$node->images = array();} $new_path = variable_get('file_directory_path','').basename($path); // Copy the image file into the appropriate directory $attachment_dir = variable_get('netnews_attachment_storage_path',variable_get('file_directory_path', 'files')."/netnews"); if(! is_dir($attachment_dir) ){ mkdir($attachment_dir);} file_create_path($attachment_dir); // Need to do advanced file name munging here, akin to the config path.module has $dest = $attachment_dir.basename($path); $new_path = $path; if (file_copy(&$new_path, $attachment_dir, FILE_EXISTS_RENAME)) { // The $new_path receives the true destination path (if rename was neccesary _netnews_trace(t('Copied %p to %np.', array('%p'=>$path, '%np'=>$new_path))); $path = $new_path; } else { _netnews_trace(t('File copy failed for %p to %tp', array('%p'=>$path, '%tp'=>$tmp_path)), 'error'); return FALSE; } // Attach it to the node // drupal_set_message("Node contains:

".print_r($node,1)."
"); // We hope, but don't assume that image.module is present if(function_exists("_image_build_derivatives") && image_get_info($new_path)){ // This is a private function from image.module $node->images['_original'] = $new_path; // drupal_set_message("image_get_info : ".print_r( image_get_info($node->images['_original']),1) ); _image_build_derivatives(&$node, TRUE); // this actually produces no useful return. Can't tell what happened image_update(&$node); // ensure this new info gets saved. This is not a true 'image' node so the hook_update would not be called otherwise _netnews_trace("Created thumbnails with image module",$node); $node->teaser = l(image_display($node, 'thumbnail'), 'node/'.$node->nid, array(), NULL, NULL, FALSE, TRUE) ; if(_image_build_derivatives($node, TRUE)) { _netnews_trace(t('Created thumbnail preview for attachment')); } else { _netnews_trace(t('Failed to Create thumbnail preview for node attachment. _image_build_derivatives() messed up on '.$node->images['_original'])); } } else { _netnews_trace("The image.module is either unavailable or has changed its internals - can't make thumbnails") ; // Whatever this attachment is, it didn't validate as an image // Adding it to the files table is all we can do to attach it _netnews_trace("File saved as an attachement in the files table"); // Register it in the files table $fid = db_next_id('{files}_fid'); db_query("INSERT INTO {files} (fid, nid, filename, filepath, filemime, filesize) VALUES (%d, %d, '%s', '%s', '%s', %d)", $fid, $node->nid, basename($path), $path, '', ''); db_query("INSERT INTO {file_revisions} (fid, vid, list, description) VALUES (%d, %d, %d, '%s')", $fid, $node->vid, '', 'Imported attachment'); } // drupal_set_message("Saving modified ".$node->type." node.
".print_r($node,1)."
"); // Is node save strictly needed here? won't the main routine take care of it // - no, as the attachment may happen after the first save node_save($node); _netnews_trace(t('DEBUG: Saved node nid=%nid',array('%nid'=>$node->nid))); watchdog('image import',t('Imported "%p" as node %nid',array('%p'=>$path, '%nid'=>$node->nid)), WATCHDOG_NOTICE, l(t('view'),"node/$nid")); // drupal_set_message("Saved modified ".$node->type." node.
".print_r($node,1)."
"); return TRUE; } /** * Given an array of attachment blocks, carefully attach them to a node * The blocks contain raw, usually binary, file contents, so * this method saves them before attaching them one by one * to the given node. * * Attachment blocks are each a 3-part array, filename, fileperm, filedata * as returned from Pear Mail UUDecode * Files are created initially in the TEMP dir, then imported * to the system by netnews_add_attachment (singular) * The given filename is used. */ function _netnews_add_attachments(&$node,&$attachments){ if(! $node->nid){ _netnews_trace("Cant add attachments to a node with an undefined nid yet, sorry. Aborting add"); return; } _netnews_trace("Adding ".count($attachments)." attachments to node"); if($attachments){ foreach($attachments as $filename => $data){ _netnews_trace(t("Found Attachment ".$data['filename'])); $data['filename']=preg_replace('/(\s)/','\$1',$data['filename']); $filepath = $_ENV['TEMP']."/".$data['filename']; // could this be a security hole? Believing the given filename if(file_put_contents($filepath,$data['filedata'])){ $fstat = stat($filepath); $sz = human_file_size($fstat['size']) ; _netnews_trace("Extracted attachment ".$data['filename'].". Saved as $filepath"); if(isset($data['fileperm']) && ($data['fileperm'] == '000' ) ){ // drupal_set_message("Saved (probably corrupt) file attachment as $filepath ... not actually attaching it" ); // file_put_contents($filepath.'.yenc',$data['fileraw']); } else{ if( netnews_add_attachment(&$node, $filepath) ){ $node->body .= "\n
  • ". l($data['filename'] , $filepath ). " [$sz]
  • "; _netnews_trace(t("Added attachment ".$data['filename'])); } else { drupal_set_message(t("failed to add attachment ".$data['filename'])); _netnews_trace("Failed to add $filepath as an attachment. "); } } } else { drupal_set_message(t("Extracted attachment, but failed to save it as ".htmlspecialchars($filepath).". Either bad filesystem path configured, permissions problems, or illegal filename.")); _netnews_trace("Extracted attachment, but failed to save it as $filepath"); } } } else { _netnews_trace(t("No attachments added")); } } /** * Scan the given headers to try and extract any * relevant keywords describing this item. * * Binary newsgroup posts often follow repeatative * patterns, by collecting these patterns, I hope to be able * to group similar posts together. There is a lot of guesswork * and there will be a few false positives. * * The post Subject will be broken down into tokens of textual, non-numeric * chunks. Each of these will be a key phrase. * The Poster and the (official messageID) source will each * also be keyed to create albums */ function netnews_infer_keywords(&$node,&$hdr){ // Try to chunk the title $ok_chars='a-zA-Z\.\ \'\"\-'; //define all characters that are meaningless if found on the outside of a string - pretty much anything non-text, although $min_length = 4; $boundries='\[\]\(\)\>\<'; $preg = "/([^$boundries]+)/"; preg_match_all($preg,$hdr['Subject'],$matches); // drupal_set_message("When matching
    $preg
    on '".$hdr['Subject']."' , I found title chunks to vaguely classify this entry :
    ".print_r($matches,1)."
    " ); $chunks = $matches[0]; $keyphrases = array(); foreach($chunks as $chunk){ $chunk=trim($chunk); //$chunk=preg_replace('/\S+\.\S\S\S$/',"",$chunk); // If the last part of the chunk looks like a filename X.xxx trim it $chunk=preg_replace('/\b\S+\.\S\S\S\b/',"",$chunk); // If any part of the chunk looks like a filename X.xxx extract it //$chunk=trim($chunk); $chunk=preg_replace('/$[^\w](.*)[^\w]/','\1',$chunk); // Super-trim. Remove any outlying non-alpha-numeric if(preg_match('/^[^a-zA-Z]*$/',$chunk)){continue;} // consists entirely of non-alphabetics (numbers and stuff) Unlikely to have anything I can use // if(preg_match('/^\W*$/',$chunk)){continue;} // consists entirely of non-words (numbers and stuff) if(strlen($chunk)<$min_length){continue;} $keyphrases[] = $chunk; } drupal_set_message("When Scanning the title , I found keyphrases ".implode(", ",$keyphrases)."" ); return($keyphrases); } /** * The opposite to _netnews_nntp_message_is_new, this returns the node id if found */ function _netnews_nntp_message_already_exists($msgid) { $q = "SELECT nid FROM {netnews_messages} WHERE msgid = '$msgid'"; return db_result(db_query("SELECT nid FROM {netnews_messages} WHERE msgid = '%s'", $msgid)) ; } /** * Given a local Node id, what was the original net messageid */ function _netnews_nntp_message_of_node($nid) { return db_fetch_array(db_query("SELECT * FROM {netnews_messages} WHERE nid = '$nid'")) ; } /** * Utils added by .dan. */ if(! function_exists("file_put_contents") ){ /* A version to support PHP4 */ function file_put_contents($filename, $data, $flags = 0, $f = FALSE) { // ensure spaces and special characters are slash-escaped ... if they aren't already $filename=preg_replace('/(?= 4.3) { $MAXSTRLEN = 64; $s = '
    ';
               $traceArr = debug_backtrace();
               array_shift($traceArr);
               $tabs = sizeof($traceArr)-1;
               foreach ($traceArr as $arr) {
                   for ($i=0; $i < $tabs; $i++) $s .= '   ';
                   $tabs -= 1;
                   $s .= '';
                   if (isset($arr['class'])) $s .= $arr['class'].'.';
                   if(is_array($arr['args'])){
                   foreach($arr['args'] as $v) {
                       if (is_null($v)) $args[] = 'null';
                       else if (is_array($v)) $args[] = 'Array['.sizeof($v).']';
                       else if (is_object($v)) $args[] = 'Object:'.get_class($v);
                       else if (is_bool($v)) $args[] = $v ? 'true' : 'false';
                       else { 
                           $v = (string) @$v;
                           $str = htmlspecialchars(substr($v,0,$MAXSTRLEN));
                           if (strlen($v) > $MAXSTRLEN) $str .= '...';
                           $args[] = $str;
                       }
                   }
                   }
                   
                   $s .= $arr['function'].'('.implode(', ',$args).')';
                   $s .= sprintf(" # line %4d,".
      " file: %s",
      $arr['line'],$arr['file'],$arr['file']);
                   $s .= "\n";
               }    
               $s .= '
    '; if ($print) print $s; } return $s; }