CVS edit link for cleaver

I wish to contribute a module I developed for the website Canweather.com. This module, tweetrss, takes an RSS feed and filters and formats it and posts it to twitter.com. It is dependent on the twitter module for all twitter-specific operations. It is also dependent on the core aggregator module for RSS feeds. Future enhancements may support feed module for more flexibility.

The filtering and formatting operation uses a hook to allow extendability.

The tweetrss module features regular expressions for filtering and formatting using the PHP function preg_replace(). Also included is a sample extension module, tweetrss_canweather, to demonstrate how tweetrss can be extended. The tweetrss_canweather module has a version of the logic used on canweather.com.

Comments

cleaver’s picture

StatusFileSize
new4.84 KB
cleaver’s picture

Status: Postponed (maintainer needs more info) » Needs review
cleaver’s picture

StatusFileSize
new61.77 KB
new36.58 KB
new24.61 KB

Explanation of use of module.

Initial Setup:
1. configure Oauth and Twitter modules.
2. register application with twitter.com
3. add twitter oauth keys to twitter module.
4. add twitter account(s).
5. enable aggregator module and add some rss feeds.

Configure tweetrss module:
6. Add a feed: admin/settings/tweetrss/add
7. Select the twitter account and feed to be linked.
8. Select the filter type.
9. For the basic tweetrss filter type, write regular expressions for the filter and transformation of the feed.
10. Save.
11. Run cron. (for normal production use, cron must be called on schedule).

Adding some screenshots...

cleaver’s picture

Link to Github repo, if it makes review any easier:

http://github.com/cleaver/TweetRSS

avpaderno’s picture

Issue tags: +Module review

Hello, and thanks for applying for a CVS account. I am adding the review tags, and some volunteers will review the code, pointing out what it needs to be changed.

avpaderno’s picture

Status: Needs review » Needs work
  • The points reported in this review are not in order or importance / relevance.
  • Most of the times I report the code that present an issue. In such cases, the same error can be present in other parts of the code; the fact I don't report the same issue more than once doesn't mean the same issue is not present in different places.
  • Not all the reported points are application blockers; some of the points I report are simple suggestions to who applies for a CVS account. For a list of what is considered a blocker for the application approval, see CVS applications review, what to expect. Keep in mind the list is still under construction, and can be changed to adapt it to what has been found out during code review, or to make the list clearer to who applies for a CVS account.
  1. The module doesn't implement hook_uninstall().
  2. Hook implementation comments should be like the following one:
    /**
     * Implements hook_menu().
     */
    
  3. function tweetrss_enable() {
      variable_set('tweetrss', array('tweetrss' => 'Tweet RSS Regex'));
    }
    
    

    What is the purpose of changing the value of that Drupal variable each time the module is enabled, or disabled? If the module is disabled, that Drupal variable is not even used.

  4. Menu descriptions and titles, schema descriptions are not passed to t().
  5. Remove any debugging code.
  6. function _tweetrss_feed_row($feed) {
      $form['#feed'] = $feed;
      $form['screen_name'] = array(
        '#type' => 'markup',
        '#value' => $feed['screen_name'],
      );
      $form['title'] = array(
        '#type' => 'markup',
        '#value' => $feed['title'],
      );
      $form['feedtype'] = array(
        '#type' => 'markup',
        '#value' => $feed['feedtype'],
      );
      $form['edit_link'] = array(
        '#type' => 'markup',
        '#value' => l(t('edit'), "admin/settings/tweetrss/edit/{$feed['tfid']}"),
      );
      return $form;
    }
    
    

    Why is the code using a form builder, if there are no form fields?

  7. function tweetrss_filter_xss($string) {
      // Only operate on valid UTF-8 strings. This is necessary to prevent cross
      // site scripting issues on Internet Explorer 6.
      if (!drupal_validate_utf8($string)) {
        return '';
      }
      // Remove Netscape 4 JS entities
      $string = preg_replace('%&\s*\{[^}]*(\}\s*;?|$)%', '', $string);
    
      // Defuse all HTML entities
      $string = str_replace('&', '&', $string);
      // Change back only well-formed entities in our whitelist
      // Decimal numeric entities
      $string = preg_replace('/&#([0-9]+;)/', '&#\1', $string);
      // Hexadecimal numeric entities
      $string = preg_replace('/&#[Xx]0*((?:[0-9A-Fa-f]{2})+;)/', '&#x\1', $string);
      // Named entities
      $string = preg_replace('/&([A-Za-z][A-Za-z0-9]*;)/', '&\1', $string);
    
      return $string;
    }
    
    

    Why isn't the code using filter_xss()?

cleaver’s picture

Status: Needs work » Needs review
StatusFileSize
new4.52 KB

Thanks for the time you put in on reviewing my module. I think I've addressed all the issues and I've attached an updated module.

  1. The module doesn't implement hook_uninstall().

    The uninstall hook was there already

  2. Hook implementation comments should be like the following one:

    I have updated the code.

  3. What is the purpose of changing the value of that Drupal variable each time the module is enabled, or disabled? If the module is disabled, that Drupal variable is not even used.

    The Drupal variable is used to store an array containing a list of the available filter submodules. As add-on filter modules are enabled and disabled, this array will have elements added and removed. The array is used to generate a drop drop down in the edit screen--this is used to select the filter module to apply.

  4. Menu descriptions and titles, schema descriptions are not passed to t().

    I did not use t(). As I understand it, D6 menu definitions should not use t(): http://drupal.org/node/103114 I think it throws a D5->D6 Coder error if you do use t().

  5. Remove any debugging code.

    I removed the debug level watchdog statements. Normally I would keep them for most logging systems, but Drupal seems to record all levels and only filters them on display.

  6. Why is the code using a form builder, if there are no form fields?

    Removed. This was a legacy of a previous version that allowed edit on the list page. It was no longer required.

  7. Why isn't the code using filter_xss()?

    This special function is required in order to sanitize a regex without corrupting it. The regex matches the XML of the RSS feed so it has to allow non-HTML tags. Additionally filter_xss() corrects mismatched tags, which could further corrupt the regex. The tweetrss_filter_xss() function keeps the check for code insertion.

    I did some checking on modules that allow users to enter a regex and it looks like most will display it without any sanitizing. This module appears to have the most secure handling of regex values that I have seen in contrib.

avpaderno’s picture

Status: Needs review » Needs work
  1.   $schema['tweetrss_feed'] = array(
        'description' => t('Record a list of tweetrss feeds'),
        'fields' => array(
          'tfid' => array(
            'description' => t('tweetrss feed ID'),
            'type' => 'serial',
            'not null' => TRUE,
          ),
          'twitter_uid' => array(
            'description' => t('Twitter user ID'),
            'type' => 'numeric',
            'not null' => TRUE,
            'default' => 0,
            'precision' => '20',
            'scale' => '0',
          ),
          'fid' => array(
            'description' => t('Feed Id'),
            'type' => 'int',
            'not null' => TRUE,
            'default' => 0,
          ),
    
    

    Menu descriptions and titles, as well as schema descriptions, are not passed to t().
    The strings used as description should start with a capitalized word, end with a period, and contain a sentence (not a phrase).

  2. The uninstall hook was there already

    I should have better reported what I meant: the module doesn't implement the code to remove the Drupal variables it defines in hook_uninstall().

    The Drupal variable is used to store an array containing a list of the available filter submodules. As add-on filter modules are enabled and disabled, this array will have elements added and removed. The array is used to generate a drop drop down in the edit screen--this is used to select the filter module to apply.

    In that case, the module should use a custom hook that allows filter submodules to report information about themselves.
    Furthermore, it doesn't make sense for the main module to delete the Drupal variable it uses for that purpose to signal to itself that there are no filter submodules; since the main module is disabled, its code will not be executed, and it cannot verify if there are filter submodules.

  3. /**
     * Implements hook_perm()
     */
    function tweetrss_canweather_perm() {
      return array('administer tweetrss');
    }
    
    

    Two modules cannot declare the same permission.

  4.   if (preg_match("/Current Conditions|Conditions actuelles/", $data)) {
        $include = TRUE;
      }
    
    

    Why is the code checking only for English or French strings?

  5.   // Remove Netscape 4 JS entities
      $string = preg_replace('%&\s*\{[^}]*(\}\s*;?|$)%', '', $string);
    
      // Defuse all HTML entities
      $string = str_replace('&', '&', $string);
      // Change back only well-formed entities in our whitelist
      // Decimal numeric entities
      $string = preg_replace('/&#([0-9]+;)/', '&#\1', $string);
      // Hexadecimal numeric entities
      $string = preg_replace('/&#[Xx]0*((?:[0-9A-Fa-f]{2})+;)/', '&#x\1', $string);
      // Named entities
      $string = preg_replace('/&([A-Za-z][A-Za-z0-9]*;)/', '&\1', $string);
    
    

    All those calls to preg_replace() can be replaced by a single call.

  6.       $sql =
            "INSERT INTO {tweetrss_feed} (
              twitter_uid,
              fid,
              feedtype,
              regex_match,
              regex_replace,
              checked )
            VALUES (%d, %d, '%s', '%s', '%s', %d)";
          db_query($sql,
            $feed['twitter_uid'],
            $feed['fid'],
            $feed['feedtype'],
            $feed['regex_match'],
            $feed['regex_replace'],
            $feed['checked']);
    
    

    Why isn't the code using drupal_write_record()?

  7. The code, in some points, is not formatted as reported by the coding standards.
cleaver’s picture

Status: Needs work » Needs review
StatusFileSize
new4.52 KB

Thanks again kiamlaluno, for your review! I've made some further changes to my module, which I've described below...

  1. Menu descriptions and titles, as well as schema descriptions, are not passed to t().
    The strings used as description should start with a capitalized word, end with a period, and contain a sentence (not a phrase).

    Sorry, I didn't understand before. I've updated my source to remove the t() and make the comments more clear. I used the output from Schema module which curiously added the t(). I've made a note in the Schema module issue queue.

  2. The uninstall hook was there already

    I should have better reported what I meant: the module doesn't implement the code to remove the Drupal variables it defines in hook_uninstall().

    The Drupal variable is used to store an array containing a list of the available filter submodules. As add-on filter modules are enabled and disabled, this array will have elements added and removed. The array is used to generate a drop drop down in the edit screen--this is used to select the filter module to apply.

    In that case, the module should use a custom hook that allows filter submodules to report information about themselves.
    Furthermore, it doesn't make sense for the main module to delete the Drupal variable it uses for that purpose to signal to itself that there are no filter submodules; since the main module is disabled, its code will not be executed, and it cannot verify if there are filter submodules.

    To simplify things, I got rid of the variable and used a hook. This let me delete the install file for the filter sub-module.

  3. Two modules cannot declare the same permission.

    Removed.

  4. Why is the code checking only for English or French strings?

    This is a sample filter module to show how to use the hooks and was developed for a specific RSS feed, which is only in English and French. (It is the Canadian government Ministry of Environment.)

  5. All those calls to preg_replace() can be replaced by a single call.

    I took an exact copy of the core function, filter_xss() and removed the checks that would corrupt the regex. I was planning to keep it as close to the core function as possible so that I could improve it when the core function improves.

  6. Why isn't the code using drupal_write_record()?

    I changed the code to use this function.

  7. The code, in some points, is not formatted as reported by the coding standards.

    I cleaned up a bit and it tests clean on Coder module. Not sure if I missed anything else...

On each round of changes I'm reducing the lines of code, so that is good...

avpaderno’s picture

Status: Needs review » Fixed

Thank you for your contribution! I am going to update your account.

These are some recommended readings to help with excellent maintainership:

You can find more contributors chatting on the IRC #drupal-contribute channel. So, come hang out and stay involved.
Thank you, also, for your patience with the review process.
Anyone is welcome to participate in the review process. Please consider reviewing other projects that are pending review. I encourage you to learn more about that process and join the group of reviewers.

I thank all the dedicated reviewers as well.

Status: Fixed » Closed (fixed)
Issue tags: -Module review

Automatically closed -- issue fixed for 2 weeks with no activity.

avpaderno’s picture

Component: Miscellaneous » new project application
Assigned: Unassigned » avpaderno
Issue summary: View changes
avpaderno’s picture