CVS edit link for subhrajit

Information about myself:
---------------------------------
I am an experienced PHP coder and have previously created modules for PmWiki, besides writing independent PHP codes for several web applications. You can read about my PmWiki contributions at http://www.pmwiki.org/wiki/Profiles/Subhrajit .
I am new to Drupal development and have recently written a security module for my web-sites for which I couldn't find any existing module to serve the same purpose.
You can read more about me at http://fling.seas.upenn.edu/~subhrabh/cgi-bin/wiki/index.php?n=Projects.... .

Below are the details of the security module for Drupal I have developed and would like to commit.

---------------------------------------------------
Node File Access Module for Drupal 6.x
*******************************************

Description:
============
Node File Access uses a host of generic approaches to identify files uploaded/created and used by modules like IMCE, Galarix and Upload, and associate them with the respective nodes that use them. The access to those files is then monitored by Node File Access, and access is granted to the files only if the user requesting it is able to access the node with which the file is associated.

Motivation:
===========
Presently a very few modules (like the Upload module) reliably monitors the access to files based on the permission of the node to which the file was uploaded. However modules like IMCE (http://drupal.org/project/imce) or Galarix does not monitor the access to files uploaded through them. For example, in creating a node, say "node/n", an authenticated user makes use of the IMCE or Galarix modules to upload a picture IMAGE.jpg. For the authenticated user this image is obviously available through, say http://mysite.com/drupal/sites/default/files/IMAGE.jpg (or http://mysite.com/drupal/?q=system/files/IMAGE.jpg). Now say, the authenticated user sets protection for the node using modules like Protected Node (http://drupal.org/project/protected_node) or Nodeaccess (http://drupal.org/project/nodeaccess) such that a visitor will not be able to access the node "node/n" or will be able to access only using a password. However, in spite of protecting the node, a visitor will still be able to access the image file using the direct URL http://mysite.com/drupal/sites/default/files/IMAGE.jpg (or http://mysite.com/drupal/?q=system/files/IMAGE.jpg).

Explanation:
------------
This happens because IMCE or Galarix (and many similar modules) do not implement a hook_file_download. Many of them (like IMCE) don't even keep record of which node was used to upload a particular file (often it doesn't even make sense to do that since a file may be used by multiple nodes). Thus protections applied to the nodes does not apply to the files themselves.

The solution:
=============
The Node File Access module tries to solve this issue in two stages:

1. First it uses a host of approaches (that is flexible for modification) to identify which files are associated to which nodes. This includes looking inside the HTML content of a node to find references to files as well as looking into different databases of other modules. How the look-ups are performed can be monitored form the module's settings page.
A file is associated with only one node (even if it is used by multiple nodes), and the author of a node will have the ability to transfer the association to a different node manually. Typically the node that used the file first will have the association of the file and will be able to "lock" the association by default. Unlocking the association from a node will enable other nodes to claim association of a file.
Thus it populates, maintains and monitors association of files with Drupal nodes.

2. Next, when a request for a file is received (say http://mysite.com/drupal/sites/default/files/IMAGE.jpg or http://mysite.com/drupal/?q=system/files/IMAGE.jpg) Node File Access first redirects to the corresponding associated node ("node/n" in our example case). If the node loads completely (i.e. without receiving access denied or a password request from some arbitrary security module) and no other module prevents the file from being downloaded, only then does Node File Access sends back the requested file.

In order to monitor access to direct URL-based file requests like "http://mysite.com/drupal/sites/default/files/IMAGE.jpg", Node File Access alters the .htaccess file at the Drupal root to redirect such requests appropriately (in this case to "http://mysite.com/drupal/?q=nodefileaccess&file=sites/default/files/IMAG..."). Node File Access also implements a hook_file_download to monitor requests like "http://mysite.com/drupal/?q=system/files/IMAGE.jpg".

The Node File Access module lets an administrator control how Node File Access looks for and create association of files with nodes. It also lets creators/editors of nodes control whether to not associate a particular file with the node or whether to lock some associations with a node.

Technical details:
==================
The following workflows describe how Node File Access works:

1. A request for "http://mysite.com/drupal/sites/default/files/IMAGE.jpg" is received.
2. Apache redirects the request to "http://mysite.com/drupal/?q=nodefileaccess&file=sites/default/files/IMAG..." (the redirect rules are written and maintained by Node File Access in .htaccess file of Drupal's root).
3. The 'nodefileaccess' menu takes over and looks up the association table to find associated node "node/n".
4. A redirect is set to "http://mysite.com/drupal/?q=node/n&file=sites/default/files/IMAGE.jpg".
5. All the callbacks corresponding to "node/n" gets processed, and hence, if at any point, a security block is encountered, the process stops.
6. If the $_GET['file'] parameter is set, Node File Access takes back control at the 'alter' level of 'nodeapi'.
7. Node File Access once again checks if the requested file is associated with "node/n", and then passes on the request to Drupal's 'file_download' function.

Note that due to the step 7, it is not possible to fake an association by sending a request like "http://mysite.com/drupal/?q=node/m&file=sites/default/files/IMAGE.jpg".

If a request for "http://mysite.com/drupal/?q=system/files/IMAGE.jpg" is received instead, Node File Access's hook_file_download takes over, and redirects to "http://mysite.com/drupal/?q=nodefileaccess&file=sites/default/files/IMAG...". Then the above work flow continues from step 3.

The population and maintenance of the association table is done using a host of SQL queries and RegEx based searches that can be defined/modified in the module's settings page. Users can see and change the associations from the individual edit pages of each node.

Comments

subhrajit’s picture

Status: Needs review » Postponed (maintainer needs more info)
Issue tags: -Module review
StatusFileSize
new27.15 KB

Please see comment #13, which contains the latest version of the module.

=============================

subhrajit’s picture

StatusFileSize
new29.55 KB
subhrajit’s picture

StatusFileSize
new29.57 KB

Please ignore the previous two uploads. The latest version of the module is in the attached zip archive.

subhrajit’s picture

Pardon my ignorance, but does the status "postponed (maintainer needs more info)" mean I need to provide more information? If so, what?

subhrajit’s picture

A simpler and to-the-point description of the module in case the above details were too lengthy:
--------------------
Node File Access module associates files (uploaded using various modules) with nodes of Drupal. It maintains those associations dynamically (that is, it keeps updating associations as the use of the files move from one node to another or new files are added or files are removed from a node; resolves conflict of association when multiple nodes use the same file; protects associated files when a node is deleted) and gives access to files based on whether an user have access to the node to which the file associated to. It is also possible to manually search for and alter associations, or explicitly mention not to associate certain files with any node.

avpaderno’s picture

Issue tags: +Module review

@subhrajit: You were supposed to change status after uploading the archive containing the proposed module.

Hello, and thanks for applying for a CVS account. I am adding the review tags, and some volunteers will review the code, pointing out what it needs to be changed.

avpaderno’s picture

Status: Postponed (maintainer needs more info) » Needs review
subhrajit’s picture

Thanks kiamlaluno. I was unaware of that. I am now going through the "CVS Applications issue queue workflows" document.
Will look forward to the reviewers' feedback.

avpaderno’s picture

Status: Needs review » Needs work
  • The points reported in this review are not in order or importance / relevance.
  • Most of the times I report the code that present an issue. In such cases, the same error can be present in other parts of the code; the fact I don't report the same issue more than once doesn't mean the same issue is not present in different places.
  • Not all the reported points are application blockers; some of the points I report are simple suggestions to who applies for a CVS account. For a list of what is considered a blocker for the application approval, see CVS applications review, what to expect. Keep in mind the list is still under construction, and can be changed to adapt it to what has been found out during code review, or to make the list clearer to who applies for a CVS account.
  1. License files cannot be committed in Drupal.org repository. Projects committed in Drupal.org repository have the same license used by Drupal.
  2. See http://drupal.org/coding-standards to understand how a module should be written. In particular, see how the code should be formatted; how Drupal variables, global variables, constants and functions defined from the module should be named; which Unicode functions the code should use.
    Check also the line ending used for the files; the coding standards suggest you use the Unix line ending characters.
    Check you are not using tabs to indent the code.
  3. Drupal variables are not initialized in hook_install().
  4. 	$nfa_association_finder[0]['description'] = "Files uploaded using upload module";
    	$nfa_association_finder[0]['type'] = "filepath";
    	$nfa_association_finder[0]['querystring'] = "SELECT f.filepath AS filepath FROM {files} f, {upload} u WHERE f.fid = u.fid AND u.nid=\$nid";
    
    

    The description is non translated.

  5. 	include_once("node_file_access.lib.inc");
    
    

    include_once() is trying to load the file from the current directory, which is the Drupal root directory, and not the directory where the file is located.
    For such cases, there is a Drupal function that should be used.

  6. Hook implementation comments should be like the following one:
    /**
     * Implements hook_menu().
     */
    
  7. Permissions follow the schema <verb> <object>, where the verb is written in lowercase.
  8. Strings used in the user interface should be translated.
  9. In a settings form, the default value should be taken from Drupal variables, or from a database table; in this way, users would retrieve the value they used the last time.
  10. 		'#description' => t( "separate multiple file paths by newlines"),
    
    

    The description of a form field starts with a word in capital case, and ends with a period.

  11. Why isn't the code using system_settings_form()?
  12. 				drupal_set_message(t('Node File Access Security Alert: The permission to folder '.$this_folder['path'].' is not set properly.'), 'warning');
    
    

    The first argument of t() is a literal string, not a concatenation of strings. The script used to create the translation template is not able to handle any dynamic value, even in the case of code similar to t($variable); this means that if the argument of the function is not a literal string, it will not appear in the translation template.
    Use t()-placeholders.

  13. 	$r = db_result(db_query("SELECT COUNT(*) FROM {nfa_association} WHERE nid=%d AND BINARY file='%s'", $nid, $fileQuery));
    
    

    I think BINARY is not used from all the database engines.

  14. 					$filepathList = eval($this_ass_finder['querystring']);
    					if (is_array($filepathList))
    						foreach ($filepathList as $thisfilepath)
    						{
    							$thisfilepath = _nfa_correct_path($thisfilepath);
    							$allFoundFiles[] = $thisfilepath;
    							_nfa_set_association($thisfilepath, $nid);
    						}
    
    

    If the module is using eval() with users input, then it should also expose a permission like use PHP for <motivation>.

  15. 	$fpath = _nfa_correct_path($fpath);
    	$handle = opendir($fpath);
    	if ($handle)
    		while (FALSE !== ($thisfile = readdir($handle)))
    		{
    			if ($thisfile=="." || $thisfile=="..")
    				continue;
    			$thisfile = $fpath.$thisfile;
    			if (is_file($thisfile))
    				$fileList[] = $thisfile;
    			elseif (is_dir($thisfile))
    				$fileList = array_merge($fileList, _nfa_get_files_from_folder_recursive($thisfile));
    		}
    
    

    Why isn't the code using file_scan_directory()?

  16. 	$form['progress_iframe'] = array(
    				'#value' => "<div id='progress_div' style='border: 1px solid #888888; margin:5px; padding: 5px; font-weight: bold; font-size: 12pt;display:none;'>Building associations. Please wait.
    							<iframe id='progress_iframe' src='http://subhasree.net/drupal/index.php?q=get_progress&onlyval=0' style='width:350px; height:20px; margin-left:10px; overflow:hidden;' marginheight='0' marginwidth='0' frameborder='0' scrolling='no'>
    								<div style='border: 1px solid #888888; background-color: #aaaaaa; margin:5px; padding: 0px; width:350px; height:20px; position:relative;'><div id='nfa_progress_bar_progress' style='border: 0px; background-color: #44ee22; margin:0px; padding: 0px; width:3px; height:20px; position:absolute; left:0px; top:0px;'></div></div>
    							</iframe></div>"
    			);
    
    

    What is the purpose of the iframe?

  17. 		header('Content-type: text/plain');
    		print("".$theProgress);
    
    

    The function to use is drupal_set_header().
    print transform already its argument in a string; there is no need to use "" . $theProgress.

  18. The function _get_nfa_progress() is outputting too much HTML; you should use a theme function.
    There is a Drupal function to add JavaScript code to a page.
subhrajit’s picture

Status: Needs work » Needs review
StatusFileSize
new17.34 KB

hello kiamlaluno. Thanks a lot for the detailed feedback. I have attached the revised version of the module. Below are my responses to the points you have raised.
Thanks!
---

1. Done.

2. Fixed almost all issues. I have used "nfa", which is shorter and easier for readability, as the prefix for function names, instead of the longer "node_file_access".

3. I think I have initialized all the variables using "variable_set" function in hook_install. Is there something else that you are referring to?

4. Fixed.

5. Fixed. I am now using module_load_include('inc', 'node_file_access', 'node_file_access.lib'). Although include_once(''node_file_access.lib.inc') was working on my website.

6. Fixed.

7. Fixed.

8. Fixed.

9. Absolutely makes sense to do that! And I guess I have done that wherever relevant. Of course not for fields like "add new item", where there is no value yet saved by the user. Is there anything particular you are referring to?

10. Fixed.

11. The main reason I am not using "system_settings_form()" is that I would like the form handler to be "nodefileaccess_settings_form_submit()" instead of the default "system_settings_form_submit()". This is because the form is more complex than just saving Drupal variables with names as the names of the input fields. For example, I have blank fields to add new data, I need to process some of the submitted data (e.g. to correct path using the "_nfa_correct_path" function), etc.

12. Fixed.

13. I would like the SQL WHERE clause to do a case-sensitive search. This is important for security reason since in Linux systems 'File.ext' and 'file.ext' can reside in the same folder, and they can be associated to different nodes with different permissions. I guess BINARY keyword is the only way to do that. Is there a better alternative that you can suggest?

14. Fixed. I have now added permission 'allow admin PHP SQL finder'.

15. Fixed. However it seems the function adds some additional overhead that is not quite requited for this purpose.

16. I too was not quite in favor of using the iframe. However it turns out that some browsers (e.g. Chrome) cannot open more than one socket between a server and a single frame/tab/window. Thus, in this example, since the form has been submitted and the browser window/tab is waiting for a response, the same window/tab will not perform any other communication with the same server (the Drupal site) until it gets back a response to the form submission. Thus, the AJAX XML-Http requests querying the progress are silently dropped, and the progress bar won't work. An iframe, on the other hand, can establish an independent socket to fetch the progress information from the server. I agree there may be more elegant and systematic way of solving this issue, but that will make the code unnecessarily complicated just for a simple progress bar. Hope this clarifies the use of the iframe.

17. Fixed.

18. Fixed (first part). I have now used 'drupal_add_js()' to add the Javascript for displaying the progress bar in the settings/contents page themselves.
However the plain progressbar display page (this is the content of the iframe. Just try the page /?q=get_progress, and you'll see what it is.) simply halts Drupal and displays a few lines of HTML and Javascript code just enough to display a green box of length equal to variable_get("nfa_progress", 100) on a grey background (the progress bar). Not much Drupal is going on in this page. So Drupal themes, etc will not take effect even if I declare any. The code is 2-line HTML, and rest is javascript.

subhrajit’s picture

StatusFileSize
new17.34 KB

I am so sorry I had mistakenly hard-coded an absolute path "http://subhasree.net/drupal/index.php?q=get_progress&onlyval=0" from my test web-site in the code. Please find attached the corrected version of the module.

subhrajit’s picture

StatusFileSize
new17.38 KB

A couple of other revisions made (item 2 fixed).

subhrajit’s picture

StatusFileSize
new17.38 KB

I have made a few minor changes/fixes (attached). Hoping to get some feedback soon.

subhrajit’s picture

Status: Postponed (maintainer needs more info) » Needs review
Issue tags: +Module review

hi... just wondering if I am supposed to do something?

avpaderno’s picture

Status: Needs review » Needs work

This is a partial review.

  1. Fixed almost all issues. I have used "nfa", which is shorter and easier for readability, as the prefix for function names, instead of the longer "node_file_access".

    In that case, you should rename the module to nfa.module (and rename the other files too); if the module name is example.module, then example_ must be used as prefix for all the functions, constants, global variables, and Drupal variables it defines. To notice that constant names are written using all uppercase characters.

  2. I think I have initialized all the variables using "variable_set" function in hook_install. Is there something else that you are referring to?

    I meant that a module doesn't initialize its Drupal variables in hook_install(); if you look at Drupal core code, there is no module that does that.

  3. The code format still doesn't follow what reported in Drupal coding standards.
  4.         if (!$res)
              drupal_set_message(t('WARNING: Could not change permission of folder ' . $this_folder['path'] . '.'), 'error');
          }
    
    

    Use t()-placeholders.
    There is no need to prefix the string with 'WARNING'; the type of message is a parameter of drupal_set_message().

  5.       print("<br/>".$path." -  ".$perm."<br/>");
    
    

    Remove any debugging code.

  6.       { drupal_access_denied(); exit(); }
    
    

    Before exit() the code should call module_invoke_all('exit').

  7. function _nfa_force_update_associations_all($forceUpdate = TRUE) {
    
    

    In the coding standards there is also a part about the names to use for PHP variables.

  8.   while ($thisrow = db_fetch_array($result)) {
        _nfa_update_associations(intval($thisrow['nid']), $forceUpdate);
        $count++;
        variable_set("nfa_progress", intval(100*$count/$totalNIDcount));
        //sleep(1);
      }
      for ($nnid=-1; $nnid<=0; $nnid++) {
        _nfa_update_associations($nnid, $forceUpdate);
        $count++;
        variable_set("nfa_progress", intval(100*$count/$totalNIDcount));
        //sleep(1);
      }
    
    

    Is there a reason to use a Drupal variable instead of a PHP variable?

  9.           $querystr = str_replace("\$nid", "" . $nid, $this_ass_finder['querystring']);
              $result = db_query($querystr);
    
    

    db_query() supports placeholders.

  10.     if (variable_get("nfa_auto_remove_unlocked_associations", TRUE))
          $ass_to_remove = array('-1','0');
    
    

    That is a funny name to give to a PHP variable. :-)

  11.     drupal_set_message(t('The path ' . $orgFilepath . ' is not a valid file.'), 'warning');
        return(FALSE);
    
    

    The first argument of t() is a literal string, not a concatenation of strings. The script used to create the translation template is not able to handle any dynamic value, even in the case of code similar to t($variable); this means that if the argument of the function is not a literal string, it will not appear in the translation template.
    return is not a function; the parentheses are not necessary.

  12.         drupal_set_message(t('Incorrect syntax in a Pattern Rule for folder ' . $this_folder . '.'), 'error');
            continue;
    
    

    Sentences used in messages are written in sentence case. (This is a sentence written in sentence case.)

  13. function _nfa_add_js_progressbar (&$form) {
      
      $form['progress_iframe'] = array(
            '#value' => "<div id='progress_div' style='border: 1px solid #888888; margin:5px; padding: 5px; font-weight: bold; font-size: 12pt;display:none;'>Building associations. Please wait.
                  <iframe id='progress_iframe' src='" . url('get_progress', array('query' => 'onlyval=0')) . "' style='width:350px; height:20px; margin-left:10px; overflow:hidden;' marginheight='0' marginwidth='0' frameborder='0' scrolling='no'>
                    <div style='border: 1px solid #888888; background-color: #aaaaaa; margin:5px; padding: 0px; width:350px; height:20px; position:relative;'><div id='nfa_progress_bar_progress' style='border: 0px; background-color: #44ee22; margin:0px; padding: 0px; width:3px; height:20px; position:absolute; left:0px; top:0px;'></div></div>
                  </iframe></div>"
          );
    
      $ProgressBarJS = "
                function SubmitAndShowProgress(theElem) {
                  
                  var branchLevel = 0;
                  var myElems = new Array();
                  myElems[0] = theElem;
                  while (myElems[branchLevel].tagName.toUpperCase() != 'FORM') {
                    branchLevel++;
                    myElems[branchLevel] = myElems[branchLevel-1].parentNode;
                  }
                  theform = myElems[branchLevel];
                  
                  document.getElementById('progress_div').style.display = 'block';
                  var inpElms = theform.getElementsByTagName('input');
                  for (i=0; i<inpElms.length; i++)
                    if (inpElms[i].getAttribute('type').toUpperCase() == 'SUBMIT')
                      inpElms[i].style.display = 'none';
                  
                  // AJAX progress display
                  setTimeout('recursiveUpdateWidth()', 0);
                  
                  // Submit the form
                  if (branchLevel==0) {
                    theform.submit();
                    return false;
                  }
                  else
                    return true;
                }
                
                
                function recursiveUpdateWidth()  // This is used only if the iframe fails
                {
                  if (window.XMLHttpRequest)
                    var oRequest = new XMLHttpRequest();
                  else
                    var oRequest = new ActiveXObject('Microsoft.XMLHTTP');
    
                  oRequest.open('GET','index.php?q=get_progress&onlyval=1',true);
                  
                  oRequest.onreadystatechange = function() { 
                    if(oRequest.readyState == 4) {
                      var progress = parseInt(oRequest.responseText);
                      document.getElementById('nfa_progress_bar_progress').style.width = parseInt(progress*3.5) + 'px';
                      if (isNaN(progress) || progress < 100)
                        setTimeout('recursiveUpdateWidth()', 250);
                    }
                  };
                  oRequest.send(null);
                } ";
                
      drupal_add_js($ProgressBarJS, 'inline');
    
    }
    
    

    The function is outputting HTML without using a theme function.
    The JavaScript code (which is not using jQuery) should be placed in an external file.

  14. Menu descriptions and titles, as well as schema descriptions, are not passed to t().
  15. See http://drupal.org/coding-standards to understand how a module should be written. In particular, see how the code should be formatted; how constant names are written; how Drupal variables, global variables, constants, and functions defined from the module should be named.
    Check also the line ending character used in the files; the module file is not using the correct one.
subhrajit’s picture

Status: Needs work » Postponed

hi... Thanks a lot for the detailed feedback. However it seems I won't have enough time in the recent future to work on this and bring it up to the Drupal standards. I wrote this module in my free time for my site and just thought this may be useful for others, and hence wrote some documentation and wanted to share it. If anybody else wants to take this up and continue, you are absolutely free to do that. Definitely this is under GNU-GPL. Having said that, I am setting the status to "postponed". Sorry about this sudden take-off!

avpaderno’s picture

Status: Postponed » Closed (won't fix)

I am closing this issue due to lack of replies.

avpaderno’s picture

Component: Miscellaneous » new project application
Issue summary: View changes

Please read the following links as this is very important information about CVS applications.