After once again having a web site that is 99% public and 1% private, I decided to have another look at finding a true workaround to this issue in Drupal 6. After a day of hacking, I came up with the following.

There are 5 main areas to look at:

1) Settings

A fairly trivial one, hook into the system modules file settings form and append an element for the private directory.

2) Hook into the FileField instance settings

Taking inspiration from the Insert module, I discovered the field settings alter hook. I use this to appends a checkbox field using the form / save operations.

3) Saving the file

Using hook_nodeapi(), insert / update hooks, all content types are checked for filefields. If one is found, and has the flag from step 2 set, the file is moved into the private files directory.

This was accomplished using core Drupal file functions and tricking Drupal into believing that the new private directory was the real file directory. This was done via:

Saving the current settings from the $config global
Updating these using the new directory settings
Moving the file and updating the database {files} table
Restoring the system settings

Notes
- The normal variable_get / variable_set functions would trigger a db save, so any concurrent HTTP requests would get the wrong file path settings. Using $config directly prevents this.
- Restores the settings, as other file calls are very likely, even if you use exit() due to shut-down functions like poor-mans cron.

4) Theming the items

As the new path no longer resides inside the system file directories, all formatters will fail to format the new link. Using hook_nodeapi() again, all node file fields are parsed again to check for the private flag. If found, the file items theming function is changed. The existing theming function is stored for latter use.

In the new custom theme function, the same $config trick is used and the file download method is set to private. Then the original theming function is called to render the element.

The standard private file download will not work as this checks the file directory against the real file directory, (see step 5), so all href and src attributes are parsed to replace the system download link with a custom download link. This URL is based on the node nid and fid for easy security checks during the download.

5) Download

The custom download does a user check on view node permission, and then tries to find the file in the nodes content fields. If found, it runs through the standard private file access checks. A second hook is called to allow modules to interact directly with this method of download, hook_file_private_download(), and has the same API as hook_file_download().

The file_transfer() function required the same $config trick to work.

Issues
Forcing CCK 2.5 or higher requirement - for hook_field_settings_alter() API [check]
Weight not being set right, so the new element is below the submit buttons on the file system settings page.
Bulk updates? Public to private and back again.
Use better logic to compare file directory paths (nested paths could cause issues)

Todo
Code clean up
Implement a more refined permission system
Will normal 'file_download' checks fail or return false results? CVS search may be required. Move the $config change may be required.
See what happens with images [Remove the force download on image files / tag src URLs maybe]

Love to hear feedback from anyone about this. It is still at very early testing stages, but if it passes the tests, I will consider publishing it, if Drupal 7 doesn't come out first!

The complete module is attached below. It will get some good stress test in the next week or two.

Comments

alan d.’s picture

---------------------------- filefield_private.info -----------------------------

; $Id$
name = FileField Private Downloads
description = "Adds private file download options when using FileField."
core = 6.x
package = CCK
dependencies[] = filefield

-------------------------- filefield_private.module ----------------------------

<?php
// $Id$

/**
 * Implementation of hook_widget_settings_alter().
 */
function filefield_private_field_settings_alter(&$settings, $op, $widget) {
  // Only support modules that implement hook_filefield_private_file_widgets().
  $widget_type = isset($widget['widget_type']) ? $widget['widget_type'] : $widget['type'];
  if (!in_array($widget_type, array_keys(filefield_private_file_widgets()))) {
    return;
  }

  // Add our new options to the list of settings to be saved.
  if ($op == 'save') {
    $settings = array_merge($settings, array('filefield_private_downloads'));
  }

  // Add the additional settings to the form.
  if ($op == 'form') {
    $settings['filefield_private_downloads'] = array(
      '#type' => 'checkbox',
      '#title' => t('Private downloads'),
      '#default_value' => (bool) $widget['filefield_private_downloads'],
      '#description' => t('Enables private downloads.'),
    );
  }
}

function filefield_private_file_widgets($reset = FALSE) {
  static $widgets;
  if (!isset($widgets) || $reset) {
    $widgets = module_invoke_all('filefield_private_file_widgets');
  }
  return $widgets;
}

/**
 * Implementation of hook_filefield_private_file_widgets().
 */
function filefield_private_filefield_private_file_widgets() {
  return array(
    'filefield_widget', 'imagefield_widget',
  );
}


function filefield_private_file_fields($reset = FALSE) {
  static $widgets;
  if (!isset($widgets) || $reset) {
    $widgets = module_invoke_all('filefield_private_file_fields');
  }
  return $widgets;
}

/**
 * Implementation of hook_filefield_private_file_fields().
 */
function filefield_private_filefield_private_file_fields() {
  return array(
    'filefield',
  );
}

/**
* Implementation of hook_theme().
*/
function filefield_private_theme($existing, $type, $theme, $path) {
  return array(
    'filefield_private_file' => array('arguments' => array('element' => NULL))
  );
}

/**
 * Theme function for overriden field formatter theme callback.
 */
function theme_filefield_private_file($element) {
  global $conf;
  $system_directory_path = $conf['file_directory_path'];
  $system_file_downloads = $conf['file_downloads'];
  $filefield_private_directory_path = variable_get('filefield_private_directory_path', $system_directory_path);
  $filefield_private_file_downloads = FILE_DOWNLOADS_PRIVATE;

  $conf['file_directory_path'] = $filefield_private_directory_path;
  $conf['file_downloads'] = $filefield_private_file_downloads;

  if (isset($element['#old-theme']) && isset($element['#old-formatter'])) {
    $element['#theme'] = $element['#old-theme'];
    $element['#formatter'] = $element['#old-formatter'];
  }
  // Stop recursion
  if ($element['#formatter'] == 'private_file_download') {
    $element['#formatter'] = 'default';
    $element['#theme'] = 'filefield_formatter_default';
  }
  $output = theme($element['#theme'], $element);
  if (preg_match_all('/(href|src)="([^"]*)'. preg_quote(basename($element['#item']['filepath'])) .'"/', $output, $matches)) {
    foreach ($matches[0] as $index => $link) {
      $private_link = $matches[1][$index] .'="'. url('filefield-private/files/'.
          $element['#node']->nid .'/'.
          $element['#field_name'] .'/'.
          $element['#item']['fid'], array('absolute' => TRUE)) .'"';
      $output = str_replace($matches[0][$index], $private_link, $output);
    }
  }
  else {
    // No match, possible error
    $output = '';
  }
  $conf['file_directory_path'] = $system_directory_path;
  $conf['file_downloads'] = $system_file_downloads;

  return $output;
}


/**
* Implementation of hook_menu().
*/
function filefield_private_menu() {
  $items = array();
  $items['filefield-private/files/%node/%/%'] = array(
    'title' => 'File download',
    'page callback' => 'filefield_private_download',
    'page arguments' => array(2, 3, 4),
    'access callback' => TRUE,
    'type' => MENU_CALLBACK,
  );
  return $items;
}

function filefield_private_form_alter(&$form, $from_state, $form_id) {
  switch ($form_id) {
    case 'system_file_system_settings':
      $form['filefield_private_directory_path'] = array(
        '#type' => 'textfield',
        '#title' => t('Custom directory for user application files'),
        '#default_value' => variable_get('filefield_private_directory_path', file_directory_path()),
        '#maxlength' => 255,
        '#description' => t('A file system path where the files will be stored if the %type option above is selected. This directory must exist and be writable by Drupal. The download method will be treated as private.',
            array('%type' => t('Private downloads for user application files?'))),
        '#after_build' => array('system_check_directory'),
      );
  }
}

/**
 * Implements hook_nodeapi().
 */
function filefield_private_nodeapi(&$node, $op, $a3 = NULL, $a4 = NULL) {
  if ($op == 'view') {
    $content_type = content_types($node->type);
    $file_fields = filefield_private_file_fields();
    foreach ($content_type['fields'] as $field) {
      if (!empty($field['filefield_private_downloads']) && in_array($field['type'], $file_fields) && isset($node->content[$field['field_name']])
        && is_array($node->content[$field['field_name']])) {
        $instance = $node->content[$field['field_name']];
        foreach ($node->content[$field['field_name']]['field']['items'] as $delta => $item) {
          $item['#old-theme'] = $item['#theme'];
          $item['#old-formatter'] = $item['#formatter'];
          $item['#theme'] = 'filefield_private_file';
          $item['#formatter'] = NULL;
          $node->content[$field['field_name']]['field']['items'][$delta] = $item;
        }
      }
    }
  }
  elseif ($op == 'insert' || $op == 'update') {
    filefield_private_move_files($node);
  }
}

function filefield_private_move_files(&$node) {
  global $conf;
  if (is_object($node)) {
    $files = array();

    $system_directory_path = $conf['file_directory_path'];
    $real_system_directory_path = realpath($system_directory_path);

    $filefield_private_directory_path = variable_get('filefield_private_directory_path', $system_directory_path);
    $real_filefield_private_directory_path = realpath($filefield_private_directory_path);

    $content_type = content_types($node->type);
    $file_fields = filefield_private_file_fields();

    foreach ($content_type['fields'] as $field) {
      if (!empty($field['filefield_private_downloads']) && in_array($field['type'], $file_fields) && isset($node->$field['field_name']) && is_array($node->$field['field_name'])) {
        foreach ($node->$field['field_name'] as $file) {
          if (file_exists($file['filepath'])) {
            $file['realpath'] = realpath($file['filepath']);
            // If within the application path, ignore the file
            if (strpos($file['realpath'], $real_filefield_private_directory_path) !== 0) {
              $file['relative_path'] = ltrim(substr($file['realpath'], strlen($real_system_directory_path)), '\\/');
              $file['dest'] = rtrim($filefield_private_directory_path, '\\/') . DIRECTORY_SEPARATOR . $file['relative_path'];
              $files [$file['fid']] = $file;
            }
          }
        }
      }
    }
    $conf['file_directory_path'] = $filefield_private_directory_path;
    foreach ($files as $file) {
      $src = $file['filepath'];
      if (file_move($src, dirname($file['dest']), FILE_EXISTS_RENAME)) {
        // update db
        db_query("UPDATE {files} SET filepath = '%s' WHERE filepath = '%s'", $src, $file['filepath']);
      }
    }
    $conf['file_directory_path'] = $system_directory_path;
  }
}

/**
 * Call modules that implement hook_file_download() to find out if a file is
 * accessible and what headers it should be transferred with. If a module
 * returns -1 drupal_access_denied() will be returned. If one or more modules
 * returned headers the download will start with the returned headers. If no
 * modules respond drupal_not_found() will be returned.
 */
function filefield_private_download($node, $field_name, $fid) {
  global $conf;
  if (!node_access('view', $node)) {
    return drupal_access_denied();
  }
  $filepath = '';
  if (!empty($node->$field_name)) {
    foreach ($node->$field_name as $file) {
      if (file_exists($file['filepath'])) {
        $filepath = $file['filepath'];
        break;
      }
    }
  }
  if (file_exists($filepath)) {
    $headers = module_invoke_all('file_download', $filepath);
    $headers = array_merge($headers, module_invoke_all('file_private_download', $filepath));
    if (in_array(-1, $headers)) {
      return drupal_access_denied();
    }
    if (count($headers)) {
      $filefield_private_directory_path = variable_get('filefield_private_directory_path', $system_directory_path);
      $system_directory_path = $conf['file_directory_path'];
      $conf['file_directory_path'] = $filefield_private_directory_path;
      file_transfer($filepath, $headers);
      $conf['file_directory_path'] = $system_directory_path;
    }
  }
  return drupal_not_found();
}

function filefield_private_file_private_download ($filepath) {
  return array(
    'Content-Disposition: attachment; filename="'. basename($filepath) .'"'
  );
}

?>

Alan Davison
joachim’s picture

Interesting -- will give it a whirl soon.

You should create a project for this!

alan d.’s picture

I was hoping to get the time to develop and test this more over the last few weeks, but a higher priority project has come up at work and it has been absorbing all of my time. Hoping to get back to this in a week or two.

For the meantime I've committed the code here:

http://drupalcode.org/viewvc/drupal/contributions/modules/filefield_priv...


Alan Davison
WorldFallz’s picture

just an fyi (i use the method described at http://drupal.org/node/189239 for d6) -- this is now built in to core for d7 (filefield replaces upload).

alan d.’s picture

That is how we have done it in the past. We have some clients that do not have .htaccess permissions and other clients on window boxes, it is all so painful! I think that is how private_downloads sort of works too. (I have not used that module)

The last company that I worked on had a stressed shared server, and it had many attacks, mainly ssl dos attacks against the shops / betting sites. For whatever reason, apache occasionally lost its access control and started listings directories. (We had to script regular restarts to minimise the risks)

So I have always used the .htaccess method on sites where it would not be the end of the world if this happened, but used true Drupal private downloads if the info was really commercially sensitive and just accepted the server overhead.

This also has the benefit of controlling FTP access if required :)


Alan Davison
izmeez’s picture

@alanD. Thanks for posting this very useful information.

I do not have the coding experience and even after two years of using Drupal feel I am just a novice, learning more every day.

I have some existing sites that have been using the public method and I would like to move them to private download. I have been trying to understand the best way to do this and tried reading everything I can find on this. Some of the stuff I've found is a bit complicated.

I would like to map out the simplest way to make the change public to private and then extend it as needed but ensure the first steps are the best to avoid problems later.

I don't want to hijack this thread and should maybe just listen in a bit.

Some of the things I have come across that look interesting are:
make private download method support css/js aggregation, color module and js translations
and the x-send file module http://drupal.org/project/xsend
and I have been trying to understand the direction D7 is going in, through the "recipe for mixed private/public"
I haven't done a lot with filefield_paths module and maybe need to get a better handle on that.
But, I'm really hoping to find a method that isn't complicated for the users of the site and performs well for both anonymous and authenticated users while securing the files that only authenticated users need access to.

Thanks very much for your post. I will follow with interest.

Izzy

alan d.’s picture

We've released it as an "alpha" module here http://drupal.org/projects/filefield_private, but it is an alpha!! Use with care. But there are a few links to the .htaccess pages that you may find useful

Switching to private from public requires a lot of work, this includes the db etc.


Alan Davison
sinasalek’s picture

Thanks @alan ,
PS : the link does not exists.

sina.salek.ws, Software Manager & Lead developer
Feel freedom with open source softwares

alan d.’s picture

sinasalek’s picture

Thanks,

sina.salek.ws, Software Manager & Lead developer
Feel freedom with open source softwares

marcoka’s picture

this is nice. i am thinking about making a new cck filefield that uses the path settings from the settingspage (add a private text input field like d7). The only problem i have is the same as you hat but solved with manipulationg global $conf.

edvanleeuwen’s picture

Hi Alan,

Just stumbled upon this post when trying to mix files with FileField. In D6 I tried creating a different architecture (using private_upload and uploadpath) in which a private subdirectory (with .htaccess written into it) was created when files had to be private. And the URL was rewritten (using the 'system' addition) when it pointed to a private file.

This way the file system could still be set to public and you would have all the attachments kept together, whether they are public or not. Further advantages: force files private when the corresponding OG is private, do not display private files entries in nodes when you do not have permission for the files themselves.

If you are interested, see http://drupal.org/node/197590#comment-1176337.

Regards, Ed

Best regards,

Ed