Just for sharing my receipe about a mixed solution with public and private files.
I'm not a Drupal guru ... and still in my learning curve of Drupal ;-) So if you see a "security hole" or a better way (simple way ;-) to achieve the same goal, let me know !

Goal:

I wanted a simple solution to control the access by role to some folder. In my example: a folder specifique to my cck content type

The problem:

- if I set all the file system for private, some fonctionalty don't work (color chooser of garland, performance css, etc ...), and I will overload my server
- only a small part of the files uploaded to my site need to be private (all the css, img etc can stay public)
- I want to control the file access by the internal Drupal Role system.

Pre-requies:

- apache mod_rewrite (if clean url work for you, you're ok)
- cck, cck filefield and a content type of your own

Solution:

For restrict the acces to a folder we will redirect automaticaly all request to /files/private/xxx to /system/files/xxxx and handle the security in Drupal:

Solution step by step:

- set the file system to public. Files are accessible by /files/xxxxxx. But they are also accessible by /system/files/xxxxxx (and this this this trick that we will use ...)
- create a folders in /files/. For example: /files/private/MY_CONTENT_TYPE_FOLDER/
- create the file in the folder /files/private/.htaccess to do the automatic redirect to /system

<IfModule mod_rewrite.c>
 RewriteEngine on
 RewriteBase /system/files/private
 RewriteRule ^(.*)$ $1 [L,R=301]
</IfModule>

Rem: if your install Drupal in a virtual directory for example www.xxx.com/mydir/ , the RewriteBase should be /mydir/system/files/private instead of /system/files/private
- Then, every request to a file in /files/private will be redirect to /system/files/private/. All othe folders are not affected by this setting.
- Create a module, activate it etc .... (perhaps there is a better solution here ... ) containing:

 function MYMODULE_menu($may_cache) {
  $items = array();
  if ($may_cache) {
    $items[] = array('path' => 'system/files/private/MY_CONTENT_TYPE_FOLDER',
      'access' => user_access('access MY_CONTENT_TYPE_FOLDER'),
      'type' =>  MENU_CALLBACK,
  );    
  return $items;
}

- then in the /admin/user/access check the "access MY_CONTENT_TYPE_FOLDER" for the role you want to grant access
- In my CCK content type, I add a filefield and set the folder to private/MY_CONTENT_TYPE_FOLDER. The

Perhaps there is a potential for integrating this principle in the filefield ?

Regards,
Nico.

Comments

Steve Dondley’s picture

I'm going to try this out when I get a chance. Have been looking for a solution for a while. Will let you know how it goes. I am running multi-sites so I'll see if I can add to this solution.

--
Prometheus

px’s picture

I have the same problem, only that I am using organic groups, so introducing a new permission is not an option for me. I need file protection based on node access control. I therefore took the recipe and worked it into a module, that will wrap around CCK File Field and let it perform permission testing (so it does not only work with OG, but pretty much any access control mechanism on the node level). I keep the module in my blog for now:
http://www.onyxbits.de/content/drupal-and-problem-protecting-uploaded-files

Edit: Whoops, wrong "reply to" button. Should have gone to the parent.

mbria’s picture

I haven been looking desperately for a solution for this scenario, as far as I'm building community and group sites with public and private file areas.

My "solution" was a combination of modules:
* webFM for private "intranet" documents...
* and drupal's upload core module for public ones (or "viceversa").

But I'm not proud of this because:

a) You need "private downloads" to secure a little this mess... and "private downloads" will kill your server's performance (http://drupal.org/node/67366), as well as collides with CSS optimizations, "colors" (garland), multisites, video and a quite long list of contrib modules.

b) It's really bizarre for my users to understand that they need to follow different interfaces and upload processes depending on document's privacy.

Last summer a SoC project aims to develop a Document Management Module... but I didn't see much results.
This new SoC project looks also promising, but looks like we need to wait until D7: http://drupal.org/node/166759
Here you can find some people asking for a similar scenario: http://drupal.org/node/126055
Some try to fix it by their own: http://drupal.org/node/116843
Here you can follow an interesting discussions about "public&private files":
http://drupal.org/node/102584
http://drupal.org/node/128876
...

I love to see someday something like Joomla-DocMan running over a Drupal but I don't have the knowledge to develop a huge tool as this... so my only choice is to wait and cross my fingers, sending luck and best wishes to the brave developers that I suppose are working to improve drupal "private & public downloads".

BTW, a "Knowledge Tree" integration module was developed some time ago... but I didn't find the moment to test.

malc_b’s picture

Looks interesting, I may give this a go. However, if the file is being read through php what effect does that have on php's memory requirement? Or is php clever enough to stream the file? Memory is often a limiting factor especially on shared hosting. Pictures are probably ok but video might be a problem.

Tried this but the code seems to be 4.7 not 5 so I think I fixed that. But then I tried a test page with this

<img src="/files/images/group.jpg">
in private
<img src="/system/files/private/images/group.jpg">

The is first image showed, the second didn't. I had no htaccess at this time so if the according to the post both should have pointed to the same place. Files access is set to public. Either I have something wrong or system access isn't working, perhaps because access is set to public? Or perhaps I've misunderstood something.

nico059’s picture

Hi Malcolm,
this solution allow you to set file access to public (for some feature of drupal: Color Picker, aggregate CSS etc ...), and at the same time control the access to some files like the private mode.
You need a .htaccess file with the directive, you need the mod_rewrite activate on your share hosting.

The idea is that any file put in /files is accessible by 2 url:
- directly: Only local images are allowed.
- or control by drupal: Only local images are allowed.

As you specify in your post it could be a bootleneck if it's a big file send via /system because php will "control" the send (and not apache alone)

In your exemple, group.jpg should be on only 1 folder:
- if you want a public acces, put your file in /files/images/group.jpg
- if you want to control access, put your file in /files/private/images/group.jpg

malc_b’s picture

Thanks for the reply.

I can't seem to get system/files to work at all. Leaving aside mod_rewrite (I have deleted .htaccess for now), I have created a page with an img tag where the src = "/files/images/group.jpg" and another where src="/system/files/images/group.jpg". File access is set to public. The first shows the image the second doesn't and this is when logged in as admin. Why doesn't this work? I have none of the code in this idea in at all.

nico059’s picture

What is the result when you try directly in the url of your browser:
www.yourdomainxxxxx.com/system/files/images/group.jpg ?
1 - a clean page "Access denied of drupal"
2 - blank page or other error ....

In the 1 case,:
you don't have the right to access this url. Ti access /system/xxx you should have on of the perm:
- administer site configuration
- access administration pages
- select different theme
But if you say you are admin (user with id=1 full right), this sould not be your case.

In the case 2, perhaps a problem with you host, for you info the code involve when a file is acces by /system/file is (drupal 5)
- file_downlaod line 582 in /includes/file.inc which call file_transfert

Hope that could help you.
Regards,

malc_b’s picture

The result is page not found.

Digging some I find that system/files calls function file_download() not function file_transfer($source, $headers). I am running 5.6 version BTW. In that $filepath gets set to images/group.jpg so that seems right. Next file_create_path($filepath) returns files/images/group.jpg so that's right. The next bit:

  if (file_exists(file_create_path($filepath))) {
    $headers = module_invoke_all('file_download', $filepath);
    print_r($headers);
    if (in_array(-1, $headers)) {
        return drupal_access_denied();
    }
    if (count($headers)) {
        file_transfer($filepath, $headers);
    }
  }

I have stuck in a print_r($headers); which prints Array() which I think means blank array so I don't understand what module_invoke_all is doing. But as $headers is empty then count($headers) is false so file_transfer doesn't fire.

The comments on file_download() suggest that something should respond to say the file is accessible, otherwise it does nothing, which is what it is doing.

headkit’s picture

I can't follow. is this a solution or is this a circle walk?

malc_b’s picture

I was trying the suggestion by nico059. It didn't work for me and I trying to establish what I'm doing wrong or what is different in nico059 drupal. Nico059 says that, for example, /private/files/restricted/myfile.jpg always aliases to /files/restricted/myfile.jpg. But that doesn't happen for me (unless I guess use private files is ticked which defeats the object). And that is where it stands as now.

nico059’s picture

Hi malc_b,
sorry to not be able to help you. To go back to the root of your problem (and forget temporarly my receipe), you seem to have a problem accessing a file with the /system/files/xyz url ... Perhaps try to solve this problem, and then try my receipe for a mixed of private / public file access.

amaria’s picture

I was looking for this exact solution. I tried Private Upload module but it only works with the Upload module. I needed it to work for CCK imagefield and was thinking I would have to do something like this so Thanks!

A couple of things:

1. There is a syntax error in the code you posted. There needs to be a closing brace } right before the line containing return $items; in order to close the if statement.

2. You should probably add SetHandler This_is_a_Drupal_security_line_do_not_remove to the first line of your .htaccess file for security reasons.

Chisholm Technologies, Inc - Custom Software Development since 1999!
http://www.chisholmtech.com

shittii’s picture

Could someone kindly help me to use this method?

Ok. I have the apache mod_rewrite on. Check. I created a directory /files/private/privatetest/ and put the .htaccess there.

Then I put module called 'privatemodule' inside sites/all/modules

The 'privatemodule' contains privatemodule.module and privatemodule.info. The .module contains this snippet:

<?php
function privatemodule_menu($may_cache) {
  $items = array();
  if ($may_cache) {
    $items[] = array('path' => 'system/files/private/privatetest',
      'access' => user_access('access privatetest'),
      'type' =>  MENU_CALLBACK,
  );
 }
  return $items;
}

Ok. Now as you can see I fixed the syntax error code in original nico059's post just as amaria adviced (also added the SetHandler This_is_a_Drupal_security_line_do_not_remove line to the .htaccess)

I can activate the module fine but when I go to the /admin/user/access to find the check the "access privatetest" for the role I want to grant access, it simply is not there! Did I miss something on the way?

amaria’s picture

Access to the private path has to be defined in the hook_perm function of your module. Something like...

function privatemodule_perm() {
  return array('access privatetest');
}

Or you can use the permissions from another module. In my case, I used the permissions of a certain cck content type.

Chisholm Technologies, Inc - Custom Software Development since 1999!
www.chisholmtech.com

Chisholm Technologies, Inc - Custom Software Development since 1999!
http://www.chisholmtech.com

shittii’s picture

Thank you so much! I added this line to my module and it works like a charm through custom CCK filefield! I assume this method does not allow "direct linking" (even with allowed roles), when I upload file to my private directory by FTP. One must always refer to the file by uploading it through CCK?

Whee, it finally works! :')

Edit: Hmm, it seems I am not able to upload big files through CCK field (~200MB) and I tried even to fool the system by uploading a small dummy file and the going to my private directory with FTP and replacing the dummy with the real big file by the same name. Unfortunately this does not work because browser downloads only exactly the amount my dummy file is. I guess the CCK filefield is too smart and writes the file size to the database. Any suggestions?

amaria’s picture

You have to make sure your PHP settings allow you to upload files that big. See http://drupal.org/node/97193

Chisholm Technologies, Inc - Custom Software Development since 1999!
www.chisholmtech.com

Chisholm Technologies, Inc - Custom Software Development since 1999!
http://www.chisholmtech.com

shittii’s picture

Thanks. I have to check that out too. I started wondering if this method really is a memory (and resource) hog as previously discussed, especially when it comes to serving big videofiles like I intend to. I've been checking alternatives (mainly searching these forums) for some time now but this seems to be the easiest solution to control private downloads by role.

Tebb’s picture

Re: "I tried even to fool the system by uploading a small dummy file and the going to my private directory with FTP and replacing the dummy with the real big file"

This is just a guess, based on some reading about how this works.

You might get this 'fooling it method' to work for the larger ftp'd file by correcting the "size" value in the 'files' database table to the actual (ftp'd) file size.

Not sure, but you might also need to change the timestamp field too.

Worth a quick try, but obviously only useful for small numbers of manual changes.

shittii’s picture

Hi. I was wondering if there was any way to refer to a file inside my private directory through CCK, so that I could upload the file by FTP instead of CCK? Or any other way to integrate files to the database of Drupal? I want to download these big private files based on roles. Thanks.

andjules’s picture

I don't know what the limitations of this situation are when you use it with CCK filefield, but it works fine WITHOUT CCK filefield (assuming you fix the brace before return $items and add the hook_perm solution in the comments above).

If you put a file (via FTP) in your /files/private/myfolder/ and try to download it when not logged in, you get redirected to the login. if you are logged in, the file downloads. simple. So, if you don't want to use CCK filefield to upload, you can just use a CCK text field to record the path...

shittii’s picture

I really can't make it work! I've double checked that content access rules are OK, I've tried to obtain the file directly by typing the address of file to the firefox (logged in my site with appropriate user), linking to it through link CCK field and text field with full html turned on. Still nothing. Only uploading by filefield CCK seems to be working correctly.

Maybe there's something fishy in my privatemodule.module code?

I'll paste it here in it's current form:

<?php
function privatemodule_perm() {
  return array('access privatetest');
}
function privatemodule_menu($may_cache) {
  $items = array();
  if ($may_cache) {
    $items[] = array('path' => 'system/files/private/privatetest',
      'access' => user_access('access privatetest'),
      'type' =>  MENU_CALLBACK,
  );
}

  return $items;
}
?>

My private folders have permission 777, so I think it's OK?

shittii’s picture

Anybody with any ideas?

andjules’s picture

I wrote earlier that it shouldn't have anything to do with filefield uploads, I'm not so sure anymore... I thought mine was working, but I was usually logged in as 'superadministrator' (uid=1) and all the files downloaded fine. Now I see that despite permissions set under access-control, authorized users still can't download...
hmmm.

capellic’s picture

I am having the same problem. I can only download as user1 (the original user). Even if I give permissions to all roles, I still get access denied when trying to access the file in any other role. The redirect in .htaccess seems to be working correctly, but there seems to be something wrong with the module. Here's what I have:

function private_files_menu($may_cache) {
  $items = array();
  if ($may_cache) 
  {
		$items[] = array('path' => 'system/files/file_library/cper',
			'access' => user_access('access private_files'),
			'type' =>  MENU_CALLBACK);
  }
  return $items;
}

function private_files_perm() {
  return array('access private_files');
}
svihel’s picture

Same problem here. Only available for user with ID 1. Do someone solved this?

capellic’s picture

Thank you for posting this! Very helpful!

omnyx’s picture

interesting read

davyvdb’s picture

TommyK’s picture

Has anyone used Davy's tutorial to make this work? I'm a newbie when it comes to Drupal, and especially when it comes to creating modules. Does anyone have any tips to help me implement his tutorial?

jarchowk’s picture

A problem I found in applying this was a nagging "The requested page could not be found." error when trying to access a file in the private folder using "/system/files/private/example.doc"

Why? The culprit is in the file_dowload(). The URL is used to find the path of the file (obviously) but finding the file path is used by collecting the func_get_args.

Since we entered /system/files/private as a menu, it is not consider an arg, so the file_download function is looking for "example.doc" instead of "private/example.doc", which it doesn't find.

To solve this I broke a golden rule and hacked the function to add "private/" if its in the REQUEST_URI ... this is temp until I can see a better solution.

This does work though! great work nico.

WorldFallz’s picture

FYI it's not necessary to hack the file_download function if you use the following for your hook_menu:

<?php
function custom_menu() {
  $items['system/files/private'] = array(
    'access arguments' => array('access private downloads'),
    'type' =>  MENU_CALLBACK,
    'page callback' => 'file_download',
    'page arguments' => array('private/'),
  );
  return $items;
}
?>

(this is for d6)

epicflux’s picture

I think this is a great solution for public/private file setup.

I would suggest using hook_file_download instead of creating a menu item. I setup a module like this:


//the files I'm protecting have the url: private/publications

/*
 * hook_file_download, check to see if user has private file access
 */
function my_file_access_file_download($filepath) {
  global $user;
  $file_args = explode('/', $filepath);
  //only work with private files
  if ($file_args[0] != 'private') {
    return;
  }
  //anonymous users don't get to view any private files
  if (!$user->uid) {
    return -1;
  }
  if ($file_args[1] == 'publications' && user_access('download my publications')) {
    $filepath = file_create_path($filepath);
    $result = db_query("SELECT f.* FROM {files} f WHERE f.filepath = '%s'", $filepath);
    if ($file = db_fetch_object($result)) {
      return array(
        'Content-Type: ' . $file->filemime,
        'Content-Length: ' . $file->filesize,
      );
    }
  }
  else {
    return -1;  
  }
}

function my_file_access_perm() {
  return array('download my publications');
}

My db_query is not the greatest, as I'm not checking any node settings, but you could get really creative with Views, or any other module, to have a really intricate file permission system.

I wonder if the CCK File module could write the htaccess file to setup the directory protection?

marcoka’s picture

bookmarking.

marcoka’s picture

no menu? how do you trigger file_download. its triggered by /system/files

WorldFallz’s picture

It's a hook so it's invoked automatically-- see http://api.drupal.org/api/function/hook_file_download and http://api.drupal.org/api/group/hooks for more info.

marcoka’s picture

thank you. do you have a hint on how to handle this if the files im protecting have a deep folderstructure with subdirectorys like /private/foo/bar, private/foo/bar/baz and so on?

And where is $filepath coming from, if you do not use a MENU_CALLBACK that provides arguments?

Thank you

WorldFallz’s picture

1. I don't think the hook cares about how deep the file structure is, as long as the filepath passed to it is correct.

2. sorry, no clue-- the hook system is still somewhat black magic to me, lol.

marcoka’s picture

i mean i try to protect

http://192.168.1.238/WORKSPACE_DRUPAL/testsite_public/sites/default/file...

But the file_download function is not entered (print "foo";) even if i put an .htacces into /privatefiles with


RewriteEngine on
RewriteBase /system/files/privatefiles
RewriteRule ^(.*)$ $1 [L,R=301]

WorldFallz’s picture

hmm... I maintain the download_count module and I use hook_file_download with files protected that way and it works fine. Are you sure you got the rewrite rule correct?

marcoka’s picture

its my fault, i posted the problem and the SOLUTION here: http://drupal.org/node/540754#comment-3091598