Use of sites/*/files as the default files directory

+1. I always change the setting to sites/[sitename]/files when setting up a multisite install. This will mean 1 less step in setting up Drupal.

+1 for the idea, but shouldn't the folder be moved aswell? I mean doens't the /files-dir come bundeled in the tarball? And what about the installer?

I'm not sure about the things I'm saying, but I think it's better to be sure than quiet.

forngren, no the files dir needs to be manually created after you open the tarball so people put it wherever they want with whatever name they want.

+1 to change the default files dir path since I always have to change it. I think making it default will also move people who don't know about using that path towards better practices.

I would support this if there is a way for Drupal to add an .htaccess file that would make the directory private when file access is set to private.

Re: #6:: Should that be another feature request? If you did want to piggyback it onto this one, you could always have PHP write a .htaccess file when it is set to private, then delete it when it's set to public. But you may need a new on/off switch for that in case someone wants to add their own .htaccess in the directory.

Back on topic: +1 for the idea. But since it's a feature, don't you think it may be pushed to the 6.x-dev tree?

This doesn't seem major enough to keep out of 5.x, except that there might be corner cases on updates (or not? I'm not sure). Anyway, updating to 5.x-dev. Feel free to push it to 6.x if I'm off the mark.

Updated patch. I would like to give this a go, but since I submitted the last patch I do not dare (nor should I).

Patch works for me as advertised and I think it would be a welcome addition. +1

I think this is a good idea (although it makes for some ugly/long paths). Feel free to RTBC if most people agree with this change.

Thanks for the review and support Dries!

Here an updated patch, there was a oneline offset in the other one.

(still no RTBC from me, add1sun?)

What happens if I have a Drupal install, manually created the top-level "files" directory, but never actually went to admin/settings/files and clicked "Save" to set the variable? Suddenly all my pre-existing file paths are broken, no..?

I think this needs an upgrade path of some kind, or else a way to be smart and not change the files path if one already exists. I'm kind of stuck as to how that would work though.

First attempt to fix an upgrade path.

/**
 * file_directory_path now defaults to conf_path().'/files'
 *  instead of just 'files'', something more clever is needed here
 */
function system_update_2001() {
  if (empty(variable_get('file_directory_path', '')) && is_dir('files')) {
    variable_det('file_directory_path', 'files');
  }
}

Stupid typo. I also take liberty to steal this issue.

+1 for this patch. Logical thinking. I like it.

I see two issues with the current patch:

1) The update.php is broken, my system_install_2001() can't seem to return a correct value and throws an "Fatal error: Can't use function return value in write context in /modules/system/system.install on line 3551"

/**
 * file_directory_path now defaults to conf_path().'/files'
 *  instead of just 'files'', something more clever is needed here
 */
function system_update_2001() {
  if (empty(variable_get('file_directory_path', '')) && is_dir('files')) {
    variable_set('file_directory_path', 'files');
  }
  
  // An array needs to be returned
  return array();
}

2) The way to check if files dir exists is not very good. E.g:

Site A and B are multisites on the same codebase.

Site A has created 'files' dir and stores files in it.
Site B has never touched file handling, and therefor has no 'files_directory_path'. Site B will anyway get 'files' as path, since it exists.

I hope you understands what I'm saying, I'm really tired atm.

subscribing to this issue (related problem was a duplicate: http://drupal.org/node/116151)

I think I got something almost bulletproof.

The voodoo is the query SELECT fid FROM {files} WHERE filepath LIKE 'files/%' LIMIT 1

This snippet needs however to be turned into something useful before patch can dodge bullets.

if (empty(variable_get('file_directory_path', '')) && is_dir('files') && count(db_fetch_object(db_query("SELECT fid FROM {files} WHERE filepath LIKE 'files/%' LIMIT 1")))) {

This looks like it will lead to automatically moving files.
-1 for doing anything automatically with existing files.

It will NOT move any existing files. The update script will try to determine if the folder can safely be changed or not.

Updated patch, please review.

Thanks to Matt Farina.

I can't get this to work with a clean install of the latest HEAD - / $Id: system.install,v 1.94 2007/04/17 08:13:11 dries Exp $
:(

Applies clean but then WSOD on install.php and this error on update.php:
Fatal error: Can't use function return value in write context in /Users/addi/Sites/drupalhead/modules/system/system.install on line 3744

Also, while you're fixing that, there are some minor whitespace issues in the patch:

conf_path().'/files' => conf_path() .'/files'

$paths['color'] = file_directory_path() . '/color'; => $paths['color'] = file_directory_path() .'/color';

See Coder module for an automated way to check for these.

Thanks for the reviews!

I found the error, but I don't how to fix it. variable_* functions are not available during the install, and therefor does it fail. So sql substitutes for variable_get/set is needed.

For someone who is somewhat familiar with db_rewrite_sql this should piece a cake. If there's documentation available for the drupal sql syntax I might be able to do it myself.

Updated patch that MAY/COULD/SHOULD work. I haven't really tested it. If you have some spare time, please look ever the sql queries and see if they makes any sense at all.

Please don't hit me, but now that http://drupal.org/node/141637 is in, maybe a better approach to this patch is to add a new field to that form which defaults to sites/domainname.com/files and sets up your file directory on submit?

/me cheers!

Sounds like a good idea. I think the upgrade part is still a good thing, but if we collect files folder during install and user then delets/resets it, we're facing the same situation again. Comments?

i just want to say i'm all for this change. it'd cut down on a ton of support requests for contrib modules that use files.

I think files config at install should be another issue. Lets try to get this in first.

The db stuff needs to be reviewed by someone far more experienced than I am. Drewish? Webchick?

From visual inspection you're missing a closing parentheses in your update_sql() line - please do a basic test of a patch before posting it.

In other news, the update number needs to be 6015 now. That IF statement could use a comment as well.

Good catch, I did only test on a clean install, which explains why the code never was triggered. I'll update the patch tonight

Sounds good. I also just noticed that there are spacing errors in the string concatenations in the other two hunks.

Does

function system_update_6014() {
  variable_set('install_task', 'done');

  return array();
}

really work? When I used variable_* I always broke something during install.

That code isn't run during an install, it's run during an upgrade to set the variable that is now set after installation is complete. That part was added to the recent install patch at http://drupal.org/node/141637#comment-241562 if you're interested.

Updated and untested patch.

I simply can't get this one to work.

empty(variable_get('file_directory_path', ''))

does throw an error (can't use function return value IIRC), while

db_fetch_object(db_query("SELECT COUNT(*) FROM {variable} v WHERE v.name='file_directory_path'")) == 0

doesn't. Also, when I used the latter chuck, the update went smooth, but file_directory_path was never set. I need serious help on this one.

ChrisKennedy? ;) Updated patch in exchange for a couple of reviews?

AFAICS we don't need any system update at all here.

- Anyone who has uploaded files already, will by definition have the variable set - since file_directory_path() must have been called as part of the file upload process (if not before). Hence, they continue using the old path.
- If they haven't uploaded files already, and haven't visited the files settings page to cause the variable to be set, then they should be all set to migrate to the new path.

As long as we only use variable_get() for the new default we are on safe ground here (assuming I didn't screw up my logic...it's getting late here!).

Here is a patch to this effect.

@Grugnog2 : That won't work since file_directory_path() will never set the variable 'file_directory_path'.

Here's a re-roll based on #38 - needs testing.

@forngren - of course it will - please read up on the second parameter of variable_get() ;)

If a file has been uploaded then file_directory_path() must have been called.
If file_directory_path() is called then variable_get() does already set the variable 'file_directory_path' to the default value of 'files' (the second parameter) - as listed below:

/**
 * Determine the default 'files' directory.
 *
 * @return A string containing the path to Drupal's 'files' directory.
 */
function file_directory_path() {
  return variable_get('file_directory_path', 'files');
}

As I said above, there is absolutely no need for an update in this patch.

Eh, variable_get() does not set anything. The default is only for that particular call to variable_get(): http://api.drupal.org/api/HEAD/function/variable_get

Duh ;)

Updated/untested patch. Thanks for the push ChrisKennedy!

And patch worked like a charm. However, the installer 6.x was just plain broken, so I could not test a clean 6.x install. The patch shouldn't affect the install anyway. I'm tempted to RTBC this, but since my hands were the latest on the code I think it's a bad idea. If someone else can verify my results it should be RTBC.

patch is working for me on update and install just fine. let's get it in.

it may not be a huge deal but that query should have a %% for a wildcard. i also make a small change to the way it was called. looks good and works correctly for me.

It needed an extra linebreak between the new update function and 6022. While I was at it I took out the column aliasing ("AS count") since we're using db_result() now.

Keeping with HEAD

Keeping with fast moving HEAD.

- Patch applied cleanly.

- I uploaded and deleted files, both with this new default files directory and with a custom one. Everything worked flawlessly.

Confirmed that this patch is RTBC.

Jumping on the next train.

Updated/untested patch just in case ;)

It's a shame we can't get this patch in for D6! I tested it a month ago and it worked perfect. Any chance of pushing this as a critical file system improvement for D6?

Related issue; http://drupal.org/node/166169

I don't think it's dupe, but that could be arguable. I don't fully understand what the patch here is actually supposed to fix...

subscriping

Get this thing in dudes ;)

Free issue give-away ;)

Sorry that I noticed this issue so late (not that it would matter much if it did earlier), but I have to completely disagree with this change, especially for multisites. Putting the "files" directory in "sites/example.com/files" is not just ugly for the links, it will often break the links.

Besides offering no benefit, It breaks all the file/image links in the content if you ever change domain, or even if you maintain a local dev installation, because it insets the domain name in the path, creating a dependency.

The only acceptable approach I can think is to use site aliases which you can keep when moving between domain, but even this can't be done in the "sites" directory because that need the real domains.

A workaround which has been suggested is to use "/files/site-alias" symlinks leading to "/sites/example.com/files", so that the image paths won't break when changing domains. This means that you still have to use "/files/site-alias" in the /admin/settings/file-system page, and then set up aliases and move the files manually wherever you want. I don't expect that this can be done reliable by an installation script.

Since "support requests" were mentioned in this issue, I have to point out also the many support requests for broken links.

cog.rusty: Surely if you change domains all the external links to your files are going to be broken anyway?

If you're going to be using mod_rewrite to redirect to the new domain, it should be simple to rewrite sites/old.domain.com/files links to use sites/new.domain.com/files at the same time.

Multisite set-ups would be the main beneficiaries of this change. Currently, every site uses the files directory by default. This patch has been delayed too long.

I am not talking about SEO and Google ratings. I am talking about links in the content such as:

<src="/sites/example1.com/files/image.jpg">

These will definitely break if you copy your site to http://localhost.site1 to do some development, or of course to http://example2.com. There would be a dependency from the domain name hardcoded into the content. Currently, moving a site anywhere you want is a breeze, by using /files/site-alias directories.

So, multisites would be victims rather than beneficiaries.

In an older discussion Boris Mann suggested the symlinks method, using /files/site-alias as a symling to /sites/example.com/files, but this (a) also requires to use /files/site-alias in Drupal's file settings and (b) it is hardly feasible automatically by an installation script (because of the symlinks).

For single sites using "sites/default". the trade off is just "one-place backup" vs "ugly links"
For multisites it is an unnecessary problem which you have to solve each time, with sole benefit a "one-place backup" for each site instead of a "two-places backup".

cog.rusty: That does make sense, but it still doesn't solve the original problem. How would you automatically set the name of the site's folder in the 'files' folder, and make sure it's unique, without using the domain? For example, if the domain is site.example.com, what would you name the folder at 'files/something'?

True, this can't be automated currently. The multisite user has to go to the /admin/settings/file-system page and set files/[a-suitable-alias-for-the-site]. But it still seems easier than the alternative.

I guess the idea of a formal "site alias", reusable and independent from the domain name, is worth investigating.

You could do a unique hash for the directory name, ala color.module.

The latest patch does not include all code from previous ones. Also by looking at the issue starter, I seem to suck too. I think most Drupal installs are single site and I don't think they deserve ugly file paths. Especially if they need to modify their site content as they move from staging to production and/or move domains or whatever. (Jumped in from http://drupal.org/node/190283 where we realized we need the files directory precreated for the installer).

Having had to convert a series of single-site installs into one big multi-site install recently (for a client who panics at the thought of having to think, but doesn't give us access to their server so they have to do things), I'm beginning to lean away from using sites/default/files as well. The primary issue is that the files table in the database stores the full path from Drupal root. That means any change in the domain you're on means the files table has to be manually edited if you're putting your files directory in sites/example.com/files.

What I'm likely going to do for our in-house D6 install profile is use /files/sitekey/ as the file root, where sitekey is some non-domain-based (and therefore portable) site-specific string. That won't change when the site moves, but will still keep a good separation between files from different sites. Some automated way of setting that up in the installer would be useful.

There maybe more single site drupal installs, but I think most large sites and multi-person developer teams use multi-site configurations. In any case, multi-site configurations are an important feature that isn't going to go away because a few people don't use it.

As for the issue where modules store the sites/[site-name]/files part of the path; that is a problem with contrib modules and/or the file api. I tried to fix this in 166169, but that patch doesn't solve the real underlying problem. I think there are other issue nodes about that bug on d.o but I can't find them right now.

THIS issue however is about setting sites/[site-name]/files as the *default* file directory. I think that's great, but as others have already pointed out, it might not go down well with the ones that generally use a single-site config. In any case it's separate issue to 166169.

Subscribing and voicing support for this change. Given multisite's usefulness, the old /files directory setup is brain dead. ALL files specific to a given site need to be within sites/*/files.

Aren: I used to agree with you entirely, until I started using multisite. See my comment in #67 as to why I am now -1-ing sites/default/files. It's not portable between servers. I will be moving away from sites/default/files for all of my Drupal 6 sites, and instead to /files/sitekey/.

Crell:

The primary issue is that the files table in the database stores the full path from Drupal root. That means any change in the domain you're on means the files table has to be manually edited if you're putting your files directory in sites/example.com/files.

What if the Drupal core developers could fix this? This shouldn't be a show stopper.

I, too, am using /files/sitekey in the interim, but I only see that as a workaround. To maximize portability, ease of maintenance, and minimize administrator error, all files specific to a site should be contained within a single directory structure specific to the site.

Don't expect a "fix" for this. What you suggest would require the files path to change every time the base URL changes. Since some sites use symlinks to make themselves accessible from multiple URLs, the base URL could change with every page request. The only efficient way to make this work would be to hard-code the files path so that the site-specific portion could be changed on the fly.

I think this is duplicate of http://drupal.org/node/191310. I haven't read that whole thread, so please un-duplicate this if I'm wrong.

No, not a duplicate. It is quite valid to discuss this for 7.x. The other issue solved an immediate issue for 6.x.

Don't expect a "fix" for this. What you suggest would require the files path to change every time the base URL changes. Since some sites use symlinks to make themselves accessible from multiple URLs, the base URL could change with every page request. The only efficient way to make this work would be to hard-code the files path so that the site-specific portion could be changed on the fly.

But wouldn't this improvement eliminate the need for the symlinks in the first place?

No. I was describing symlinks in the sites directory that are used to duplicate a site at a different URL.

I hope that preserving that exotic functionality would be lower priority than a fundamental usability/maintainability issue like this.

Aren - I don't think symlinking the sites directory (or inside it) is 'exotic' functionality - every single Drupal site I have worked on in the last 3 years has used this :)

That said, I don't see symlinks as a problem - moving to a sites/sitename/files setup (better than files/sitekey, I agree) does not break symlinks at all - it just means that if you want to offer different domains, each with *different* files directory you need to configure each one or add a line of code to settings.php. If you want the *same* files directory then no problem as the files part of sites/sitename/files can just be a symlink to a shared files directory above the docroot.

The only problem here is that there is no way this will get into D6. Please help push this early in D7 though, and I am sure we can get it in!

Color has already been fixed to use file_directory_path() so these are the remaining updates, rerolled to HEAD. This can't make it in for Drupal 6?

In Drupal 6, creation of the file path is part of the installation process. Also what to modify the default files directory to is still under debate here as far as I have seen.

Grugnog2, symlinks are not a viable option for many people. As long as it's not mandatory, options are generally good.

@sepeck good point, but the actual location of the files directory can always be overridden per-site with $conf['file_directory_path'] or at admin > settings > file system.

I think the issues comes down to; 'where do most people put their files dir' and 'what's the best location for it for new sites and drupal newbies'.

My thoughts are that all site specifics go into the sites directory, so an upgrade of Drupal just requires the replacement of all files, excluding that in the sites directory. Also, having the default files directory share the files of all other installed multi-site systems, can result in some security issues.

@sepeck - I wasn't actually suggesting symlinks as a solution for anything, but trying to say that they are a valid approach and whatever we do should support them, but also that I think the sites/sitename/files approach works fine with symlinks, so I don't think we have a problem here. (Darren Oh suggested that the sites/sitename/files approach would break with symlinks, which is what triggered this very confusing sub-thread)...

Concerning symlinks, they are probably only used by the advanced users who are most able to adapt to changes.

Therefore, if moving all site-specific files to the sites/... hierarchy makes most sense for the broader community, it should be done even if it temporarily inconveniences advanced users.

Go for it!

By the way, since Drupal 6 is still in beta (not RC), and since this fixes a major oversight in file handling, and since patches are above that would fix this in Drupal 6, are we sure this shouldn't make it into Drupal 6? I.e., go ahead and get it over with now?

Inexperienced people are just as likely to move sites, and as explained above, this makes problems for them.

What I said earlier about the base URL was incorrect for two reasons:

1. File URLs are relative, not absolute.
2. Even if they were absolute, they would work as long as they matched a valid domain. They wouldn't have to match the current domain in order to work.

I'm surprised no one caught this.

I understand how annoying it is to have Drupal create a directory I'm not going to use when I visit the file system settings page for the first time. But if future versions will allow the path to be chosen during installation, this is no longer a problem.

This was conceived as a stupidity-prevention patch, but the previous default really wasn't stupid. Given that we will now be able to configure the files directory during installation, I would support anyone who wants to set this issue to "by design".

What I said earlier about the base URL was incorrect for two reasons:

1. File URLs are relative, not absolute.
2. Even if they were absolute, they would work as long as they matched a
   valid domain. They wouldn't have to match the current domain in order to
   work.

I'm surprised no one caught this.

I understand how annoying it is to have Drupal create a directory I'm not going to use when I visit the file system settings page for the first time. But if future versions will allow the path to be chosen during installation, this is no longer a problem.

This was conceived as a stupidity-prevention patch, but the previous default really wasn't stupid. Given that we will now be able to configure the files directory during installation, I would support anyone who wanted to set this issue to "by design".

-1 for the proposal.

My multisites are using files/site-alias for the files for the same reasons given by cog.rusty and crell in #62, #67. A more complicated localhost setup (additional entries in hosts file, httpd.conf edits) can avoid some of the issues related to site transfers from a test localhost site to a production site. But I still reckon files/site-alias is a less ugly URL than sites/clienthasalongdomainname.com/files and has increased portability such as going from testing.example.com to www.example.com

The 'two places to backup' downside of files/site-alias is a very low overhead to obtain the benefits of cleaner test and review cycles.

There is actually one "think outside the box"/hack solution.

Why not create the drupal url 'files' which links to the files directory. It could either read the content and then transmit them or do a http redirect. This is portable whit old urls, is clean url friendly and does not expose settings dir.

The catch is that a php script must be executed in order to download a file. This could be solved with a non db minimum bootstrap or a full bootstrap process which would make file access control easy.

example.com/files/file.ext
example.com/files/dir/ectory/structure/file.ext
example.com/?q=files/dir/ectory/structure/file.ext

Thoughts?

(btw, I'm willing to accept this task as a G-HOP-task ;)

This could also be solved in .htaccess with a rewrite-rule.

forngren: What you're proposing is essentially what the private download method does now.

The underlying problem is that the files table in SQL stores the full path of the file from Drupal root, regardless of whether you use public or private downloads. If it stored paths relative to the specified files directory, then all of the portability problems would be solved. As is, though, the raw data itself in the database is locked to a domain if you use the sites/domain/files route. Until that issue is resolved, any sort of "dynamic placement" solution is an ugly hack. That definitely won't happen in D6. The only question is how we mitigate it with a recommended best practice for the initial placement of the files directory.

This is a reply to vjordan at #90....

My multisites are using files/site-alias for the files for the same reasons given by cog.rusty and crell in #62, #67.

Your setup is basically the same as the proposed solution, except it would be active by default. Every site's file system would be made unique by default and wouldn't require setting up to make them such. This means that when you make a new site, you wouldn't have to set the file system location to "files/site-alias", as it would be set to "sites/site-alias/files" by default. This produces same exact solution you have, except it's implemented by default.

A more complicated localhost setup (additional entries in hosts file, httpd.conf edits) can avoid some of the issues related to site transfers from a test localhost site to a production site. But I still reckon files/site-alias is a less ugly URL than sites/clienthasalongdomainname.com/files and has increased portability such as going from testing.example.com to www.example.com

If you enable Clean URLs and the Private File System, all file downloads are cleanly handled through system.

Reply to #91.....

Why not create the drupal url 'files' which links to the files directory. It could either read the content and then transmit them or do a http redirect. This is portable whit old urls, is clean url friendly and does not expose settings dir.

Turn on Clean URLs and the private file system. I believe this is what you're talking about...

#87 @Gábor Hojtsy - agreed. I know I learned a bit about SQL tables from figuring out how to update them manually when I moved the /files directory on my sites. I have always wondered (but not acted on) if it was possible to dynamically update the pointer for existing file locations when you change them in the GUI.

@#90

A more complicated localhost setup (additional entries in hosts file, httpd.conf edits) can avoid some of the issues related to site transfers from a test localhost site to a production site.

I don't think we should compromise drupal core, default settings or drupal's recommended practices for a preferred sandbox configuration. However you're idea of handing of requests to files in a sandbox to the production site is excellent and something I'd like to try out.

I propose these changes;

• Make (conf_path() . '/files') the default file_directory_path. (which is sites/default/files for single-site configs, and sites/[DOMAIN]/files for multi-site configs.)
• Require that all paths in the files table are relative to file_directory_path. (IMO This should have been the policy anyway, and has resulted in ugly work around patches like http://drupal.org/node/166169 and a generally ugly file API). Enforce this by not supporting paths relative to the drupal root in the file API, and requiring modules to change adhere when they port to drupal 6/7.
• Support paths relative to drupal root, and absolute URLs in the user-configured drupal variable file_directory_path. (This makes it easy to do the suggestion in #90 and configure another web server to server files to better distribute server load. It also allows an admin to configure a family of sites to share a files directory.)

I think those three changes cover all the use-cases mentioned in this thread. Have I missed or misunderstood any?

I just skimmed through the whole thread checking for anything I have missed or forgotten and found a few things, but I don't think any show stoppers;

#14; upgrade path for d5 sites that never saved 'file system settings' and don't have file_directory_path set.
I think this was addressed in the patch. Not sure if it was solved.

#6; dynamically making files inaccessible for 'private files access' with .htaccess.
Can we, and do we need to create or modify an .htaccess file when file access is set to 'private'?

#59 It breaks all the file/image links in the content if you ever change domain, or even if you maintain a local dev installation, because it insets the domain name in the path, creating a dependency.

This is about links to files in the body of nodes. This is a bad but common practice. If you are deprecating a domain to your site without a 30X redirect or a symlink from the old files or sites directory to the new one, then you should be prepared to deal with the consequences! :) One SQL query using replace() can fix this too.

#66 I think most Drupal installs are single site and I don't think they deserve ugly file paths.

Gabor has a valid point here. Although this means that making a single-site into a multi requires either foresight, or some tweaking, which is rather annoying for the non-advanced user or drupal newbie.

I think #67 and #70 is an exceptional case and is handled by user-defined files directory locations, but I'm afraid I've misunderstood Crell's motivations for opting for files/[sitekey]?

Bevan writes in #98:

This is about links to files in the body of nodes. This is a bad but common practice.

How else are you supposed to do it?

Ideally with a filter, something like "[file:some/dir/and/file.extension]". Although I know of no such filter. There's certainly not one in drupal core. I should have phrased that differently; "Linking to a file in publicly-displayed html without knowing that the file will continue to be accessible at that URI, is a less-than-ideal, although common practice" or something.

Re #100: http://drupal.org/project/inline I use that on many sites for exactly that reason.

Bevan: it's not just the links which are problems. When moving files, you loose all the indexed content of those in search engines, all the ranks, etc.

When moving files, you loose all the indexed content of those in search engines, all the ranks, etc.

With all the ideas presented above, surely Drupal could present some option to preserve URLs for site owners with this dilemma? It could be as simply as a backwards compatibility mode that keeps the current (and brain dead) files directory scheme in place?

@Gabor, #102.

Yes, but is it in drupal's scope to handle that issue? IMHO I think not, at least not in the foreseeable future of drupal. If drupal provides a default location that is future-proof for new drupal sites (e.g. for single-sites that become multi-sites, or circumstances requiring that the files directory be moved), then I think drupal is handling this issue as well as can be expected.

Advanced drupal admins who move the files directory can fix references with an SQL replace() query on node bodies and 'move' SE indexes by setting up a 302-redirect to the new location. Drupal could help automate this redirect, but that's probably out of scope for d6. Maybe not for d7 or another future version.

Yes, moving the files directory should be discouraged due to the problems inherent of the internet that you mentioned. IMO drupal should still support moving the files directory -- even if it were only to make staging and sandboxed sites easier to manage.

So we would like:

• short URLs
• public files
• all site specific files in one directory
• one files directory per site
• easy hostname switching

How about a Drupal hostname-to-directory mapping table, just like path aliases are used now? Instead of this directory structure:

/sites/www.example.com/*
/sites/www.anotherlongerexample.com/*
/sites/www.anotherstilllongerexample.com/*

we would get:

/sites/foo/*
/sites/bar/*
/sites/baz/*

and a hostname alias table:

www.example.com -> foo
www.anotherlongerexample.com -> bar
www.anotherstilllongerexample.com -> baz

install.php would ask for the hostname and the alias, including some explanation of how and where the aliases are used. /sites/foo/files/ would be automatically created.

Gaele: where would that mapping be stored? Drupal needs to select a site config file first before loading the DB, so if you require the mapping to be used before the site config file can be loaded, you need a multisite level settings file (which is not present in Drupal 6 or earlier, so you need to introduce that one too). Then that settings file can have the mappings. That's what is on your mind?

Using the proposed solution, we can achieve what Gaele suggested....

Defaults:

/sites/default/files
/sites/example.com/files
/sites/anotherlongerexample.com/files

This would give us the public site-specific files directories for easy hostname switching by default. Through admin/settings/file-system, we can provide a cleaner URL for these, by changing them to something like:

/sites/foo/*
/sites/bar/*
/sites/baz/*

... Problem solved. The solution is provided in the previously uploaded patch (needs review and works against 6.x).

You could make URLs even cleaner by adding a rewrite rule to the .htaccess file.

So how having both sites/example.com/... and sites/example-com-alias/... solve the problem of a single root to archive files? How is it any different of having the site alias anywhere else?

I use sites/*/files extensively on my lighttpd simple_vhost multisite setup.

Gábor: Yes, a /sites/settings.php.

Rob: We're talking about public files, so Drupal is not available for cleaner URLs.

Gaele: sites/foo and sites/bar would be physical directories that you manually create, and then map admin/settings/file-system to it. The files directory would default to sites/example.com/files, and you'd just have to change it to your newly created directories.

A mapping at sites/settings.php relating certain site directories would be neat.....

$conf['sites']['example.com'] = 'foo';
$conf['sites']['myextralongexample.com'] = 'bar';

... Would make sites/example.com read from sites/foo. When you have a large amount of sites, however, it might be hard to maintain.

Rob: Map where? The directory is chosen before Drupal gets to the settings file. That's how it gets to the settings file. You'd need a separate, install-specific settings file in the site root for that, and just thinking about that makes me cringe.

Crell: The default files directories would be sites/*/files, you'd set the custom files directories just like you do normally, in admin/settings/file-system. I think the current patch that's up is probably the best solution. Now that I think about it, adding another settings.php file would make my head explode.

There won't be any /sites/www.example.com/* left, only /sites/foo/*. So there would be only one directory per site. A file /sites/settings.php could provide the mapping to the right hostname. Perhaps this file could be administered via Drupal, just as install.php currently does with /sites/default/settings.php, so no manual configuration would be necessary.

Thinking of this, default is already an example of a foo directory. (The default foo ;-)

subscribing.

Summarizing:
* and correcting from my view of things

The point is to make a system that is usable for beginners while still powerful for advanced users. I believe that these methods will accomplish that.

1. Fresh installs ONLY: Any change we make has to deal with the default files directory for fresh installs only. You may of course configure any directory. Trying to automate a move/change of existing file structures is way beyond this conversation considering all the possibilities. This decision needs to be based off of making a fresh install usable for a beginner. An advanced user may still do as they wish.
2. File paths: This has to be changed to: [Corrected according to #119]
   1. Public files: Path in files table should be relative to the files directory. The config path should be the path to the files directory relative to the drupal root.
   2. Private files: Path in files table should be relative to the files directory. The config path should be an absolute system path to the files directory.

   These changes will obviously break upgrades, so part of the drupal upgrade script will have to look at the current style and change the path field appropriately. Note that this will not break links because the files have not been moved. Only the path in db changed to allow it to work with the new way of building a file path. As for installs that never saved the drupal file path, their file path is obviously the default /files so if the config is not set, default to that.

3. Single-site & Multi-site: Many users will be using both styles but we have to look at the skill level of those users and make decisions accordingly.
   1. Beginner: The beginner starts off with a single site and might move up to multi-site but does not have enough knowledge yet to help them chose a directory and file method. They just want to get up and running easily. So I suggest the install script write the default files directory to /sites/default/files. This allows them to get up and working without issues. If they want to move up to multi-site and don't yet have the skill to create a directory, set it writable, and set the config page, then they can still have multi-site but their sites will share the /sites/default/files directory.
   2. Intermediate: The intermidiate user can do everything like the beginner but now has the option to create a files dir where they want and configure it correctly. This includes creating a /sites/[site_name]/files or /files or /sites/default/files/sitekey or whatever directory they want as long as it is a physical directory.
   3. Advanced: The advanced user can do any thing they want including using symbolic links to map files where and how they wish. There is nothing here that limits what an advanced user can do. In fact with the changes made to the paths in the DB, things are actually easier for them.
4. Pretty URLs: Yes example.com/sites/default/files/example.file is not as pretty as example.com/files/example.file but neither is index.php?nid=1234 either. If you have pretty urls turned on we can easily map the files directory to something prettier like the rest of the URLs. This even helps to alleviate moving of the files directory (if you did so) because the outside URL has not changed even though the directory did.
5. Dev & Staging sites: Development and staging sites are an advanced user function. If you have this level of workflow then you should have the skills to map directories with symbolic links and any other tweaks you have to do to accomplish the level of development that you are after.

If I missed any concerns let me know.

Since there is only one files directory per site, both public and private file paths should be relative to the files directory.

Correction: Sorry about that. You are correct. I should have said:

1. Public files: Path in files table should be relative to the files directory. The config path should be the path to the files directory relative to the drupal root.
2. Private files: Path in files table should be relative to the files directory. The config path should be an absolute system path to the files directory.

Subscribing

I think dragonwiz sums it up very well in #117, sounds like the overall best solution. A big +1 from me, that is!

As for the /tmp directory, its contents isn't really that vital.. With backups in mind, it's maybe not such a good idea to move it to /sites/default/tmp by default?

And yea, this change should not affect the existing setup when upgrading from an earlier version. As far as I can see, if the change to how file paths are stored in the database can be achieved without upgrade problems, all that remains is to change the default value for the variable file_directory_path to "sites/default/files", and finally include this directory upon installation... Of course, the variable should not be touched and the directory not be created upon upgrade.

One other thing to keep in mind is what Crell mentioned here, that this needs to be implemented in the install profile rather than the core install system.

Didn't mean to change the status, don't know how that happened..

I'm wondering, have you folks done programming that might be related to this issuie:
http://drupal.org/node/219416

the sites/default/files directory path no longer uninstalls cleanly in v6 for people who don't have superuser priveleges or web group privileges (Which means everyone on a shared hosting service, basically).....

Subscribing to track this post. I totally agree with what cog.rusty brought up in #59 and #62. gaele's approach to solve this problem (#105) looks interesting to me.

Pretty URLs: Yes example.com/sites/default/files/example.file is not as pretty as example.com/files/example.file but neither is index.php?nid=1234 either. If you have pretty urls turned on we can easily map the files directory to something prettier like the rest of the URLs. This even helps to alleviate moving of the files directory (if you did so) because the outside URL has not changed even though the directory did.

Given the numerous practical benefits of moving files to /sites/*/files, I hope that pretty URLs would not trump this issue.

Even though the proposed /sites/*/files URL is not the superlative pretty URL, 1. it is less crufty than traditional non-pretty node URLs and 2. I'd bet the typical use case for /files stuff is <img> tags or direct links from the node from which a file was uploaded, in which case both the user won't see the URL until after clicking because it will be hidden in a hyperlink and the author can easily copy and paste the URL from the Upload files section.

This patch deals with at least some of the issues here: http://drupal.org/node/231298

Just to clarify that pretty URLs was the least of my concerns.

My main concern was that a dependency of the file paths from the domain name makes it impossible to move sites (e.g. to a local development site) -- except if you use Unix symlinks or Windows junctions, something which the installation script can't handle automatically for all cases.

So, the basic Drupal installation loses mobility for any sites/example.com site. It retains mobility only for the sites/default site. And I don't believe that there are "numerous benefits" to justify this. Only one benefit: one-place-backup.

Hence, the patch mentioned goes to a good direction.

This may go beyond the scope of this specific issue, but should Drupal take more active management of "files"? E.g., a node represents each file, and when the node is referenced, ship a copy of the file to the user?

@Aren: Yes, that's far beyond the scope of this issue as well as being a long-running debate about what is a node, what is a first-class entity, etc. There are a dozen other threads where that's been debated. Let's keep this one simple and focused. :-) (Or as simple and focused as any thread can be that's over 131 replies...)

catch (#127): I believe the patch you mention more or less implements what I suggested in #105.

I'm a begginer to Drupal, but I've learned fast. I now have 3 sites up and running. 2 of them have the files directory at the dropal root (i.e., the default) and one has it in the sites/*/files location. My question is why is this preferred (aka, why do I "suck")? If it's a security issue, could I keep the default but secure it with .htaccess? One site is too big to start over, but the other just has a hand full of files...I suppose I could delete the content and move the files folder...but I'd rather not.
thanks

I'm a begginer to Drupal, but I've learned fast. I now have 3 sites up and running. 2 of them have the files directory at the dropal root (i.e., the default) and one has it in the sites/*/files location. My question is why is this preferred (aka, why do I "suck")? If it's a security issue, could I keep the default but secure it with .htaccess? One site is too big to start over, but the other just has a hand full of files...I suppose I could delete the content and move the files folder...but I'd rather not.
thanks

I'm a begginer to Drupal, but I've learned fast. I now have 3 sites up and running. 2 of them have the files directory at the dropal root (i.e., the default) and one has it in the sites/*/files location. My question is why is this preferred (aka, why do I "suck")? If it's a security issue, could I keep the default but secure it with .htaccess? One site is too big to start over, but the other just has a hand full of files...I suppose I could delete the content and move the files folder...but I'd rather not.
thanks

I'm a begginer to Drupal, but I've learned fast. I now have 3 sites up and running. 2 of them have the files directory at the dropal root (i.e., the default) and one has it in the sites/*/files location. My question is why is this preferred (aka, why do I "suck")? If it's a security issue, could I keep the default but secure it with .htaccess? One site is too big to start over, but the other just has a hand full of files...I suppose I could delete the content and move the files folder...but I'd rather not.
thanks

Earnie: If it's changed to sites/*/files in Drupal 6, then why is this feature request still open?

Good point.

From Drupal 6 file.inc since http://cvs.drupal.org/viewvc.py/drupal/drupal/includes/file.inc?view=dif... :

function file_directory_path() {
  return variable_get('file_directory_path', conf_path() .'/files');
}

Drupal 6 doesn't handle the upgrade case that was attempted (kinda) in this issue, but considering that there is a fairly diverse set of places to upgrade from (making an automated upgrade fairly complex), and nothing is broken by default that is a reasonable compromise. In other words, this is a duplicate of http://drupal.org/node/194369 (or perhaps the other way round...).