In short

Drupal 7 incorrectly detects if a directory is writeable for httpd's user when permissions are granted via Linux native (extended) ACL.

The is_writable() PHP function, when working with local files successfuly detects writablity:

* is_writeable('sites/all/default/files/languages') returns TRUE

But when using stream wrappers (Drupal 7 way) check fails:

* is_writeable('public://languages') returns FALSE

Long story

I was unable to find an issue about this problem, but since I ran into troubles with writing/uploading files I can't sleep in the search of appropriate fix :(
Recently I started upgrading all my servers configuration to ACLs which I personally consider as the best way to do things.

In my setup write access to httpd's user on the 'sites/default/files' directory and subdirectories is granted via ACLs. For example, this is getfacl output for 'languages' directory:

# file: www/sites/default/files/languages
# owner: onkeltem
# group: onkeltem
user::rwx
user:www-data:rwx
user:onkeltem:rwx
group::rwx
mask::rwx
other::--x
default:user::rwx
default:user:www-data:rwx
default:user:onkeltem:rw-
default:group::rwx
default:mask::rwx
default:other::--x

As you see it is writeable for 'www-data' user. But when locale module tries to rebuild language js files I got error message:

The specified file temporary://fileZdBOPz could not be copied, because the destination directory is not properly configured. This may be caused by a problem with file or directory permissions. More information is available in the system log.

and one more in the log:

File temporary://fileZdBOPz could not be copied, because the destination directory public://languages.

The problem

The file_prepare_directory() function checks if a directory is writable using PHP function is_writable():

  ...
  // The directory exists, so check to see if it is writable.
  $writable = is_writable($directory); 
  if (!$writable && ($options & FILE_MODIFY_PERMISSIONS)) { // (1)
    return drupal_chmod($directory);                        // (2)
  }

which in its turn calls stat() implementation of the stream DrupalPublicStreamWrapper which is DrupalLocalStreamWrapper which implements url_stat():

    ...
    if ($flags & STREAM_URL_STAT_QUIET || !file_exists($path)) {
      return @stat($path);
    }
    else {
      return stat($path);
    }

thus returning regular PHP stat array to the is_writable(), but since this array has nothing about extended ACLs all subsequent checks are senseless.

To finish the story lets see how things continue going wrong after is_writable() returns FALSE:

(1) Attempt to check - if we called with the option to modify permissions
(2) Try to chmod a directory...

Well, I'm not a guru and just interesting why we do this even? What are chances that having NO write access on a dir, we still have permissions to modify permissions? The only situation of I can think about and when this will work:

dr-xr-xr-x    www-data:www-data    mydir

.... and we already owning a dir. But after all, how happened that a dir got r/o - who and why would need this and under what circumstances? It was definitely not the Drupal who previously set r/o, then it should be an administrator who did this, right? Then why Drupal now making the directory writeable on its own? Maybe I'm missing somthing but yet I don't understand the reasoning of this action :)

Need help

Well, I don't know how this could be easily fixed, except of patching url_stat() to return modified stat array (with faked UID = httpd's UID, or with 777 mode), but IMHO the real solution would be:

* Don't check writability at all using any kind of PHP weird stuff - just try to _really_ _write_ a (test-) file.
* Don't chmod anything - this either won't work (when httpd is not owing a file) or would probably break some (of couse stupid) administrator's arrangements.

Also, to support ACLs Drupal shouldn't move files - just copy/unlink them (maybe it already does, I didn't check, but faced with this problem in Drush and thanks PHP devs - this is fixing in php.ini via to disable_function = rename.

Comments

OnkelTem’s picture

Version: 7.12 » 7.9
Priority: Major » Normal
Status: Needs work » Needs review

I created simple replacement for malfunctioning PHP's is_writable() which works like charm:


function drupal_is_writable($path) {
	//NOTE: use a trailing slash for folders!!!
  if ($path{strlen($path)-1} == '/') { // recursively return a temporary file path
    return drupal_is_writable($path.uniqid(mt_rand()).'.tmp');
  } elseif (is_dir($path)) {
    return drupal_is_writable($path.'/'.uniqid(mt_rand()).'.tmp');
  }
  
  // check tmp file for read/write capabilities
  $rm = file_exists($path);
  $f = @fopen($path, 'a');
  if ($f === FALSE)
      return FALSE;
  fclose($f);
  if (!$rm)
   unlink($path);
  return TRUE;	
}

Originally this variant was posted in php.net comments in 2007: http://php.net/manual/ru/function.is-writable.php#73596

Now I'm ready to create patch which will replace all occurences of is_writable() with the new function.

OnkelTem’s picture

Status: Active » Needs review
StatusFileSize
new5.37 KB

The patch attached replaces all occurences of is_writable() with drupal_is_writable() and affects files:

  • includes/file.inc
  • includes/install.inc
  • includes/updater.inc
  • modules/system/system.admin.inc
  • modules/system/system.module

After applying the patch on fresh D7.9, I successfuly managed to install Drupal w/o a single chmod/chown, having all permissions for Apache user set in extended ACLs. Before now the install process was failing, requiring old-style permissions to be set.

The code of drupal_is_writable definitely needs cleaning up and review. Please, don't ignore this issue. After all, how long we would tied to use old style backwarded *nix 777 permissions while having so brilliant extended ACLs?

jlporter’s picture

please do not commit this patch, writing files to see if the directory is writable?! for this odd use case...no way. It would be better to find the underlying cause of is_writable returning a false incorrectly. I would imagine it has something to do with the stream wrapper implementation in core. This behavior is not able to be duplicated with a regular file stream wrapper.

IMHO writing random files and deleting them is a bad idea....just as bad as 777.

OnkelTem’s picture

please do not commit this patch, writing files to see if the directory is writable?! for this odd use case...no way.

Since when using ACLs is oddity? :)

I would imagine it has something to do with the stream wrapper implementation in core.

This is in the issue:

return @stat($path);

There is nothing in the stat array which would help (without cheating).

IMHO writing random files and deleting them is a bad idea....just as bad as 777.

What are you talking about? :) What's wrong with this?

1) This is not frequent operation so will probably not affect performance.
2) It is only touching a file/dir - when Drupal cared about this?
3) I agree actually - this is not TRUE and not RIGHT. It is yet another workaround we do regulary to fix PHP's malfunctioning stuff.

jlporter’s picture

If it's a php problem then duplicate it with filestream wrapper. Prove it's php before blaming it and hacking some junk to work around it.

When it comes to performance this is a big penalty. You are exponentially increasing not only the stats to the fs but also filesystem operations which on a busy server is a performance penalty.

OnkelTem’s picture

If it's a php problem then duplicate it with filestream wrapper.

If you find an alternative fix, I would gladly take it :)

Prove it's php before blaming it and hacking some junk to work around it.

I did.
url_stat() should return what stat() returns. And stat() is unable to return ACLs: http://php.net/stat
Even if it could, them we have to check ACLs ourselves.
Meanwhile, how to fix the problem with completely unworking ACLs in Drupal?

When it comes to performance this is a big penalty. You are exponentially increasing not only the stats to the fs but also filesystem operations which on a busy server is a performance penalty.

Any is_writable() call supposes calling stat, ok?
So I'm not increasing anything.

orlitzky’s picture

Version: 7.9 » 7.12
Priority: Normal » Major

This prevents imce from working, so in my opinion is a major priority. I agree that writing and deleting a temporary file is the simplest way to check this correctly.

catch’s picture

Status: Needs review » Closed (duplicate)
OnkelTem’s picture

Status: Closed (duplicate) » Needs review

@catch

Duplicate means the solution is the same.
What made you think this one is duplicate?
The target issue has nothing to do with ACLs using _NOT_WORKING_ PHP's is_writable() stuff.

catch’s picture

Status: Needs review » Closed (duplicate)

Duplicate means the underlying bug is the same and that one of the issues will fix it. is_writable() doesn't work with stream wrappers, and both issues are introducing a custom wrapper around is_writable() to workaround this - however the other issue has considerably more information on it, and the patch is much closer to being RTBC.

If you can still reproduce the bug with the patch from #944582: Check for execute permissions on directories that require file write permissions then feel free to re-open this issue or follow-up there with explicit steps to reproduce.

OnkelTem’s picture

If you can still reproduce the bug with the patch from #944582: ./sites/default/files directory permission check is incorrect during install AND status report then feel free to re-open this issue or follow-up there with explicit steps to reproduce.

I won't even bother with trying that patch, since the code clearly states: we don't deal with ACLs and will never do.
Ok, its your decision to merge this into that.

I'm laying down the hope to get this working in Drupal. It works for me with my patch, so I don't care anymore. Good luck.

catch’s picture

Status: Closed (duplicate) » Needs work

OK if it's not a genuine duplicate let's re-open this then.

orlitzky’s picture

I agree that they're very similar issues; but I think the patch at #944582: Check for execute permissions on directories that require file write permissions should be using OnkelTem's writability test rather than is_writable() && drupal_is_executable(). The existing patch there kind of works, but still fails to determine writability in many cases (such as this one).

paularmand’s picture

Also have ACLS set on the files directory, and this is not a marginal phenomenon.

I read both issues (this and #944582: ./sites/default/files directory permission check is incorrect during install AND status report). I agree with mjorlitzky: They seem similar, but imho are not duplicates. I also tend to agree that writing and handling errors to test writability is at this moment our best possible option.

xjm’s picture

So is the other issue instead a duplicate of this one? Or do they have the same solution such that we should merge them?

orlitzky’s picture

The two issues are slightly different, but could be fixed with one patch.

The patch at #944582 fixes the executability test by introducing a wrapper around is_executable() called drupal_is_executable(). However, it also adds a wrapper, file_directory_is_writable(), around is_writable(), which calls both is_writable() and the new drupal_is_executable().

The latter is incorrect; the only writability test that has any chance of working is the one in this issue: you write a file. If we replace the file_directory_is_writable() in that issue with OnkelTem's implementation of drupal_is_writable() from this issue, it should fix both.

xjm’s picture

Version: 7.9 » 7.12
Priority: Normal » Major
Status: Needs review » Closed (duplicate)

Alright, in that case, I am going to re-close this as a duplicate (again), and instead let's broaden the scope of #944582: Check for execute permissions on directories that require file write permissions to include this issue and update the patch there to use the technique from the patch here.

Thanks mjorlitzky and paularmand!

anarcat’s picture

I confirm that #944582 fixes this issue.

dooug’s picture

dooug’s picture