Not sure what the problem is but will try and explain. I am using Hostgator with a VPS account

I create a new domain on CPanel and install D7 RC3 and at the Drupal installation complete / Congratulations, you installed Drupal! / Visit your new site. stage I get these errors

# Warning: unlink(/tmp/update-cache/pathauto-7.x-1.0-alpha1.tar.gz) [function.unlink]: Permission denied in drupal_unlink() (line 2139 of /home/co2n8yfo/public_html/includes/file.inc).
# Warning: unlink(/tmp/update-cache/880354) [function.unlink]: Permission denied in drupal_unlink() (line 2139 of /home/co2n8yfo/public_html/includes/file.inc).
# Warning: unlink(/tmp/update-cache/typebased-7.x-1.5.tar.gz) [function.unlink]: Permission denied in drupal_unlink() (line 2139 of /home/co2n8yfo/public_html/includes/file.inc).
# Warning: unlink(/tmp/update-cache/live_person-7.x-1.3.tar.gz) [function.unlink]: Permission denied in drupal_unlink() (line 2139 of /home/co2n8yfo/public_html/includes/file.inc).
# Warning: unlink(/tmp/update-cache/site_map-7.x-1.x-dev.tar.gz) [function.unlink]: Permission denied in drupal_unlink() (line 2139 of /home/co2n8yfo/public_html/includes/file.inc).
# Warning: unlink(/tmp/update-cache/pathauto-7.x-1.0-alpha2.tar.gz) [function.unlink]: Permission denied in drupal_unlink() (line 2139 of /home/co2n8yfo/public_html/includes/file.inc).

And about 50 others - this is also the same if I go to the /admin/reports/status and choose - You can run cron manually

If I install a fresh account on CPanel and install D& RC2 - I do not get these errors

I also have 8 domains installed on another account using the same core files. I upgraded from D7 RC2 to RC3 and get the same errors if I go to the /admin/reports/status and choose - You can run cron manually - BUT only on six of the domains two seem to work OK

I reverted to the D7 RC2 back up and they all worked ok

All permissions seem ok and the same

Russell

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

XTCHost’s picture

Just to clarify on the fresh install of D7 RC3

I create a new domain on CPanel and install D7 RC3 and at the Drupal installation complete / Congratulations, you installed Drupal! / Visit your new site. stage I get these errors

I also had created a new database

The stage mentioned was at the point of seeing the message "Drupal installation complete / Congratulations, you installed Drupal! / Visit your new site"

Not after visiting the site

David_Rothstein’s picture

Title: Errors on Install and upgrade » Permission denied errors deleting /tmp/update-cache/* files when running cron
Component: install system » update.module

Where did the files like /tmp/update-cache/pathauto-7.x-1.0-alpha1.tar.gz come from? I guess those are files you uploaded via one of the other Drupal sites on the server?

I don't know how the permissions are set up on your server but it seems like there is a collision going on. Does the site on each domain run as a separate webserver user?

Perhaps it would make sense for the Update Manager in Drupal to avoid these kinds of issues by putting its files in a directory with a unique name associated with the site, rather than always in the same /tmp/update-cache. Not sure.

David_Rothstein’s picture

Title: Permission denied errors deleting /tmp/update-cache/* files when running cron » "Permission denied" errors deleting /tmp/update-cache/* files when running cron
XTCHost’s picture

I will look into where the files may be, but cannot see them being anywhere else as a lot of them mentioned in the errors have never been used

I have uploaded all the errors to a page

http://pastebin.com/qek152Ph

This is a completely fresh install, using just the drupal-7.0-rc3.tar.gz file and I have not changed anything in the admin

Also - with the same server and a fresh account / database, why would it not happen with D7 RC2 ?

XTCHost’s picture

I have sorted out the problem with this by deleting the root/tmp/update-cache/ folder, as the files where found in there. I had been looking in the home/username/tmp

Interesting that it did not have this problem when installing D7 RC2

XTCHost’s picture

Status: Active » Fixed

Fixed

gp.mazzola’s picture

I also experienced this issue on a shared host (bluehost) and it took some minute to figure out the solution (thanks given to this thread too).
I experienced the problem on a very fres RC3 installation so at first I was not able to understand why drupal was complaining about those modules.
Then I thought that the default path for tmp folder in file system settings is /tmp/ , wich on a share d host machine is accessed by different users. In fact those files were not mine and consequently I was not able to delete them to fix it.
In this case my solution was changing the tmp directory to the tmp in my home directory, so

/home/myusername/tmp

Hope it can help!

Ciao,
GP

gp.mazzola’s picture

Status: Fixed » Active
dww’s picture

Category: bug » support
Status: Active » Fixed

Update manager just uses whatever temp directory you tell it to. I think you're asking for trouble if you have multiple sites sharing the same tmp directory.

David_Rothstein’s picture

Version: 7.0-rc3 » 7.x-dev
Category: support » bug
Status: Fixed » Active

#1008776: Unable to unlink files in /tmp directory after installation was duplicate.

Although configuring your site to use an isolated tmp directory is definitely the best way to deal with this, I don't think that necessarily means Drupal shouldn't do anything to avoid it.

Doesn't Drupal already generate unique names every time it creates a file in /tmp, to avoid these types of collisions? So shouldn't it do the same thing when it creates a directory? (I am not sure if there is anywhere else in core where we create a directory within /tmp.) It seems like one simple avenue to fixing this would be to have the directory be named "update-cache-xxxxx", where the "xxxxx" is based off the site's private token, something like that.

gp.mazzola’s picture

Hi David,
I perfectly agree with you and it was just what I was trying to point out here:

http://drupal.org/node/1009028 (could this be set as duplicate?)

As you say, if choosing an isolated tmp directory solve the problem, that does not mean that Drupal should not care about it. A temporary directory is just a temporary directory, so the user should not be so concerned about what happens inside it.
Then maybe a better design probably would be adding a unique string to the name. Moreover update-cache is not the only directory, there is also update-extractions, and I don't know if other ones used by other modules.
So a more general way that does not care about the directory chosen in the file system settings could be creating a directory with a unique name bound to that single drupal installation inside the chosen tmp directory. Then inside that you can create whatever you want without collisions problems:

/tmp/dir_with_unique_safe_name/

and inside here

update-cache
update-extractions
...

I think it would be a better design that would not request too much effort.

Finally, if they decide that this is not worth to be "fixed", I think that at least the user after the installation of Drupal should be presented with the file system settings page with some notice about the way he should choose safely the tmp directory for his site. By the way, if I am not wrong, Drupal 5 used to do this, or in the status report page after installation it used to show that cron had not been run yet and that file system infos had not been set

jduguid’s picture

Version: 7.x-dev » 7.0

I appreciate all the documentation you guys have provided on this issue however I seem to be missing the solution for my particular hosting scenario. I've just installed Drupal 7 (today's release) and I'm receiving the same errors. I've created the following directory:

/tmp/drupaltemp/

But I still get an error when trying to set the temp directory:

* Warning: mkdir() [function.mkdir]: Permission denied in drupal_mkdir() (line 2244 of /mnt/stor3-wc2-dfw1/505291/drupal7.yhc.edu/web/content/includes/file.inc).
* The directory /drupal7.yhc.edu/web/content/tmp/drupaltemp does not exist and could not be created.

The /tmp directory doesn't seem to exist on my installation, so I created it after the errors appeared on install. Any help you could provide would be appreciated.

gp.mazzola’s picture

Hi jfduguid.
Are you giving Drupal the correct path to that directory?
Have you checked the permissions setting for that directory? It has to be writable by the web server user

bye,
gp

jduguid’s picture

gp,
I'm embarrassed to say I didn't try the path from the root, I used the server path. And that's what the problem was. It's solved. Thank you for your help!

James

waverate’s picture

I see that the discussion for #1008776: Unable to unlink files in /tmp directory after installation is here but I have the original issue with errors after installation;

Warning: rmdir(/tmp/update-extraction/ckeditor/includes) 

I am not able to delete the update-extraction or update-cache directories from the CLI. It is possible that another user has written to these directories on the server as they have a default name. To me this is expected on a shared host that I would not want to put files in /tmp.

Wouldn't it be easier to specify the location of the temp directory in the installation or at least configured through settings.php prior to starting the installation?

gp.mazzola’s picture

Hi waverate.
If you are on a shared host things most likely went this way:

  1. drupal set by default the temporary directory to /tmp/ wich is common to the users of the shared host machine, and writable by all
  2. a user installed d7 before you and ran the update manager through web interface. This created that directory. The first user has the luck to be the first one :)
  3. you arrived and installed D7 wich set temporary directory as /tmp/ by default (point 1)
  4. at first run cron gave you the error as I tries to delete that directory, but it has not the right as it is owned by another user

You CAN'T delete that dir neither from shell as it is not owned by you. You can work around it by going on file system settings and changing the tmp to another dir inside your home, by example

/home/you_user_name/tmp/your_drupal_site_tmp

Answering to your question, IMHO yes. Of course. The user should be presented with a screen to make the choice.
Even if the use of a unique name by the application itself, would be a far more elegant solution, IMHO.

waverate’s picture

gp,

I think you hit the nail on the head in your summary.

It is a little silly to change the temp directory AFTER the installation but as you mentioned, it is a workaround.

I have not been able to change the default temporary directory in settings.php with:

$conf['file_temporary_path'] = 'tmp';

Any suggestions?

bfroehle’s picture

Title: "Permission denied" errors deleting /tmp/update-cache/* files when running cron » Uniqueify tmp directory to prevent "Permission denied" errors deleting /tmp/update-cache/*
Status: Active » Needs review
FileSize
694 bytes

I like David_Rothstein's idea in #10 (and similar ideas mentioned later) of having unique directory names.

One approach would be to alter file_directory_temp to return to automatically append '/drupal-UNIQUESTRING' to the end of the tmp directory it would have normally found. This is similar to the operation of many applications in linux.

I've attached a patch, which is completely un-tested, which creates a unique 8-digit name by doing

substr(drupal_hmac_base64('file_directory_temp', drupal_get_hash_salt()), 0, 8);

Anyway, let's kick around this idea for a bit and see what everybody thinks.

Status: Needs review » Needs work

The last submitted patch, unique-tmp-dir.patch, failed testing.

bfroehle’s picture

Needs a comment like:

      // Since the temporary directory may be shared amongst many different
      // users and Drupal installations, restrict to a subdirectory which is
      // unique to this installation of Drupal.
+++ includes/file.inc
@@ -2361,6 +2361,10 @@ function file_directory_temp() {
+      $temporary_directory .= $path_delimeter . 'drupal-' . $unique_key;

Should be $path_delimiter.

Does drupal_hmac_base64 return filesystem safe paths?

Also we'll need to make this directory the first time around, if it isn't made already.

bfroehle’s picture

Status: Needs work » Needs review
FileSize
3.33 KB

New patch, fixing my own complaints in #20. Verified that on a new install it set the tmp directory to

/Applications/MAMP/tmp/php/drupal-8fb86b90

Status: Needs review » Needs work

The last submitted patch, 1008328-21-unique-tmp-directories.patch, failed testing.

bfroehle’s picture

Status: Needs work » Needs review
FileSize
880 bytes

I accidentally included a bunch of other things I'm working on in that last patch. Trying again...

waverate’s picture

Patch at #23 works perfectly and fixes #15 above.

Thank you.

hass’s picture

Isn't this not a buggy server config? This shared tmp directories can lead to security issues if they are shared. I would never put my tmp files into a shared folder where all users of a vps have access too. Maybe i'm uploading a secret file to my private folder and someone who monitors the tmp folder may read my files. I would like to know if there are permissions set for the unique site tmp folder that prevent reading the folder from other users?

This is the reason why I always add a tmp folder into a dedicated site structure that is protected against other sites. It may only a bit difficult to automate this... We should check and set the tmp directory permissions to lock it down, too! Not sure if drupal does this...

bfroehle’s picture

hass: Let me know what you find out after you review the code... I don't know the answers to your questions off-hand.

waverate’s picture

@hass,

I'm not quite sure why you think this is a server issue. It is Drupal that is trying to install into /tmp. To me that is outside a shared host directory and part of the actual server root.

For those of us who do not completely own /tmp, it would make sense to be allowed to set it in settings.php or at least be asked where we want to put it.

bfroehle’s picture

This will need to be reworked slightly if #1019470-5: file_directory_temp() fails to set valid temporary directory on Windows lands -- essentially the revised patch would need to change all $path_delimiter's to /.

zacpod’s picture

Component: file system » update.module
Priority: Major » Normal
Status: Needs work » Needs review
Issue tags: -Needs tests

This issue is hitting me too - hosting on Godaddy shared server.
Basic install, and PHP is apparently returning a tmp dir that I don't have rights to.
Tried hardcoding the dir to one I control in ./includes/file.inc but that doesn't seem to be working either. For now I'll have to manually update+install modules via ssh & wget. :(

Thanks DWW (post #30) I can't believe I didn't see that! *blush*

dww’s picture

Component: update.module » file system
Priority: Normal » Major
Status: Needs review » Needs work
Issue tags: +Needs tests

Core has a setting for where you want your tmp dir. No one needs to hack core to change where the tmp directory is for a site. You can do it in the admin UI (in D7 at admin/config/media/file-system) or just put this in your settings.php:

$conf['file_temporary_path'] = /some/where/specific/to/this/site

Visual inspection of the patch looks good, but we probably should write some tests for this. Also, since this is changing on the core file system is working, not just how update manager uses that system, I think this should be bumped over to the file system component to get those folks to see this, too. And, given how much grief this is causing with 7.0, seems we should bump this to major.

Damien Tournoud’s picture

The real bug seems to me that the update modules hardcodes a name in the temporary directory ("update-cache"). Temporary files should all use tempnam(), and should not be hardcoded.

dww’s picture

@DamZ: that's a fair perspective. ;) That's why I'm asking for wider visibility of this issue. However, before we just go back to that route, I'd like to point out that it sure seems a lot safer to just have the tmp dir be unique for each site, to prevent other similar bugs.

That said, if we're doing a unique subdir of /tmp for each site like #23, we're going to need something that notices when that directory is gone and re-creates it. A lot of systems will periodically purge all files in /tmp (e.g. on a reboot), so we can't assume that just because we created it once that it'll be around forever.

Damien Tournoud’s picture

#31 was just to put things into perspective. I think that #23 is a good idea anyway, and yes the directory is automatically created:

+      $temporary_directory .= $path_delimiter . 'drupal-' . substr(hash('sha256', drupal_get_hash_salt()), 0, 8);
+      drupal_mkdir($temporary_directory);
dww’s picture

Re: #33. I saw that it's automatically created once, when the variable is still empty. I'm talking about re-creating it in the future if/when /tmp is emptied. Let's call that needs work A. ;)

B) Are we doomed on an upgrade path here? Every existing site in the wild is already going to have file_temporary_path defined, so they're not going to benefit from this at all. I see you can empty out the setting at admin/config/media/file-system and press "Save configuration" and it resets it to a site-specific subdir, but who's going to know to do that when upgrading to 7.1? And we can't just automatically clobber this setting so it's rebuilt, since we'll lose the settings that people have already customized to somewhere sane. I guess we could have some logic in an update method that replicates the base directory logic from file_directory_temp(), compares the existing variable to that, and removes it if they match. Or rather, we could move the base directory logic into _file_directory_temp_basedir() and share that between the update_N() function and the real file_directory_temp().

C) If you apply the patch, and you clear the old default, the new one is put in place and the subdir is created. However, if you clear again, you end up with the same subdir (obviously) which already exists, and you get a "Warning: mkdir() [function.mkdir]: File exists in drupal_mkdir() (line 2244 of /.../includes/file.inc)" error. So, basically, once we have the new tmp name, we should see if it exists and is writable. If it's not writable, we need a different subdir. If it is writable, we don't need to create it (although maybe it's actually a collision). Only if it doesn't exist should we try to create it.

waverate’s picture

@dww,

Thank you for confirming that the tmp directory can be changed in settings.ini. To be complete, the syntax is:

$conf['file_temporary_path'] = '/some/where/specific/to/this/site';

and a relative path is from Drupal root. The tmp directory is created if it does not exist. Thus the patch at #23 is no longer required to get Drupal installed in my case. I do like the concept of making the directory unique as it can help with multisite.

That being said, if $file_temporary_path is set in setting.php, it is not reflected in /admin/config/media/file-system. Temporary directory still shows /tmp.

Shouldn't this reflect the change?

bfroehle’s picture

That said, if we're doing a unique subdir of /tmp for each site like #23, we're going to need something that notices when that directory is gone and re-creates it. A lot of systems will periodically purge all files in /tmp (e.g. on a reboot), so we can't assume that just because we created it once that it'll be around forever.

Yes, this needs to be addressed before the patch goes in. A lot of stuff breaks if the directory location is defined but does not exist.

bfroehle’s picture

Just want to remind folks that this whole file_directory_temp code already needs to be massaged a bit due to other issues like #1019470: file_directory_temp() fails to set valid temporary directory on Windows.

bfroehle’s picture

Component: update.module » file system
Priority: Normal » Major
Status: Needs review » Needs work
Issue tags: +Needs tests
Adam S’s picture

I'm on a shared host. I started with a Drupal 7 installation, deleted the database, reset settings.php and on cron run I had these errors too. Changing the /tmp directory settings help. However, I think it should be the responsibility of the Drupal installation to make sure that files put in /tmp are cleared if anything happens to it.

dww’s picture

Issue tags: +Update manager

Since this effects the Update manager (perhaps most of all) tagging this so it's still visible even though it's in a different component now.

barbank’s picture

How do I execute it? I don't know much about working on the shell.

+ $temporary_directory .= $path_delimiter . 'drupal-' . substr(hash('sha256', drupal_get_hash_salt()), 0, 8);
+ drupal_mkdir($temporary_directory);

How do I run this?

bfroehle’s picture

Tor Arne Thune’s picture

FileSize
97.38 KB

So, I get these errors on installation on my shared host, because someone's Drupal installation has already created the update-extraction and update-cache directories in /tmp. This makes them only writeable by their account. Since one has to set file_temporary_path in settings.php before the installation, there is a good chance these errors will come up during the installation process for many users. There is not even a comment about setting this variable in default.settings.php.

Warning: opendir(/tmp/update-cache) [function.opendir]: failed to open dir: Permission denied in DrupalLocalStreamWrapper->dir_opendir() (line 700 of /home/content/32/5671032/html/includes/stream_wrappers.inc).
Warning: opendir(temporary://update-cache) [function.opendir]: failed to open dir: "DrupalTemporaryStreamWrapper::dir_opendir" call failed in file_scan_directory() (line 1975 of /home/content/32/5671032/html/includes/file.inc).
Warning: opendir(/tmp/update-extraction) [function.opendir]: failed to open dir: Permission denied in DrupalLocalStreamWrapper->dir_opendir() (line 700 of /home/content/32/5671032/html/includes/stream_wrappers.inc).
Warning: opendir(temporary://update-extraction) [function.opendir]: failed to open dir: "DrupalTemporaryStreamWrapper::dir_opendir" call failed in file_scan_directory() (line 1975 of /home/content/32/5671032/html/includes/file.inc).

Conclusion: Unique temporary directories would be perfect to avoid these errors. By the way, the shared host provider in question is GoDaddy. A pretty common host.

I will test the patch on this host tomorrow.

Tor Arne Thune’s picture

Title: Uniqueify tmp directory to prevent "Permission denied" errors deleting /tmp/update-cache/* » Uniqueify tmp directory to prevent "Permission denied" errors

So this issue was set to 'needs work' because it needs tests. Any takers?

bfroehle’s picture

General comments:

  • Drupal assumes that the tmp directory always exists. It creates the directory when it is set via the admin interface, but not by script. Often computers will periodically purge /tmp which is problematic as Drupal will not remake the directory if it no longer exists.
  • Another possibility is to just replace update_* subdirectories with update_*-UNIQUESTRING. Avoids the first problem...
Tor Arne Thune’s picture

Try #1:

Normal installation with temp dir as default.

  1. Extracted drupal 7.
  2. Set up empty database
  3. Copied the default.settings.php file to settings.php and changed permissions to make it writable by all.
  4. Navigated to base url and installed as usual.
  5. Received these errors on the 'Finished' step of the installation:

    Warning: opendir(/tmp/update-cache) [function.opendir]: failed to open dir: Permission denied in DrupalLocalStreamWrapper->dir_opendir() (line 700 of /home/content/32/5671032/html/includes/stream_wrappers.inc).
    Warning: opendir(temporary://update-cache) [function.opendir]: failed to open dir: "DrupalTemporaryStreamWrapper::dir_opendir" call failed in file_scan_directory() (line 1975 of /home/content/32/5671032/html/includes/file.inc).
  6. Navigated to Administration >> Configuration >> Media >> File system.
  7. Temporary directory is set to /tmp as expected.
Try #2:

Installation with temp dir set to directory in user dir, by using variable in settings.php.

  1. Extracted drupal 7.
  2. Set up empty database.
  3. Copied the default.settings.php file to settings.php and added line $conf['file_temporary_path'] = 'sites/default/files/temp'; to this file.
  4. Navigated to base url and began installation as usual.
  5. The directory temp inside sites/default/files was created on or after the 'Verify Requirements' step during the installation process. No files inside.
  6. After the 'Set up database' step, the line $conf['file_temporary_path'] = 'sites/default/files/temp'; has been removed from settings.php.
  7. Received these errors:

    Warning: opendir(/tmp/update-cache) [function.opendir]: failed to open dir: Permission denied in DrupalLocalStreamWrapper->dir_opendir() (line 700 of /home/content/32/5671032/html/includes/stream_wrappers.inc).
    Warning: opendir(temporary://update-cache) [function.opendir]: failed to open dir: "DrupalTemporaryStreamWrapper::dir_opendir" call failed in file_scan_directory() (line 1975 of /home/content/32/5671032/html/includes/file.inc).
    Warning: opendir(/tmp/update-extraction) [function.opendir]: failed to open dir: Permission denied in DrupalLocalStreamWrapper->dir_opendir() (line 700 of /home/content/32/5671032/html/includes/stream_wrappers.inc).
    Warning: opendir(temporary://update-extraction) [function.opendir]: failed to open dir: "DrupalTemporaryStreamWrapper::dir_opendir" call failed in file_scan_directory() (line 1975 of /home/content/32/5671032/html/includes/file.inc).
  8. Navigated to Administration >> Configuration >> Media >> File system.
  9. Temporary directory is set to /tmp.

The only way to avoid the errors during installation is to re-add the line to settings.php (after changing the permissions on the directory and file to writable again) on the 'Configure site' step. See try #3 below.

Try #3:

Installation with temp dir set to directory in user dir, by using variable in settings.php.

  1. Extracted drupal 7.
  2. Set up empty database.
  3. Copied the default.settings.php file to settings.php and added line $conf['file_temporary_path'] = 'sites/default/files/temp'; to this file.
  4. Navigated to base url and began installation as usual.
  5. The directory temp inside sites/default/files was created on or after the 'Verify Requirements' step during the installation process. No files inside.
  6. After the 'Set up database' step, the line $conf['file_temporary_path'] = 'sites/default/files/temp'; has been removed from settings.php.
  7. Changed the permissions on the directory and settings.php file to make it writable again.
  8. Re-added $conf['file_temporary_path'] = 'sites/default/files/temp'; to settings.php.
  9. Changed the permissions on the directory and settings.php file to make it non-writable.
  10. No errors!
  11. Navigated to Administration >> Configuration >> Media >> File system.
  12. Temporary directory is set to sites/default/files/temp as expected.
Try #4:

Installation with temp dir as default, but with patch in #23 applied to Drupal installation.

  1. Extracted drupal 7.
  2. Applied patch in #23.
  3. Set up empty database
  4. Copied the default.settings.php file to settings.php and changed permissions to make it writable by all.
  5. Navigated to base url and installed as usual.
  6. No errors!
  7. Navigated to Administration >> Configuration >> Media >> File system
  8. Temporary directory is set to /tmp/drupal-b2595571.
  9. Tried to install a module and a theme to verify that the temp directory worked correctly.

Conclusion:

Try #3 worked, but it's a hack, and I wouldn't recommend this method to anyone. The installer should not remove any custom lines in settings.php after the 'Set up database' step. It makes no sense, as settings.php is also meant to set variables before an installation.
Try #4 worked. The patch makes sense and works as described. It successfully solves my problem with the errors during installation.

bfroehle’s picture

Version: 7.0 » 7.x-dev

1V: Now with your "Try #4", try deleting the /tmp/drupal-b2595571 directory and see if file uploads, etc, still work. I don't think Drupal will re-create the directory if it is externally deleted.

Tor Arne Thune’s picture

I don't have any means to access /tmp. Not with an FTP client nor the GoDaddy file manager. I also don't have SSH access.

bfroehle’s picture

Title: Uniqueify tmp directory to prevent "Permission denied" errors » Uniqueify update-cache and update-extraction directories to prevent "Permission denied" errors
Status: Needs work » Needs review
Issue tags: -Needs tests
FileSize
3.4 KB

Since uniqueifying the temporary directories is going to be challenging at this point, especially since 7.0 is already out the door, I think we should shift gears and consider Damien's perspective in #31

The real bug seems to me that the update modules hardcodes a name in the temporary directory ("update-cache"). Temporary files should all use tempnam(), and should not be hardcoded.

Now it's a hard to use tempnam as we need a directory, not a file, and we need to keep track of the files over some period of time for cron jobs, etc.

So, instead I propose changing the update-extraction and update-cache directories to be of the form update-cache-unique_identifier where unique identifier is an eight character string generated by a sha1 hash of the Drupal hash salt. This is robustly unique, easy to generate, and permanent for the Drupal installation.

To accomplish this change, I've introduced a function _update_manager_unique_identifier() which returns the unique identifier, and a function _update_manager_cache_directory() which returns (and optionally creates) the update-cache-xxx directory. A similar _update_manager_extract_directory() already exists.

This change is minimal, effecting only the behavior of update module. It is robust against scripts which clear out /tmp on reboot. It requires no settings changes by the end user. I'm removing the Needs tests tag since testing this is essentially impossible since simpletest cannot mimic a shared hosting environment with multiple users with different user ids.

Tor Arne Thune’s picture

Newly posted issue proving that this patch is wanted: #1044040: Files in /var/tmp/update-cache are not named uniquely creating conflicts on shared hosts.. Marked the issue as duplicate of this.

mjlF95’s picture

I have unlink warnings so I tried this patch and I get "Call to undefined function _update_manager_cache_directory()" during installation. I'm on a virtual private server on Dreamhost.

bfroehle’s picture

mjlF95: Thanks for your report!

I've revised the patch to move _update_manager_cache_directory() and _update_manager_extract_directory() function into update.module (instead of update.manager.inc). I've tested installing Drupal manually with this patch and it does so without warning.

Something to ponder: is it better to have separate _update_manager_cache_directory() and _update_manager_extract_directory() functions, or should they instead be merged into a _update_manager_temporary_directory($type) where $type is 'cache' or 'extraction'?

mjlF95’s picture

My pleasure. Just wanted to share that I was able to apply this patch without errors and it seems to have solved the unlink errors. Thanks!

dlee20’s picture

I was able to resolve this warning by giving the absolute path to the /tmp dir instead of simply having a /tmp dir I specified /home/content/(info removed)/(info removed)/html/webapp1/tmp

where 'webapp1' is the folder in which I installed D7, I removed the two folders in the path for privacy concerns, but any godaddy user should know what this is for their particular site.
I also manually created the update-cache and update-extractions folders and set permissions.

I still have no idea where D7 thought '/tmp' was supposed to be but there was obviously no check for it during the install. in a shared hosted environment like godaddy, it would be a good idea to make sure that D7 installer checks where /tmp is and set the appropriate path, or ask the installer to specify it.

Tor Arne Thune’s picture

/tmp is always at /tmp in Linux. That is, in the directory tmp in the root of the filesystem (on the host machine, like GoDaddy's shared hosts). Drupal just does what it's told.

waverate’s picture

bfroehle,

Patch #52 works perfectly on a brand new install on a shared host. It fixes my issue at #15.

Thank you very much. This needs to be rolled into core.

David_Rothstein’s picture

Component: file system » update.module
Status: Needs review » Reviewed & tested by the community

I think it's good that this issue is going back to where it started. Trying to fix this generically for /tmp as a whole sounds pretty reasonable, but also would be better done as a separate issue (starting with Drupal 8).

I reviewed and tested the patch pretty carefully and can't find anything really wrong with it. For sites that are upgrading, this should just silently switch everyone over to the new directories with no side effects (except the old directories left lying around, which Drupal itself won't clean up anymore but they're in /tmp so they're fine staying there until /tmp itself gets purged).

I could find a minor nit or two (such as the PHPDoc using "Return" rather than "Returns") but that's an existing issue with the _update_manager_extract_directory() function anyway. I also wonder if it would be better to generate the ID from drupal_get_private_key() rather than drupal_get_hash_salt() - both semantically, as well as because it's less likely to ever change - but it really shouldn't matter on a practical level.

It sounds like others above also reviewed and tested, so I'm marking this RTBC.

TimelessDomain’s picture

+1 on patch #52

timtunbridge’s picture

path #52 worked for me

Critical Tinkerer’s picture

Don't know if this has already been patched in the dev version but, for what it's worth, I am getting this in Drupal 7.0 release version on a clean install:

    * Warning: unlink(/tmp/update-cache/entity-7.x-1.0-beta3.tar.gz) [function.unlink]: Permission denied in drupal_unlink() (line 2139 of /home/hfcom/public_html/international/includes/file.inc).
    * Warning: unlink(/tmp/update-cache/tablefield-7.x-2.0-alpha4.tar.gz) [function.unlink]: Permission denied in drupal_unlink() (line 2139 of /home/hfcom/public_html/international/includes/file.inc).
    * Warning: unlink(/tmp/update-cache/omega-7.x-1.0.tar.gz) [function.unlink]: Permission denied in drupal_unlink() (line 2139 of /home/hfcom/public_html/international/includes/file.inc).
    * Warning: unlink(/tmp/update-cache/entity-7.x-1.x-dev.tar.gz) [function.unlink]: Permission denied in drupal_unlink() (line 2139 of /home/hfcom/public_html/international/includes/file.inc).
    * Warning: unlink(/tmp/update-cache/webform-7.x-3.4-beta1.tar.gz) [function.unlink]: Permission denied in drupal_unlink() (line 2139 of /home/hfcom/public_html/international/includes/file.inc).
    * Warning: unlink(/tmp/update-cache/omega_tools-7.x-1.0-beta11.tar.gz) [function.unlink]: Permission denied in drupal_unlink() (line 2139 of /home/hfcom/public_html/international/includes/file.inc).
    * Warning: unlink(/tmp/update-cache/styles-7.x-1.0-alpha4.tar.gz) [function.unlink]: Permission denied in drupal_unlink() (line 2139 of /home/hfcom/public_html/international/includes/file.inc).
    * Warning: unlink(/tmp/update-cache/advanced_help-7.x-1.x-dev.tar.gz) [function.unlink]: Permission denied in drupal_unlink() (line 2139 of /home/hfcom/public_html/international/includes/file.inc).
    * Warning: unlink(/tmp/update-cache/webform-7.x-3.x-dev.tar.gz) [function.unlink]: Permission denied in drupal_unlink() (line 2139 of /home/hfcom/public_html/international/includes/file.inc).
    * Warning: unlink(/tmp/update-cache/translation_overview-7.x-2.x-dev.tar.gz) [function.unlink]: Permission denied in drupal_unlink() (line 2139 of /home/hfcom/public_html/international/includes/file.inc).
    * Warning: unlink(/tmp/update-cache/webform-7.x-3.3-beta1.tar.gz) [function.unlink]: Permission denied in drupal_unlink() (line 2139 of /home/hfcom/public_html/international/includes/file.inc).
    * Warning: unlink(/tmp/update-cache/views-7.x-3.x-dev.tar.gz) [function.unlink]: Permission denied in drupal_unlink() (line 2139 of /home/hfcom/public_html/international/includes/file.inc).
    * Warning: unlink(/tmp/update-cache/mollom-7.x-1.x-dev.tar.gz) [function.unlink]: Permission denied in drupal_unlink() (line 2139 of /home/hfcom/public_html/international/includes/file.inc).
    * Warning: unlink(/tmp/update-cache/omega-7.x-1.0-beta8.tar.gz) [function.unlink]: Permission denied in drupal_unlink() (line 2139 of /home/hfcom/public_html/international/includes/file.inc).
    * Warning: unlink(/tmp/update-cache/media-7.x-1.0-beta2.tar.gz) [function.unlink]: Permission denied in drupal_unlink() (line 2139 of /home/hfcom/public_html/international/includes/file.inc).

Congratulations, you installed Drupal!

Visit your new site.
maggielanoue’s picture

As a Drupal newbie I am having issues with installing "Views" module to a sub account on green geeks host - a "seed account" i have there. I see 2 basic solutions on this page we are on.

One is to change the name/location of the installation of the temp directory to a unique name to avoid collisions (how do i do that?? how do I tell the new module where to go?)

Two - the other solution (I like this one better) is maybe this issue is resolved in the dev version of drupal 7. (This would not be the first time I have started over from scratch to get things to go right.)

Which would work - and if I have to use the first solution, how exactly do I do that??

Thank you in advance for more details. I am not a programmer obviously.

TimelessDomain’s picture

@maggielanoue
1) go to http://yourdomain.com/admin/config/media/file-system -> change "Temporary directory" to "tmp/files" -> then create that directory on your site in the files folder (or relative to root directory depending on your settings) -> then give the folder write permissions. New modules should automatically use the temp directory set by drupal.
2) not yet fixed in dev. - although i didn't do a fresh install, i upgraded & still needed to do #1

jaymckinsey’s picture

I solved this problem by changing the temporary folder in the file system section to point at the tmp folder sitting at the root of my account on greengeeks hosting. i.e. changed /tmp to ../tmp

maggielanoue’s picture

Thank you @TimelessDomain and @jaymckisey for your replies. @TimelessDomain - this site is not yet associated with the domain name, I need to get it set up and working before pointing the domain there. (so this drupal association is located http://xxx.xxx.xxx.xxx/~xxxxxxxx/
I am accessing the file manager thru "cpanel". I do not see an "admin/config/media/..." but did look around esp. in the public_html where the drupal installation is. I do see a modules directory in there. Seems like that would be a good place to get the "Views" module to install to. I also see a tmp directory (not in the public_html directory) - and i see the way to change permissions and know a little bit about that. I do not know how to change directories to point to other directories, nor know if I am looking at the right ones in the first place. It should not be so hard to install a module that is as basic as "views." Should I just submit another ticket and ask the server to do this for me? I really want to learn but nothing matches what you describe. any additional help is much appreciated.

David_Rothstein’s picture

@maggielanoue, http://yourdomain.com/admin/config/media/file-system refers to a URL to visit in your web browser, not a location in the filesystem. It's a regular admin page on your Drupal site.

There's documentation for this stuff at http://drupal.org/node/15368, although that is for Drupal 6 so it refers to a different URL. Maybe when you understand how it works, you could file a documentation issue and work on creating a separate version of that page for Drupal 7?

Also, just some advice to everyone - in general it's better not to post these kinds of questions on bug reports. File a support request instead. This issue is just focused on getting the bug fixed (so that hopefully when Drupal 7.1 is released it won't be a problem anymore). The patch is marked "reviewed and tested by the community" which means that it's considered ready to be committed, but hasn't actually been committed yet. We should generally leave this issue alone unless there are additional comments about the patch itself (or if someone tries the patch and finds that there are conditions under which it doesn't fix the bug). That will make it easier for people to follow what's going on with the issue when they go to read it, and hopefully get it fixed as quickly as possible.

David_Rothstein’s picture

Oh, just reread your comment and I may have misunderstood slightly. It sounds like you're saying you don't have a domain yet so no way to visit the site in a browser yet. If so, you really have to set that up to be able to do anything useful with your site (unless you want to start becoming a serious user of Drush). Drupal URLs don't usually correspond to locations in the filesystem - it's all handled via code.

maggielanoue’s picture

I can visit my site in a browser, just need to use the IP number and a tilde with the account name. The html site looks better than the drupal site by far at this time. Since I am in a learning curve I am developing the drupal versions simultaneously. Less stress for me. I can send you the info if you want to look inside but don't want to post publicly. Not much there yet but I do have a database and learned from someone who did what I want to do that I need VIEWS. I am keeping track of everything i learn too. I have to get the database operational on the drupal site by the end of the month because the old host for that is shutting it down.

maggielanoue’s picture

One more idea (sorry for taking over this discussion) I decided to go ahead and try to do this in my top level which is a regular domain name. I figured I could get it going there as a separate searchable database and move it over when everything else was ready. I am still having the exact same issues as in my note #64 above. I will resubmit a ticket to green geeks. It seems to me that it ought to be easy to install a module in a web host that caters to drupal users. I am installing now into a regular domain name site. if anyone has any ideas please do post or contact me.

maggielanoue’s picture

ok - now i got it. the answer was also here https://drupal.org/node/944582
so I didn't need to be in cpanel where i was.
it was right in the dashboard of my drupal site.

all I had to do was type in one tilde where it said here.

Go to 'Configuration' > 'Media' > 'File System'
[*root*]/admin/config/media/file-system
and change the 'Temporary directory' from
/tmp to ~/tmp

i am just spelling it out in case anyone else is as green as I am at this. thanks everyone.
now I will go and learn VIEWS.

catch’s picture

Version: 7.x-dev » 8.x-dev
Issue tags: +Needs backport to D7
PMorris’s picture

Version: 8.x-dev » 7.0
Component: update.module » file system

I am doing a fresh install of drupal 7 on shared host (hostpapa) and getting the same errors as in this thread when cron is run:

Warning: rmdir(/tmp/update-extraction/zen) [function.rmdir]: Permission denied in drupal_rmdir() (line 2277 of /home/glenw073/public_html/test/includes/file.inc).
Warning: rmdir(/tmp/update-extraction/zen/templates) [function.rmdir]: Permission denied in drupal_rmdir() (line 2277 of /home/glenw073/public_html/test/includes/file.inc).

etc

The thing is I changed my /tmp directory to ~/tmp and it has fixed cron error, but for some reason Drupal's built in drag and drop reordering of menu links is now broken. (It sticks when attempting to drag) Is this related to problem with getting this above error on install? Or should i post in separate thread. Never had this reorder problem before with drupal 7 but this is the first shared host I've tried it on. I have no other modules enabled except core ones.

bfroehle’s picture

Version: 7.0 » 8.x-dev
Component: file system » update.module

The menu problem you describe seems unrelated. I'm resetting the issue status.

Anonymous’s picture

Thnks @jaymckinsey

I Solved it when change /tmp to ../tmp into drupal:

Inicio » Administración » Configuración » Medios audiovisuales » Sistema de archivos
admin/config/media/file-system

willem77’s picture

For me it ended up being easy. Drupal 7.0.

I went to the website root, found the /tmp folder, and a created a sub directory called el_tmp.

Then I went to the website, to Configuration/Media/File System and then under Temporary Directory changed the setting from /tmp to /tmp/el_tmp.

Problem solved.

Thanks for all the suggestions.

Timothy Sharpe’s picture

This worked for me too.
I first tried to use the root as I see it in the ftp:
/[websitename]/public_html/tmp
(after I had created the tmp folder)
I then got an error since it couldn't find this folder, the error revealed the full path!
Permission denied in drupal_mkdir() (line 2244 of /storage/content/[secret]/[secret]/[websitename]/public_html/includes/file.inc).
The directory /thesciencepedia.com/publdc_html/tmp does not exist and could not be created.

Changing the temp folder to the full path solved all my problems.
Thank you for helping me keep a fully functional forehead! I was about to start banging the wall.

Dries’s picture

Status: Reviewed & tested by the community » Fixed

The fact that on a shared host, /tmp gets shared with all users may be a bigger problem. This kind of file-conflicts may appear in other parts of the site too. Hence, one could make the case that we should move this kind of unique-site-key-logic to a generic, reusable sub-system.

For now though, I've decided to commit this patch so we can move forward with 7.x and 8.x. We can abstract this later when necessary.

pwolanin, Scott Falconer, Berdir, jhodgdon, a.mikheychik, mtift -- thanks for your contributions!

dww’s picture

Thanks for finally committing this -- nice to know this isn't going to remain broken forever. ;)

For anyone else as confused as I was by the comment, Dries actually committed this patch to the 7.x branch via the following:

http://drupalcode.org/project/drupal.git/commit/02dd3c5

    - Patch #843162 by pwolanin, Scott Falconer, Berdir, jhodgdon, a.mikheychik, mtift: creating vocabularies with machine-names 'List' or 'Add' breaks links in taxonomy overview admin area.

@bfroehle: Sorry buddy! You should have been credited with this fix. It happens. Especially when he's scrambling to clear out a huge backlog...

Anyway, very happy to see this issue finally put to rest.

Cheers,
-Derek

LordQuackstar’s picture

So is this only in 8.x or is it available in 7.x?

dww’s picture

Issue tags: -Needs backport to D7

@LordQuackstar: The fix was committed to both branches. I guess that means we should remove the "needs backport to D7" tag, since this issue no longer needs that. We really need a solution for #1050616: Figure out backport workflow from Drupal 8 to Drupal 7, perhaps involving #66484: Allow issues to be filed against multiple versions/branches..

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

Christopher James Francis Rodgers’s picture

Updated 2013.09.21 FYI:

Drupal 7 work-around solution at
https://drupal.org/node/1106492

katannshaw’s picture

My setup: Drupal 7.15

I see that this was from over a year ago, but I've been running into this problem on my latest install. I didn't get these errors with my 7.13 install. They started only after I updated my version to 7.15.

Whenever I clear my cache and perform other tasks within Drupal, I receive the "Warning: unlink" errors like this:

Warning: unlink(C:\inetpub\wwwroot\drupal\sites\default\files\css\css_-RYDnR0HiAgC8cqARI8LAA0vWwFMlnvijbCNoeisFu4.css): Permission denied in drupal_unlink() (line 2199 of C:\inetpub\wwwroot\drupal\includes\file.inc).
Warning: unlink(C:\inetpub\wwwroot\drupal\sites\default\files\css\css_-RYDnR0HiAgC8cqARI8LAA0vWwFMlnvijbCNoeisFu4.css.gz): Permission denied in drupal_unlink() (line 2199 of C:\inetpub\wwwroot\drupal\includes\file.inc).
Warning: unlink(C:\inetpub\wwwroot\drupal\sites\default\files\css\css_069qXCo8lOiEoO1RiKNVBjG0J-dDyEiv9VrMBRBpyMI.css): Permission denied in drupal_unlink() (line 2199 of C:\inetpub\wwwroot\drupal\includes\file.inc).
Warning: unlink(C:\inetpub\wwwroot\drupal\sites\default\files\css\css_069qXCo8lOiEoO1RiKNVBjG0J-dDyEiv9VrMBRBpyMI.css.gz): Permission denied in drupal_unlink() (line 2199 of C:\inetpub\wwwroot\drupal\includes\file.inc).

...and so on...

So I tried several of your suggestions without success:

Option 1.
#69 - Change "tmp" to "~/tmp"

Resulted in this error:

Warning: mkdir(): Permission denied in drupal_mkdir() (line 2338 of C:\inetpub\wwwroot\drupal\includes\file.inc).
The directory ~/tmp does not exist and could not be created.

Option 2.
#73 - Change "tmp" to "../tmp"

Resulted in this error:

The directory ../tmp exists but is not writable and could not be made writable.

NOTE: I made sure that this directory was completely writable for this testing.

Option 3.
#74 - Change "tmp" to "/tmp/tmp"

Resulted in all of the same "Warning: unlink" errors as before.

I'm a newbie at this, but it appears that this "fix" should have been applied to my version of Drupal, so I'm not sure why this is still happening. Does anyone have any suggestions on how I can get this to work properly?

katannshaw’s picture

Version: 8.x-dev » 7.15
Status: Closed (fixed) » Active

This problem is happening to me in version 7.15. I've made a post on this issue but received no responses. This issue is still open in my case because of the following and what's written on #82 as well:

Every time that I put the tmp file in a location and then reference that location from within Drupal, I receive a message that the folder can be accessed but is not writable. Even if for testing purposes when I set that directory to be completely writable by everyone, I received a confirmation message that everything's working from Drupal's Configuration » Media » File system section.

I can see that it's working because Drupal then automatically creates a matching .htaccess file for the tmp directory that I've referenced. However, I still receive those annoying errors when doing regular maintenance tasks (i.e. clear cache, update, etc.).

Please help!

David_Rothstein’s picture

Version: 7.15 » 8.x-dev
Status: Active » Closed (fixed)

Your issue doesn't seem to be related to the /tmp directory, nor to the update-manager directories within it (which is what this issue was about).

I think you need to look at the file permissions on your sites\default\files\css directory and change them so that the webserver user has permission to write to that directory. I'm not sure what would have caused the directory permissions to change after upgrading from 7.13 to 7.15, but if you can figure out a reliable set of steps to reproduce that, please file it as a new issue...

katannshaw’s picture

@David_Rothstein: Thanks for getting back with me. I wondered about that, and these are the permissions that the sites\default\files\css directory has:

CREATOR OWNER: Just "Special permissions"
SYSTEM: All rights except for "Special permissions"
My Department: All rights except for "Special permissions"
Administrators: All rights except for "Special permissions"
Users: Read & execute, List folder contents, Read, and Write permissions. They do not have "Full Control", "Modify", or "Special permissions"
IIS Users: Read & execute, List folder contents, Read, and Write permissions. They do not have "Full Control", "Modify", or "Special permissions"
Trusted Installer: All rights except for "Special permissions"

So with this setup, what do you see that I need to modify to stop getting those errors? Drupal is able to create files in the sites\default\files directory without a problem. I just get those stupid errors whenever I run regular tasks.

Also, whenever I have updated my version of Drupal (from 7.13 to 7.14 and 7.15), the "sites\default\files" directory always loses it's ability to be written to, and I have to reset that manually. is that a normal occurrence, because I always found that to be odd?

Thanks for your help.

UPDATE:

@David_Rothstein: Thanks again for your response. I (somewhat) solved the problem. I thought more about it, and remembered that these errors started appearing when I updated Drupal from 7.13 to 7.15 (and eventually 7.15). Each time I updated it, the "sites/default/files" directory would give me an error about it not being writable. Each time, I'd have to make that directory writable again.

So I again placed the tmp directory within the files directory, went directly onto the server using IIS 7 Manager, and added write permissions for IIS users only. The errors still appeared, so then I added write permissions for all users of that server. That solved the problem.

Now the problem I have is that I don't want to have full write access for *all* users of the server, right? That seems very insecure. Do you (or anyone else) know if this is how Drupal intended it to be with a regular install, regardless of the server-type Thanks for any help?

David_Rothstein’s picture

In general, yes, that's not very secure (exactly how insecure depends on what else the server is used for and which people/other software have access to it). See http://drupal.org/node/244924 and http://drupal.org/server-permissions for more information... especially if you need help with Windows permissions.

I think if you can reliably reproduce a scenario where the sites/default/files directory keeps becoming un-writeable, it would be worth filing a separate issue about that. Perhaps some module is changing the directory's permissions when it shouldn't.

katannshaw’s picture

In general, yes, that's not very secure (exactly how insecure depends on what else the server is used for and which people/other software have access to it). See http://drupal.org/node/244924 and http://drupal.org/server-permissions for more information... especially if you need help with Windows permissions.

Thanks for the resources. I've seen the first one before, but not the second one. I'll check them out.

One thing that I noted from the first article is that I need to create a new account that has read, modify, and write permissions on that directory. I believe that this setup will help me to create a secure environment for users to upload files to our sites (hopefully) using the LDAP module with Drupal's user accounts by me creating one Active Directory security group, and then assigning only certain users to that group depending on what the LDAP Query pulls in on them. Lets hope it works.

I think if you can reliably reproduce a scenario where the sites/default/files directory keeps becoming un-writeable, it would be worth filing a separate issue about that. Perhaps some module is changing the directory's permissions when it shouldn't.

I should be able to do that pretty easily, as it always happens when I either update the version of Drupal or revert to an older version. Thanks for that advice.

cog-axis’s picture

I fixed the issue in D7 by changing the permission to 777 recursively to all folders and also moved by tmp folder in drupal root directory.

Christopher James Francis Rodgers’s picture

Update:

The solutions I had previously recommended in comment # 81 above (now deleted), that 'jayhawkfan75' tried and listed in comment #82 above, did NOT work universally.

The following, however, was a newer solution that seemed to work in all cases:

All you really need to do if you are having problems is to remove the forward-slash ("/") from "/tmp"
and specify only...

tmp

...for the "Temporary directory" at:
[Your-D7-root-home]/admin/config/media/file-system

Administration » Configuration » Media: File system » see heading: "Temporary directory".

Additional info:
D7 Manual Webhost Install Errors from "/tmp" - the default "Temporary directory" in "Media: Filesystem" - Solution.
https://drupal.org/node/1106492