Problem/Motivation

Calls to file_unmanaged_copy() fail when using Drupal 7.7+ and the safe_mode (deprecated) or open_basedir (recommended security) configuration directives. This is due to the copy() PHP function not working properly with stream wrappers and these directives.

Resolution

Create workaround in file_unmanaged_copy() using realpaths if inital call to copy() fails.

Remaining tasks

Patch #65 has been reviewed and tested and needs to be committed to core.

User interface changes

None.

API changes

None as other systems should already be using file_unmanaged_copy() instead of copy().

Original report by jenspeter

I have a problem that I need to decide on fast for different reasons. I made this topic in forum yesterday and have not heard back. I am sorry if this is wrong to add it here but I hope this place on Drupal.org is seen by more.

I have made an update from version 7.4 I think it was to the newest 7.7 of Drupal core.

During the update all worked fine but I got an error with Watchdog_err when I returned to Front page.
Found a solution for that (should be Watchdog_error in files.inc) on Drupal.
Then I found that none of the layout was used.
I solved that by removing the aggregration of the CSS and JavaScribts.

Now my problem is that I cannot update any other module.
I get the following error:

http://ftp.drupal.org/files/projects/google_analytics-7.x-1.2.tar.gz could not be saved to temporary://update-cache-6f83ea9d/google_analytics-7.x-1.2.tar.gz.

But it is a similar problem to all modules I try to update.
I have looked for help on Drupal but I find only the help about setting the right tmp-location and rights for the folder.
I have not changed this and it worked fine before the update.
But even if I change to another folder (e.g. tmp3) I seems the folder is created just fine but I get the same error.
I can see that the file that is downloaded the to folder has the rights 600 and the folders that are created is set as 755.

Any one that have ideas why this is happening?

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

lyricnz’s picture

It looks like your webserver doesn't have permission to write to the temporary directory configured. Remember that the webserver may not be running as the same user as your login session. Could be something like disk-quota/storage too.

marcingy’s picture

Category: bug » support
Jens Peter’s picture

well, why is it different now and not before I updated if it is anything to do with the server.
That don't really make much sense to me. I have lots of Drupal sites running on similar servers.
Please advise where I should look.

And it is for sure not storage as there is lots of disk space.

lyricnz’s picture

No idea. It's probably something to do with your server setup. Hard to diagnose without adding debug code, or looking at your logs etc.

Jens Peter’s picture

you are right but I dont see the logic in a problem that is not present on any of the other Drupal 7 sites but only the one I upgraded to 7.7.
This is an error that happened because of the update so why would you suspect the server suddenly? That has not changed.

lyricnz’s picture

I suspect server/configuration only because that's where the error message points. And the fact that many other people made the same upgrade without issue. Sorry.

VM’s picture

as someone who made this upgrade to 7.7, I can state that I cannot reproduce this issue.

Jens Peter’s picture

I have had my hosting company checking their server settings and they have returned to me saying there is no problem and they cannot help with this error.
I would be very thankful if anyone that get this error to report it here... I feel a bit alone right now :-)
And I don't dare upgrading my other installations in case I get the same error on those.

Thank you for any help or feedback.

VM’s picture

Modules can be updated manually and the UI interface bypassed.

Modules should also not be directly updating on a production site. They should be tested away from the production site.

In administer -> configuration -> file system

what is the tmp folder path set to? If using the server default /tmp

perhaps shifting to sites/all/files/tmp as a test

no offense to hosting support but if you don't check the current tmp folder perms yourself then you really have no idea if the support from the host is correct.

Jens Peter’s picture

seem the problem is known by others too
http://drupal.org/node/1002048#comment-4855698

So it is a problem in Drupal 7.7.
There is a patch there that I will try and test with to see if that will solve my issue.

VM’s picture

Status: Active » Closed (duplicate)

based on #10 closing this issue as duplicate of the one linked in #10.

Jens Peter’s picture

that is ok with me to close this but it doesnt seem the right way.
The one I mention is a bug report for version 8 and my problem is just a solution idea way down that issue.
I have been testing and it is not solving the issue - so now we have an issue that is not open unless we stay on the wrong issue as in 1002048 instead.

I will follow on with the other but this way if this ever get solved, will not be any easy help for anyone getting the same problem.

VM’s picture

bug fixes are fixed in 8.x and then backported to 7.x. There won't be a separate fix for Drupal 7.x whether this issue is left open or marked as a duplicate.

Based on the discussion the patch still needs work. Until the patch is completed, tested, and fixed it won't be put into 8.x or 7.x.

Having two issues open for the same bug, doesn't aid in getting anything fixed any faster. What does aid in getting this fixed faster is for you to test the patches when the are written and help the dev's as much as you can by reporting.

Please see http://drupal.org/node/156119 for clarification of the status component with reference to this issue as well as future and past issue you want come in contact with.

Barney Gumble’s picture

Title: Cannot upgrade modules after update to 7.7 » Cannot upgrade modules, cannot install modules from URL after update to 7.7
Category: support » bug
Status: Closed (duplicate) » Active
FileSize
19.23 KB
15.93 KB
18.14 KB

Greetings to everyone,

I think this issue should be re-opened. It was closed for duplicate with #1002048: Work around move_uploaded_file issues with safe_mode and open_basedir, but it is separate issue and it has AFAIK nothing to do with #1002048: Work around move_uploaded_file issues with safe_mode and open_basedir. I have the same problem, which I firstly encountered when using Drupal 7.7. I will now describe the errors as specificly as posible.

Firstly I had to use this patch for "file.inc" file to avoid #1206916: Notice: Use of undefined constant WATCHDOG_ERR error. After that, I encounter the same error as described in the first post of this thread.

There are three symptoms:

1) Module cannot be updated when using automatic updater (path: admin/modules/update). See the "error01.png" screenshot for error message. I believe that the temporary directory is set correctly in my Drupal 7.7 installation and that the temporary directory is writable for PHP scripts. In fact, there is always a new temporary file (e. g. "filefClZaq") created during the download of the module. The timestamp of this temporary file corresponds well. But this file cannot be extracted to subdirectory ("update-cache-859ece52" in that case), which is subdirectory of "global Drupal temporary directory".

2) Module/Theme cannot be installed from URL (path: admin/modules/install, "Install from a URL"). This symptom is very similar to previous one. See the "error02.png" screenshot for error message. Again, there is a new temporary file with corresponding timestamp created during the download of the module. And again, this file cannot be extracted to subdirectory ("update-cache-859ece52" in that case), which is subdirectory of "global Drupal temporary directory".

3) Default color scheme for Bartik or Garland can be altered, however, the layout is somewhat incomplete. See "error03.png" attachment. It is because the corresponding files (css or graphic files when required) are not stored into "sites/default/files/color" directory.

I assume all three symptoms have something to do with file handling in Drupal. Now some additional information that I found out during further testing. As stated above, this error - all three symptoms - occur in Drupal 7.7. This error did not occur in Drupal 7.4, 7.2 or 7.0 (I did not test 7.5 version). I use shared webhosting running on FreeBSD OS, Apache 2.0.x and PHP 5.3.6 with Safe_mode = On. I was able to reproduce the error (all three symptoms) on local machine, running Windows Vista, Apache 2.2.19 and PHP 5.3.6 with Safe_mode = On. When Safe_mode = Off on the Windows machine, the error(s) did not occur.

This error/errors also did not occur on the shared webhosting in Drupal 7.4 (see parameters and settings above). The settings for "Public file system path" and "Temporary directory" (path: admin/config/media/file-system) and the directory permissions are pretty the same for Drupal 7.4 (error-free) and Drupal 7.7 (in-error) installations.

I will also try to rename this thread and change its settings.

If anyone will need any additional information related to this error, do not hesitate to contact me.

VM’s picture

Version: 8.x-dev » 7.x-dev

Is jenspeter working in safe mode or open base dir as well?

Based on the new poster in #14 who reopened, and states with safe mode enabled the errors can be reproduced and with safe mode off they cannot be reproduced the issue continues to be a duplicate of the previously linked issue as that issue is dealing with file issues and safe mode being enabled.

I'd also like to turn posters to this thread to http://drupal.org/requirements which specifically states that safe mode when enabled may interfere with file uploads.

safe_mode: off. Safe mode may interfere with file and image uploads.

VM’s picture

Version: 7.x-dev » 8.x-dev

moving to 8.x as that's where bugs are fixed.

Jens Peter’s picture

Version: 7.x-dev » 8.x-dev
FileSize
60.86 KB

I am working on safe mode enabled.

My issues are very much like Barney Gumble explains.
I also fixed the watchdog error.
#1 is exactly my problem
#2 and 3 have not been issues for me and I have not been able to test this on the site I am working on for this error.

I have no problem adding files to the site however and all upload seems to work fine for me. I have tested using TinyMCE and IMCE.
To me it looks like something is preventing the uploaded file to be copied/unpacked to a folder inside the tmp-folder.

On the FTP I can see the module file is added and the subfolder within tmp is added too - see picture. I have tested by only choosing one module to update at any time.

VM’s picture

open_basedir should also be checked as to enabled or disabled based on http://drupal.org/node/1002048

lyricnz’s picture

Summary: Original poster is running with safe_mode enabled, despite the requirement that this be disabled - see http://drupal.org/requirements . This may have worked in the past, but is "not supported", and might have changed with newer Drupal. A specific issue is being worked on: #1002048: Work around move_uploaded_file issues with safe_mode and open_basedir that may work-around this limitation in some cases.

It does look like this issue is either "closed (won't fix)" or "closed (duplicate)".

Akaoni’s picture

This definitely isn't a duplicate of #1002048: Work around move_uploaded_file issues with safe_mode and open_basedir as that issue deals exclusively with uploads. They look similar but this issue is centred around Drupal 7.7 (only), Safe Mode, and the temp directory.

Most likely this issue would be "closed (won't fix)" as Safe Mode is deprecated in PHP and not supported in Drupal.
Unless, of course, this issue is also present when the open_basedir configuration directive is set.

That said, I'll do some digging and try to create a patch that those using Safe Mode can apply themselves. ;)

Akaoni’s picture

Title: Cannot upgrade modules, cannot install modules from URL after update to 7.7 » Upgrade/Install from URL Fails with Drupal 7.7 and Safe Mode

Applied issue summary template.
Updated title.

Akaoni’s picture

@jenspeter, @Barney Gumble Can you guys check your error logs (/admin/reports/dblog/) and see if you have any messages like this:
The specified file %file could not be copied to %destination.

You might need to clear the log and perform an install/upgrade by URL again if you have heaps of messages in there.

If not, I reckon there will be some other file related messages in there that will give me more of a clue where the error is occuring.

lyricnz’s picture

People with this issue - can you upload a copy of your phpinfo() please?

naduvko’s picture

I have the same problem, and yes, in the log there are entries such as:
The specified file temporary://fileRjgaa2 could not be copied to public://languages/bg_aaj....

naduvko’s picture

What do you mean by

install/upgrade by URL again

?

Jens Peter’s picture

FileSize
2.57 MB

Thanks for those of you that are not busy just closing this issue down and try helping.

@Akaoni
My log has a set of:
The specified file temporary://filelxTfFL could not be copied to temporary://update-cache-6f83ea9d/metatags_quick-7.x-1.9.tar.gz.
I have tried upgrade different modules. I have not updated any through FTP, but I do expect to be able to do that at any time but with the number of Drupal sites I have running it would be a very big step back for me.

@lyricnz
PHPinfo attached this.
I know that the Drupal specification are safe_mode off but except for minor things it has never been an issue earlier but this time it simply got to technical for me to figure out.
For me the alternatives would be setting up my personal webserver, but it will cost a lot of money and will take very much time.
I could of course change to another system but with Drupal 7 I think again Drupal got up in front of the other CMS out there.

lyricnz’s picture

Did you print, and then scan that, man? Can you upload a HTML version of that? The values are wrapping, and/or obscured. In particular the full value of open_basedir.

Where is your Drupal temporary set to? Please provide the location of the three settings at /admin/config/media/file-system

naduvko’s picture

open_basedir no value no value
safe_mode On On

Public file system path
sites/default/files
Private file system path
sites/default/files/private
Temporary directory
sites/default/files/tmp

lyricnz’s picture

You need to look at the local value for settings, including open_basedir, not the master/default value.

Did you *try* the patch from that issue, to see if it resolves your issue? I didn't check with the update module uses the same underlying functions (but I suspect not).

Edit: removed incorrect assumption about other issue

naduvko’s picture

I tried patch from #1002048: Work around move_uploaded_file issues with safe_mode and open_basedir and this resolved problems with upload files. I still can upload files, but cannot update modules! And this is from 7.7. With 7.4 there was no problem at all.
upload_tmp_dir has no value in my php settings. I'm on shared hosting :(

lyricnz’s picture

I can reproduce all the symptoms (failure in CSS aggregation, module update failure, etc) by fiddling with open_basedir/safe mode etc. It's quite possible some of this stuff changed with 7.7. Investigating a bit more.

lyricnz’s picture

Answer the question - what's your open_basedir

lyricnz’s picture

Without a doubt, it's safe_mode that breaks install-from-URL. I turn it on/off (no other changes), and get the error message

http://ftp.drupal.org/files/projects/coder-7.x-1.0.tar.gz could not be saved to temporary://update-cache-d0fb0dab/coder-7.x-1.0.tar.gz.
Unable to retrieve Drupal project from http://ftp.drupal.org/files/projects/coder-7.x-1.0.tar.gz.

(despite the fact that the file is successfully downloaded to a temporary name in the drupal temporary files directory)

lyricnz’s picture

The problem is basically that safe_mode copy() fails when copying between two paths - in my case temporary://file51V4oc to temporary://update-cache-d0fb0dab/coder-7.x-1.0.tar.gz - when it shouldn't really.

You can apply the same logical workaround as in the other patch. See attached.

Jens Peter’s picture

@Lyricnz
Seems my post returned a lot of questions. Sorry for that.
I didnt scan that. I made the file just normal and saved it as PDF.
The only thing I removed was the domain... mydomain.xx - nothing else and the pdf looks perfectly as it did on my screen when I made it.

My temp is: sites/default/files/tmp
Seems Naduvko had no problem finding the information or he is gussing. What he write it totally correct.

If the patch is the one from the other issue that I referred to earlier - yes, I have used that. Actually it is added and has been for a week or when it was I found that. It made no difference at all.

Open_basedir I am not sure what is. Sorry. Can that not be seen in the PHP info. Or just tell me what to look and I will do it.

I see the last post and the patch. I will look into that.
Just a quick look and it seems that patch is wrong. If I remove the [-] with the [+] it looks wrong but I might be wrong.
If I follow this I will add a new IF not dealing with the first one. And it seems I will add an extra { at the end without adding one to begin with.
I might be wrong here - please advise and I will try it out as soon as I see it.

Jens Peter’s picture

@naduvko
This is the problem I have had all the time. I have no problem adding pictures as part of content.
It is the update module only as far as I can see.
I have not had that problem before 7.7 either.

lyricnz’s picture

The attachment is a patch, describing the *changes* that need to be made to the code.

See http://drupal.org/node/367392 etc

I applied this to d7 here with safe_mode and it resolved the problem with module installs, in particular.

Jens Peter’s picture

@Lyricnz
Yes I know how to patch.
I made the patch and tested it. Same result as before the patch.
Then I deleted the content of the tmp-folder but it didn't change anything.

The error message is the same and the information in the error log is also the same.

lyricnz’s picture

It works for me (*installing* a module via URL), but the issue could depend a bit on PHP versions etc. The line that's actually failing is in file.inc, at the end of file_unmanaged_copy():

  // Perform the copy operation.
  if (!@copy($source, $destination)) {
    if ($real_destination && !@copy($source, $real_destination)) {
      watchdog('file', 'The specified file %file could not be copied to %destination.', array('%file' => $source, '%destination' => $destination), WATCHDOG_ERROR);
      return FALSE;
    }
  }

perhaps try changing it to

  if (!@copy($source, $destination)) {
    if ($real_destination && !copy($real_source, $real_destination)) {
      watchdog('file', 'The specified file %file could not be copied to %destination.', array('%file' => $source, '%destination' => $destination), WATCHDOG_ERROR);
      return FALSE;
    }
  }

(using $real_source and $real_destination on the second line, and removing the error-suppressing @)

naduvko’s picture

Well, after last proposal, my log is clear :) Hope this will solve the problem... I have nothing to update right now to see if it's ok.

Jens Peter’s picture

How wonderful !!!
I just tested the last idea and added the patch.
I have only tested with two modules and they installed perfectly and with no problems.
This seems to have done it.

Thank you for all your hard work, lyricnz. It is very much appreciated.

Akaoni’s picture

Version: 8.x-dev » 7.7
Component: update.module » file system
Status: Active » Closed (fixed)
FileSize
825 bytes

@lyricnz: Great work, mate!! ; )

@jenspeter, @Barney Gumble, @naduvko: Thanks for reporting this and helping fix it!!

Drupal 7.7 patch created, issue updated.

Akaoni’s picture

Issue summary: View changes

Applying issue summary template.

VM’s picture

without this actually being put into the next core release those with the issue will continue to have the issue with subsequent releases of core. I'd suggest pinging the 7.x branch maintainer.

Akaoni’s picture

I'm unsure what you mean by "[ping] the 7.x branch maintainer". ??

marcingy’s picture

Version: 7.7 » 8.x-dev
Status: Closed (fixed) » Needs review

This issue should not have been closed

marcingy’s picture

Adding tag

Status: Needs review » Needs work

The last submitted patch, core-fix_safe_mode_unmanaged_copy-1240256-42.patch, failed testing.

marcingy’s picture

Status: Needs work » Reviewed & tested by the community

Patch in #34 still applies and given the comments in #41 I think this can be marked rtbc. The patch in #42 is not the same as that in #34 but the #34 patch still applies to d7 so that looks like a goer as well.

Akaoni’s picture

Version: 8.x-dev » 7.7
Status: Needs review » Needs work

@marcingy: Patch #42 failed because it is a patch for Drupal 7.7, not 8.x-dev. Patch #34 doesn't solve the problem (see #38). As such, this is not RTbC.

If you (or someone else) wants to try to get this fixed in Drupal core, despite safe_mode not being supported, that's cool with me.
Make sure you update the Issue Summary and best of luck. ;)

Akaoni’s picture

Version: 8.x-dev » 7.7
Status: Reviewed & tested by the community » Needs review

#42: core-fix_safe_mode_unmanaged_copy-1240256-42.patch queued for re-testing.

Re-test against Drupal 7.7.

Akaoni’s picture

Version: 7.7 » 8.x-dev

Re-test sent.
Moved back to 8.x-dev.

marcingy’s picture

Version: 7.7 » 8.x-dev
Status: Needs work » Needs review

Sorry my bad :(

marcingy’s picture

Well if this patch is not intended for core then it should be set to won't fix rather than remaining in the issue queue tbh. And instead the appropriate function on api.drupal.org should be updated with a comment explaining how to get round this issue.

VM’s picture

IIRC you won't get a test against 7.7? doesn't it have to be against 7.x-dev since fixes don't go directly into 7.7

asking for my own knowledge ...

Akaoni’s picture

Is no sweat!! ;)

webchick (7.x branch maintainer) pinged so she can weigh in.

@VM: Not sure - I'm from the camp of "try it and see". ;)
Edit: Looks to have worked!! $our_knowledgebase++

lyricnz’s picture

Status: Needs review » Needs work

The patch above (from my suggestion) is not commit-worthy - it has the possibility of consuming errors that need to be emitted. I'll make a better one.

lyricnz’s picture

Attached patch applies to both D7 and D8.

lyricnz’s picture

Status: Needs work » Needs review

CNR

lyricnz’s picture

Version: 8.x-dev » 7.x-dev

Go testbot, go.

Akaoni’s picture

Version: 7.x-dev » 8.x-dev

If you want to try to get this commited to core, I'm pretty sure it should be commited to 8.x-dev then backported to 7.x (see "Needs backport to 7.x" tag). ;)

webchick’s picture

Issue tags: +Needs backport to D7

Fixing tag.

Akaoni’s picture

Thanks webchick.
If I could remember where I copy-pasted it from I'd fix that too. ;)

Akaoni’s picture

Title: Upgrade/Install from URL Fails with Drupal 7.7 and Safe Mode » file_unmanaged_copy() Fails with Drupal 7.7+ and safe_mode or open_basedir
Priority: Normal » Major
Status: Needs review » Reviewed & tested by the community

OK, I've *finally* managed to look at this, and I can confirm that it is also a problem when open_basedir is set.
I got confused and thought this wasn't the case - sorry, my bad. :(

Applied lyricnz's patch and all works as expected.
Based on this and other people's testing I'm marking this RTbC.

Updated title and issue summary.

Akaoni’s picture

Issue summary: View changes

Issue fixed.

sun’s picture

Status: Reviewed & tested by the community » Needs work
+++ b/includes/file.inc
@@ -906,8 +906,11 @@ function file_unmanaged_copy($source, $destination = NULL, $replace = FILE_EXIST
+    // Attempt to work around PHP issues with safe_mode and open_basedir.

I'm sorry, but in 6 months from now, no one (including everyone of you on this issue) will be able to remember, recall, and know to what kind of "WTF" this comment refers to.

14 days to next Drupal core point release.

Akaoni’s picture

Rerolled lyricnz's patch with more comprehensive comments.

Akaoni’s picture

Status: Needs work » Needs review

Run patch through QA again.

Akaoni’s picture

Issue summary: View changes

Issue is relevant to open_basedir directive.

Akaoni’s picture

Issue summary: View changes

Patch needs improved comments.

Akaoni’s picture

Status: Needs review » Reviewed & tested by the community

Patch #65 has no functional changes so moving back to RTbC.
Updated issue summary.

bfroehle’s picture

+++ b/includes/file.incundefined
@@ -907,8 +910,12 @@ function file_unmanaged_copy($source, $destination = NULL, $replace = FILE_EXIST
+    // If the copy failed and realpaths exist, retry the operation using them

I think the comment could still be better. For example, something similar to the comment in drupal_move_uploaded_file() might be appropriate:

PHP's copy() does not properly support streams if safe_mode or open_basedir is enabled. If the copy failed, retry the operation using the files' real paths, if available.

Another option would be to create a new wrapper, like drupal_move_uploaded_file(), for the copy operation. I think this isn't necessary as file_unmanaged_copy() already serves as that wrapper.

Akaoni’s picture

@bfroehle: drupal_move_uploaded_file() is what I based these new comments on however I moved the detailed bit up into the file_unmanaged_copy() function's description. Happy to move it down or duplicate it here if you think that's best.

A new wrapper function was my first thought too, however I agree that file_unmanaged_copy() is essentially a wrapper already.

catch’s picture

Damien Tournoud’s picture

Status: Reviewed & tested by the community » Needs work

If I understand correctly, this is a bug in PHP, or at least some versions of it.

The comments in the code should point to the issue open on https://bugs.php.net/ in which this bug is being handled with, and detail which version(s) are affected.

If (as I suppose) nobody has cared to do this research yet, we need to:

  • Confirm if PHP trunk and PHP 5.4-dev are affected (you can use my nightly packages of PHP 5.4 to do so)
  • If this bug has been fixed, identify when it has been fixed, and which historic versions of PHP are affected
  • If this bug has not been fixed, we need someone to search for issues of bugs.php.net, and open a new issue if the search is unsuccessful

After doing our homework, we can have a workaround in core.

lyricnz’s picture

@Damien: this issue covers similar ground as #1002048: Work around move_uploaded_file issues with safe_mode and open_basedir which has already been applied. Suggest that the research you describe above could be better added to http://drupal.org/node/515192 and referenced from here (it is already referenced from the other patch).

FWIW, I wouldn't be surprised if the same issue is elsewhere, as most core devs don't run with safe_mode or open_basedir.

lyricnz’s picture

Possibly related PHP bugs

Bug #46888 copy() : safe_mode / allow_url_fopen does not allow opening urls
https://bugs.php.net/bug.php?id=46888

Bug #51949 open_basedir and move_uploaded_file does not harmonize!
https://bugs.php.net/bug.php?id=51949

bfroehle’s picture

I agree with Damien about doing our homework and submitting a proper PHP bug report.

For another issue I implemented a basic PHP stream wrapper (much like the ones Drupal uses) -- see #1270862-18: ArchiverZip::extract() needs workaround for open_basedir & stream wrappers. This has the benefit of being Drupal-like without requiring a Drupal installation. It can be easily modified to provide a test script for this issue, by changing the parts at the end which relate to ArchiverZip and replacing them with some copy() calls.

Once we have a working test script we can try it on other platforms and PHP versions easily as Damien suggests.

Akaoni’s picture

@Damien Tournoud, bfroehle: Guys, I'm all for digging deeper into this issue (and related safe_mode/open_basedir/streams ones) - I really am. But not at the cost of getting a fix committed for this particular issue. It might not be the most ideal or maintainable solution but it does fix a major issue. Surely commiting this then creating a task linked to these issues to look deeper into it is the way to go??

@lyricnz Thanks for all your work so far on this issue. ;)

onco_p53’s picture

I can confirm that this fix works for a Drupal 7.8 installation running php 5.2.6-1+lenny13, with php safe_mode on, on a Plesk shared hosting environment.

I know that safe_mode should be off, but we don't all have this option, this seems like a simple fix that should make it to the core.

czigor’s picture

I can confirm that #65 is working for D7.9.

Akaoni’s picture

@onco_p53, czigor: Thanks for testing, guys!!

Akaoni’s picture

I had the thought that maybe because Drupal registers stream wrappers as STREAM_IS_URL, turning this off or turning on the allow_url_fopen PHP directive may solve this issue globally.
No dice unfortunately. :(

Next step would probably be to dig into the Drupal Stream Wrapper classes to see if this is caused or can be fixed there.

Unfortunately I can't commit anymore time to this one. :(
Hopefully someone can pick it up or we settle for a less ideal, but implementable now solution.

acp’s picture

Took me 3 hours of searching/testing/tweaking to retrace this whole bug... That's what you get when you don't search drupal's issues ;).

FWIW, replacing only the source uri by the corresponding realpath seems to be enough (tested with both uris being temporary:// ones).

lyricnz’s picture

Status: Needs work » Reviewed & tested by the community

This issue needs to be fixed, whether or not we get the PHP bug fixed. Another example of the same problem was already fixed in #1002048: Work around move_uploaded_file issues with safe_mode and open_basedir - it's silly to hold this one up for more research, while real people are getting bitten. Reverting to RTBC as it was prior to #71

chx’s picture

Status: Reviewed & tested by the community » Needs work

I agree with Damien that we need to identify the PHP bug and link to it in the patch. It's fine to code around a bug but in case we want to remove this hack from D9 or D10 we need to have a pointer. It happened already (check_plain) and I expect it to happen in the future. I am not asking that anyone go into the PHP source and check that way. Just please reproduce the problem with a very simple script, file it as a bug report -- or identify with one of the existing ones and add a @TODO: remove when bugs.php.net... is fixed.

bfroehle’s picture

For another issue I implemented a basic PHP stream wrapper (much like the ones Drupal uses) -- see #1270862-18: ArchiverZip::extract() needs workaround for open_basedir & stream wrappers. This has the benefit of being Drupal-like without requiring a Drupal installation. It can be easily modified to provide a test script for this issue, by changing the parts at the end which relate to ArchiverZip and replacing them with some copy() calls.

borazslo’s picture

Also I can confirm that #65 is working for D7.9. Thanks for everybody!!!

hkkdt1’s picture

#65 is my Life Saver. I don't know why it's not included in 7.10.

bfroehle’s picture

#65 is my Life Saver. I don't know why it's not included in 7.10.

This bug here is with PHP, not Drupal. The patch in #65 is a band-aid to get Drupal working again, and I'm sure it could easily be included in Drupal if the instructions in #71 or #82 were followed. Perhaps you could do the requisite leg work?

lyricnz’s picture

LAME

lyricnz’s picture

FileSize
1.85 KB

Here's a script that demonstrates the problem using plain PHP.

lyricnz’s picture

Submitted another useless bug report to PHP - https://bugs.php.net/bug.php?id=60456

lyricnz’s picture

Status: Needs work » Needs review
FileSize
1.52 KB

Reroll for D8 change, and add reference to PHP bug above. No other changes.

bfroehle’s picture

I was under the impression that this bug could be triggered with safe_mode off and only open_basedir set...? At least that's what the Issue Summary says. Of course it should be noted that Drupal System Requirements includes the following PHP requirement:

safe_mode: off. Safe mode may interfere with file and image uploads.

lyricnz’s picture

I am unable to reproduce with open_basedir only in effect, though #63 reports this. Fix is trivial, and enables Drupal for these cases (it used to work).

bfroehle’s picture

Status: Needs review » Reviewed & tested by the community
lyricnz’s picture

Thanks

catch’s picture

As far as I can see the only person who's reproduced the open_basedir issue is Akaoni, yet we have a code comment talking about PHP bug with this. I'd like to either remove that reference from the comment and open a follow-up issue, or preferably get Akaoni to confirm he can reproduce that bug with open_basedir (and no safe-mode) before committing this.

muka’s picture

Hi,
I've tried both safe_mode on|off, but the problem persist and server logs say nothing about that.
In my case there is either mod_security enabled on apache2,

I get css/js cache working ONLY if I delete then recreate the css/ and js/ folder from an ftp account and set them 777, all by hand.

Seems that the drupal created folders causes strange behaviors when the $destination or $real_destination is going to be copied.

Akaoni’s picture

Sorry, folks - was on holidays. ;)

Great to see this one might finally get committed!!!

@muka: The fact that you can fix your issue by recreating the folders manually suggests it's not related to this issue. Perhaps search for another related issue and, if there isn't one, open one?

@catch: Yes, I can confirm that this is an issue with open_basedir enabled and safe_mode disabled.

I just retested this with Drupal 7.10 and the Windows versions of PHP 5.2.17 and 5.3.8 - maybe it only affects open_basedir on Windows?

catch’s picture

Version: 8.x-dev » 7.x-dev
Status: Reviewed & tested by the community » Patch (to be ported)

Thanks Akaoni!

OK that's enough for me to get this committed. It seems like we could use some work trying to isolate exactly which environments and configurations this fails on, but that can happen in a follow-up issue at this point.

Committed/pushed to 8.x, moving back to 7.x, will need a quick re-roll for that.

Boobaa’s picture

Status: Patch (to be ported) » Reviewed & tested by the community

The patch applies cleanly to latest 7.x (fdf48f2b9962e04529e6b1a4bb3252ed9c01fc15) without offsets.

muka’s picture

This issues is related to open_basedir and safe_mode, which affect my installations too.
Using realpath of file for copy() isn't enough in my case so I've shared a solution, nothing more.

murfi’s picture

I can confirm that this is an issue with open_basedir enabled and safe_mode disabled with Drupal 7.10, PHP 5.2.6 on Debian (lenny)

The fix at #65 worked for me; thank you

webchick’s picture

Status: Reviewed & tested by the community » Fixed

Cool, thanks for filing the upstream bug and for the additional testing. Looks good to go here, too.

Committed and pushed to 7.x. Thanks!

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

sfyn’s picture

totap’s picture

#65 works great!!! thx!

totap’s picture

Issue summary: View changes

Latest patch