Jump to:
| Project: | Drupal core |
| Version: | 7.x-dev |
| Component: | file system |
| Category: | feature request |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | closed (fixed) |
Issue Summary
Spinoff from #874326: Invalid scheme fails to halt processing of a stream request
Streams are a way to make it so that all kinds of files and streams look the same to an API.
We added public, private, and temporary for that reason. And PHP has file (the default), ftp, and about 8 other standard schemes. (PHP (in my installation) provides https, ftps, compress.zlib, compress.bzip2, php, file, data, http, ftp, and zip.)
The current code (http://api.drupal.org/api/function/file_stream_wrapper_valid_scheme/7), determines whether a scheme is valid based on whether it was registered with Drupal rather than whether it's a "valid" scheme. Perhaps the name of that function was intended to be something like file_stream_wrapper_authorized_schema(), but it seems odd that when we have this nice, very generalized PHP setup for streams, that we treat it like it's our territory only.
I should mention that all of the standard streams are easily accessible from Drupal code.copy('/home/rfay/Desktop/something.txt', 'ftp://someuser:somepass@example.com/something.txt');
andfile_get_contents('http://drupal.org')
work absolutely as expected. It's just the File API stuff that has hiccups (and apparently some deliberate restrictions).
And I should mention that a standard filepath is a stream (the default scheme is 'file'). Current code croaks on file:// and on regular filepaths. That seems counterintuitive unless it's exactly what we want to do.
So:
1. If this is intentional and for security reasons, there are probably some comments that should be added to that effect. I would suggest function name changes but it's not worth doing at this point.
2. If it's not intentional, we should probably unroll it, as it's quite an awkward intrusion into the PHP world.
Comments
#1
See DrupalStreamWrapperInterface. We extend the standard PHP StreamWrapperInterface because the standard interface doesn't provide everything we actually need.
#2
The goal is also to make the Drupal API more controlled than the generic PHP API - e.g. like we sugest you use the Drupal database API.
#3
@Damien: The question here is why can't you use any scheme. Why does ours artificially restrict it.
@pwolanin: The database API is far more *like* general PHP streams than it is like our current restrictive "Only Drupal streams can be used" approach. Database API allows you to access many different backends without knowing what you're accessing. That's the idea of streams too. And we disallow anything but our own.
I am *not* arguing the case here. That's for you all. I just want a clear statement. I'm fine with "disallowed because of security". It's not terribly hard to fix the File API to use any scheme.
#4
The database API case is similar to the stream case; in both the cases an existing API has been extended. Damien already explained why the stream has been extended.
AFAIK, it's not possible to implement a class to handle a stream that is already handled by a PHP extension; if that would be the case, then Drupal could have defined a new stream handler for ftp, in example.
#5
Shucks, we're not getting to the right point here. Drupal specifically and arbitarily *prevents* already existing PHP streams from being used by the Drupal File API. It's not a matter of having to extend anything. Drupal *adds* streams into the PHP framework, which already works fine. It's just in the File API, most specifically in file_stream_wrapper_valid_stream(), we arbitrarily call all streams that are not added by the Drupal framework "invalid".
All I want here is for us to say "that's what we wanted, for sure" or "no, maybe we should pull our the 10 or 15 lines of code that prevent regular streams from working".
#6
@kiamlaluno - you should be able to re-implement the ftp one to make it accessible via Drupal as far as I know. Part of the general stikciness is that PHP doesn't actually define an interface or classes for the stream wrappers - it just tell you what methods they implement.
#7
The limitations are intentional.
The FileAPI has requirements beyond those of the typical php stream interface, we extended it. If you wish to create wrappers for all of the native stream handles that implement the additional requirements of the DrupalStreamWrapperInterface, by all means please do.
Additionally the streams are limited to those known by Drupal to provide a more consistent developer experience when handling files within the FileAPI.
The ability to use copy, move, unlink et all is intentional so that you can use that native PHP interfaces if you choose.
You're completely disregarding the strata of the API here...
At the lowest level of the API you have the native PHP stream aware functions. These can also leverage the stream wrappers registered by Drupal.
The next layer up, the unmanaged files layer which gives you the ability to use the drupal extended stream wrappers, and supports some cool functionality such as copying between schemes and enforces some drupal centric behavior on how your files are handled.
Up another layer we hit the managed files api which gives you Drupal's enhanced stream wrapper support + keeps the database in sync with your file operations.
I suggest you read the code more closely and maybe consult with some of the people who designed or implemented the FileAPI. It's clearly stratified to meet the needs of a number of development styles and to provide as much additional functionality at all layers of the API as possible.
Does that help answer your question? The basic answer is if you want native PHP functionality, use the native php file functions, do not use the fileapi. Just about everything in the File API is there for a reason, even if you don't quite grok it. It may take 2-3 years of working with Drupal 4.5, 4.6, 4.7, 5.0. and 6.0 to really understand all the bits and pieces in there and the reasons for them.
#8
Sounds like we need some architectural/design documentation to come out of this, if nothing else. "Read the code more closely and consult with File API designers" and "Spend 2-3 years working with Drupal 4.5 -> 6.0 to understand" are not really acceptable answers for your average Drupal dev who needs to understand and work with this new API.
#9
The actual resolution to this is in #874326-36: Invalid scheme fails to halt processing of a stream request. The decision is to allow all schemes in File API unmanaged operations, and only Drupal schemes in File managed API operations. Sorry I didn't get back and update this one.
#10
@dopry, I did want to respond to your #7.
The reason for this issue was in fact to consult with the people who wrote this, and because so much time had passed (a year since this went in) nobody could completely remember the decisions made.
I should point out that streamwrappers are an extension of an old, old technique pioneered in Unix and that they're not strictly a layering technique. The idea is to give access to many kinds of stream-oriented devices using a single set of system calls. So this technique is a bit outside the traditional idea of layering.
There is very, very little in the code that prevents arbitrary streamwrappers from working with the Drupal managed and unmanaged APIs. And there is no documentation I've discovered that explains why those few things are done that way. The Drupal streamwrapper class requires just a tiny bit more than regular streamwrappers, and it doesn't actually look like it's there necessarily to support the managed or unmanaged APIs.
Anyway, we seem to have a direction over in #874326: Invalid scheme fails to halt processing of a stream request, where you've also checked in, and your knowledgeable assistance will be awaited. Also, since you've obviously got a lot of background here, your review and improvement of http://drupal.org/node/555118, http://drupal.org/node/877212, and http://drupal.org/node/877214 will be much appreciated.
#11
Automatically closed -- issue fixed for 2 weeks with no activity.