Frankly drupal's file handling is in need of serious love. I know several people have taken stabs at it... I'm hoping this thread will create a space where we can discuss/document what we want for file handling in 4.8/5.0. I've been giving this a lot of thought lately.
What I've had on my mind....
1) Why files should be independant of nodes...
I would love to see everything in drupal be a node, but I don't think we can really get away with that where files are concerned. a) we do not store their contents in the database. b) we need a fast path like alternative for private downloads to avoid the full drupal boot strap when delivering local files.
2) all files table functions be moved into the file.inc/File API, and to redefine the file api...
file_upload - saves newly uploaded file to Drupal's tmp dir.
file_save($file,$dest) - copies file into Drupal's file area - INSERTS into files table.
file_copy($file,$dest) - copies file to destination - INSERTS into files table.
file_move($file, $dest) - moves file to destination - UPDATE files table
file_derive($file, $dest) - create a derivative file - INSERTS into files table, parent = $file->fid
file_delete($file) - remove file from file system.
file_download($path) - send file to user.
--convert upload module to just manage file<->node relationships...
3) Fastpath file downloads. Store a download key in the database when a file is rendered as a private file
that has a limit to the number of downloads, so we only need session, db, and url parsing functions for
file download.... table file_keys(path,uid,key,expiration)...
4) flexible rendering based on extension, node-type, user role, per node settings.
define a set of themes / extension... then have a default theme per role/node->type, that can be overridden on a per node basis... Theme's could include .mov (quicktime player, list item, inline link), .swf(flash player), .rm(real player), .[jpg|gif|png] (open in new window, list item, thumbnail w/ fullsize popup).
5) file owner, group, and permissions. (works for linux, it can work for drupal)
6) Need an independant fileBrowser to browse unassociated files.
7) file attributes table for storing flexible metadata and searching and sorting by metadata.
table file_attributes (fid, key, value)...
8) hook_file() to hook into each of the file api processes above...
Comments
Make it possible to add freestanding (not linked to nodes) files
I had said...
... but just noticed this is now fixed in HEAD. Fantastic! All seems to work fine.
Looking further into uploads, I would suggest that as a start we take away the need for a file to be attached to a node:
1. We make it possible to add a nid of 0 to the 'files' db table, meaning the file doesn't belong to a node. One good reason for doing this is at the moment, if I create a node and attach a file, I may want to also put an img tag for that file in another node. But if I later delete the node the file was attached to, the file gets deleted from the server too. Not ideal.
2. Maybe on deleting a node we make an option of "delete attached files or unassociate them with a node?"
3. Have upload.module add a menu item to 'upload file' separately from a node. Importantly though, the upload shouldn't create a node, just upload the image. So we don't want to create a new node type of 'image' or anything like that - that's for contrib modules to do. Ideally the menu item would be /node/add/upload "file upload" but of course that would mean it *is* a node type, which isn't what we want.
4. Have a page which lists uploaded files (those in the file db table) with various filters and order by options.
Jake
Drupal web services, London
File Nodes
I'm coming around to the idea that files should really be tracked as nodes. 1:1 mapping, if you delete the node you delete the file. At least in our use-cases, the meta-data that we'd like to track with files closely maps to what's tracked with nodes.
Uploading attachments to a content node would create "file nodes" for each uploaded file. These "file nodes" would be minimal, with enough smarts to render/manage their binary content (e.g. "image files" handle derivatives). This way, even an attached image can have derivatives.
From the visitor's perspective, the "files nodes" should never be directly accessible --- they wouldn't appear in search results, except that any searchable text they might have would be indexed with the content node that referenced them.
I'll see next week how this all works out, heh.
that may be fine for some
that may be fine for some kinds of files, but overkill for many.
making an upload a node is the job of a contrib module, whilst the core should make them freestanding.
Jake
Drupal web services, London
I'm against it
It sounds nice at first, but when you consider what's in the works for the audio module you realize that you'll get a node explosion. There's work in progress to store album art, both as ID3v2 tags and as files, those files would each become a node. There are plans/some prototype code for adding multiple file encodings, it just gets too crazy having a 1:1 mapping. The file API needs to be as minimal as possible with everything non-essential pushed out to hooks/plug-ins.
what we want for file
Excellent work and agree with line of thinking! wish we can implement a interim solution for 4.7 with cck
Also should allow for development of new AJAX based modules similar to the Flickr image snippet inserter module
----
Darly
Ideas
I think this discussion is a good start but it should probably be taking place over in the FileAPI group.