Since nobody is answering my handwringing request for a working fileMOVER to S3 anywhere I have taken the issue into my own hands and been pretty successful I think. Attached is a new version of the MediaMover S3 module that makes a unique Complete function (in addition to the normal Storage functions that does nothing else then store your files). This function DELETES your original local file after it was copied successfully to S3. It changes the file table to reflect the new location of the file - and therefore should work with a majority of other file handling routines out there. In addition I added something that I am not sure why it was missing - the folder structure of the original file is also copied over to S3 - while its also left intact locally. That means at one point its possible to make a "move back" function that moves all your files from S3 back to your local server if you are inclined to do so. Also its not a total mess in your buckets.
After all is done your cache is cleared so that themes grabbing the filepath and doing something with it are also updated when you have the cache on.

BE AWARE. I am a lousy programmer never studied it have just picked up PHP because drupal forced it on me and its about the 4th SQL query I have ever written/checked. So please someone with eagle programmer eyes glance over the code and tell me if I am missing something obvious. Also It would be tremendous to have this back on the main MediaMover agenda as a big plus functionality wise.

ALSO BE AWARE: This patch depends on a patching (and has also only be tested with) the content field harvest module so it saves a fileID ('fid') in the $file table. This is essential and I think it was "just" missing from the harvest module by mistake (I have seen references somewhere else that need the 'fid' so I guess it should definitely be there. This patch (along with some other correcting alterations) is found in issue #330649.

CommentFileSizeAuthor
#2 mm_s3.module.patch9.68 KBfalk_g
mm_s3.module.zip3.88 KBfalk_g

Comments

arthurf’s picture

arthurf commented

Hi-
It seems like it would be easier to make a complete function which removes the original file and does the file table updates. However, I'm not sure that you want to update the file table this way because it doesn't support urls and might get screwy at some point. I think you'd be better off writing a theme and form alter function that adds your S3 file during display rather than trying to do this with the files table. In D7 this should not be an issue, and there maybe a new module which will provide a file api which can do URI handling (think s3:// ).

Anyway, can you provide me with the actual patch files for each module and I'll review them?

thanks

falk_g’s picture

falk_g commented
StatusFileSize
new mm_s3.module.patch9.68 KB

patch file (normal diff) attached.

seems like it would be easier to make a complete function which removes the original file and does the file table updates.

I think I did something along the lines but I had to include the "_mm_s3_send" again otherwise it would not work. But generally you have a "storage" and a "complete" function as two seperate things to choose from... (again no clou if I am doing this right)

However, I'm not sure that you want to update the file table this way because it doesn't support urls and might get screwy at some point. I think you'd be better off writing a theme and form alter function that adds your S3 file during display rather than trying to do this with the files table. In D7 this should not be an issue, and there maybe a new module which will provide a file api which can do URI handling (think s3:// ).

I know all the promises of a great file api (I have looked extensively around for an S3 solution) but I needed something now ;) What kind of "support" does the file table not have? I mean in the end it stores a string no? In my limited testing it works quite well so far - but I am only using filefields (no attachments or some sort). Where would you look for any trouble indicators?

arthurf’s picture

arthurf commented

Well the problem starts when any module that accesses your file assumes the file is local (one obvious example is if you have private files enabled). I think the better way to do this is to use the hook_media_mover to add extra data to the node that has the S3 url to it.

This being said, there is a theme layer that you can grab and just render the media mover files that match your configuration- something like:

function theme_show_my_s3_files($node, $cid) { 
  if ($files = $node->media_mover[$cid]) {
     foreach ($files as $file) {
        $path = $file['complete_file'];
        print  l(t('Download!'), $path);
     } 
  }
}

Or something. The thing to do is save the s3 url into the complete_file col. Though this could have similar issues as drupal's file table, no other module relies on it, so youre less likely to have issues.

arthurf’s picture

arthurf commented
Status: Needs review » Closed (fixed)