Posted by Lausch on July 27, 2008 at 3:58am
Jump to:
| Project: | Media Mover |
| Version: | 5.x-1.0-alpha1 |
| Component: | Code |
| Category: | feature request |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | active |
Issue Summary
hi guys,
it is possible with this module to support file mirroring to my site?
for exampe if someone upload a file medamover will mirror it on some other servers (flp?!)
regards
Lausch
Comments
#1
Yep. Media Mover can pick up ether local files or remote (via ftp) and put them somewhere else.
#2
thanks, but this isnt what i mean...
mirroring! not relocating...
you upload a file and it will be mirrord to many different servers
and if you want to download a file or can choose a mirror or it will use a "amutmatic load balancing system" to choose a mirror itself...
regards
Lausch
#3
Sure it would be possible to build this with media mover, but doesn't exist right now. Basically what you'd do is create a server manager which stored account information for each of the mirrors, a media mover storage operation which moved the files, and a display handler that would keep track of which mirror to use based on some criteria (probably time). The only really yucky part about this is that storing server account data in the drupal database is not so secure- it could be done with ssh keys which is the secure way to do it, but one would have to build a complicated interface to make this possible.
If you have a strong interest in this, I could definitely work on laying some ground work and maybe some other developers were interested.
FYI, I posted some additional information about this here: http://www.24b6.net/2008/08/27/ssh-key-manger-module
#4
hm thanks for your post arthurf,
very interesting...
but i think this could be easy in another way, "maybe" also without media mover...
what about this:
a new module....
if an user upload a file drupal will store it to the "mainserver" (like original method)
then drupal on "mainserver" will mirror the files via cron to the other servers... (ftp or rsync)
and some little db stuff... (add file references for example or a "mirror choosing script")
code this should pretty easy. why you want thinks make so "complicated"?
what makes you sure that your "idea" is good or maybe the best?
we only need a module with "4 main functions"
add mirrors
fetch files from drupal files dir
mirror it to added mirrors
(add file reference or ...)
and
a load balancing script that choose a downloadserver
or
a mirror choosing script (rapidshare like)
what do you think?
regards
Lausch
#5
Cross posting my comment from the blog post:
Considering that you will need to work on the command line to create the vhost and user on the remote user adding the ssh key to that users trusted key file seems a pretty minor addition - I am not sure it is worth the effort to write a GUI interface for this part. If you have the vhost/user creation scripted then it seems like it would be easy to add the key as part of the script also.
In terms of transferring the actual files then I would suggest looking at the CDN module (http://drupal.org/project/cdn), which is designed for this I believe - although it seems like it would be pretty easy to add this as a 3 line cron job or a mediamover hook. [Another option is to add the files via an authenticated http post, in the same way as mediamover does for S3.]
In terms of the wider use case, I am not sure how important this kind of functionality is - in general a modern server can saturate a huge amount of bandwidth serving static files, so you would generally be limited by your peak bandwidth (i.e. $$$) in most cases. The main advantage you could gain from this is in managing very large numbers of files (more than 1 servers worth) and/or locating the files nearer the users by distributing geographically. I think the latter problems are beyond the reach of Drupal really though :)
#6
Closing this ticket- this functionality is possible with Media Mover, but not sure if there is interest to make it happen.