Amazon S3 storage adapter

jackmaninov - November 20, 2008 - 03:47
Project:Bitcache
Version:6.x-1.x-dev
Component:Storage adapters
Category:feature request
Priority:normal
Assigned:Arto
Status:postponed
Issue tags:Amazon S3, scalability
Description

Hey guys. Great module, I'm in love with fileframework and its use of this backend.

My file repository is starting to get large (about to get much larger!) and I'm hoping to migrate it to S3. I know bitcache is supposed to support S3 as a backend, but haven't been able to find any example configurations or documentation to work off of. Since my file repository will eventually grow to larger than my server, I don't want to just use bitsync to sync against S3, I want to actually be serving off it.

The only way that I can guess to do this would be to set up a repository against a S3 stream wrapper, but this doesn't seem like it would use bitcache's (the ruby module, not the drupal project) ability to use S3 as a backend.

I'm willing to pitch in development time to work on this, but I don't know where this fits into the current strategy of the module.

Thanks!

#1

Arto - December 1, 2008 - 07:15
Assigned to:Anonymous» Arto

#2

true-pal - February 23, 2009 - 20:42

+1 (Supporting Amazon S3 without Drupal DB load i.e. direct upload to S3 server)

#3

Arto - March 17, 2009 - 20:01
Title:Is Amazon S3 supported as a backend?» Amazon S3 storage adapter
Version:6.x-1.0-alpha2» 6.x-1.x-dev
Component:Documentation» Storage adapters
Category:support request» feature request
Status:active» postponed

Good news for you guys: this is coming soon, probably as of 6.x-1.0-beta3 (that is, after the imminent 6.x-1.0-beta2).

While you could already, as jackmaninov stated, use Bitcache with the PEAR Services_Amazon_S3 package that provides an s3:// stream wrapper for PHP, that has had some issues; notably the performance isn't great, as the S3 paradigm doesn't map cleanly onto the Unix file system concepts that PHP's stream wrapper API expects.

The upcoming S3 support in Bitcache will be more specific, more performant, and suitable for seriously large repositories (currently being tested with repositories containing about a million objects, and we'll keep pushing that higher).

If you look in the current CVS development version (soon to be 6.x-1.0-beta2), there is already support added for several new storage adapters (that is, repository backends), including MySQL, PDO and GDBM. The S3 backend will work in a similar vein.

#4

asak - April 23, 2009 - 16:54

Subscribing... sounds great!

#5

rhangelxs - May 15, 2009 - 18:09

Subscribing

#6

XerraX - August 17, 2009 - 01:22

Subscribing

#7

JerryH - August 21, 2009 - 19:56

Subscribing

#8

LegacyAtty - October 21, 2009 - 19:46

It's been 7 months since Arto said this would be coming "soon".

Just wanted to check to see if there was an update on the status of the Amazon S3 adapter. Is there an ETA?

#9

sean69 - November 24, 2009 - 17:15

subscribing

 
 

Drupal is a registered trademark of Dries Buytaert.