Integrate backup.module and Amazon S3 for effortless, automated backups [#103742]

That's the dream anyway. Attached is what I've got so far, which is actually working on a site in development.

A couple of caveats:

This patch takes advantage of Geoffrey Gaudreault's Amazon S3 PHP class (here's the zip download). Just unzip this into the same directory as backup.module. While free, this class is not GPL'd and as such can not be included in the distribution officially. This patch should eventually be upgraded to use dopry's filesystem module which will eventually offer S3 support.
The patch adds a backup settings page at admin/settings/backup which allows you to enable and disable S3, as well as set its frequency. There are fields for the S3 keys, but are not working (see below). I'm sure a lot of discussion could (will?) be had about the settings page could be better arranged. The current arrangement is not optimal (because, for example, you can't make a backup manually if you have S3 enabled), but hopefully this is a step in the right direction.
For some reason, I was unable to figure out how to set the Amazon S3 key id and secret key using the class, and had to put them directly into the class (as instructed by the instructions in the class, so maybe this is by design?). In any case, the fields on the settings page don't actually do anything, and you MUST INPUT THEM MANUALLY :)

Would love to hear your feedback.

-Matt

Comment	File	Size	Author
#4	backup_s3_0.patch	6.64 KB	matt@antinomia
	backup_s3.patch	6.6 KB	matt@antinomia

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Comment #1

dmuth CreditAttribution: dmuth commented 17 December 2006 at 04:02

Holy crap that's awesome!

I investigated S3 awhile ago, but one thing that has stopped me from wanting to do more with it is the requirement of installing PEAR modules (ugh) and then patching the HTTP_Request module. I don't think it's reasonable to expect users to have to do that, nor is that going to scale well.

(On an unrelated note, I personally have found it frustrating that there don't seem to be any decent command line tools for interacting with S3. :-( )

But I think S3 would be great to have in eventually. Maybe we can find an easier way to make it work?

-- Doug

Comment #2

matt@antinomia CreditAttribution: matt@antinomia commented 17 December 2006 at 04:20

Mmm. but PEAR is sweet and juicy :)

I completely agree that this is not going to scale, and is not ready for distribution. But wanted to get it out there in the hopes that people smarter than myself can perhaps help fix the misgivings of this patch. I also forget to mention the whole PEAR/HTTP_Request hack as one of the caveats. My hope is that the work that is being done on the File API will bypass all of this mess. In the meantime, though, this is what I came up with after losing 1000 lines of code this past week, and for me, is well worth the hacking involved in implementing it.

That's a good point about a CLI for S3. I hadn't looked into it at all.

How about this figure? The monthly cost of transferring and maintaining a 50 MB backup 1) hourly for the past day and 2) daily for the past month is about $7.50/mo. Sounds like a value to me.

Comment #3

matt@antinomia CreditAttribution: matt@antinomia commented 17 December 2006 at 04:22

It's also worth noting that the HTTP_Request issue was described as being a "bug" in 1.30. If indeed it is a bug, perhaps it's been fixed in the (current) 1.40 version of the PEAR module.

Comment #4

matt@antinomia CreditAttribution: matt@antinomia commented 17 December 2006 at 04:31

File	Size
backup_s3_0.patch	6.64 KB

Crap, there's a typo on one line of the patch. Please use this attached one instead.

Comment #5

ggaudrea CreditAttribution: ggaudrea commented 19 December 2006 at 15:23

Hi!

I'm Geoff Gaudreault, and I wrote the S3 Class you cite in your description. You can feel free to package the class in your patch, so long as you do not remove the attribution in the source code. I should probably not be so lazy and actually release it under a specific license. If I did, it would be under the BSD License. Regardless, feel free to use it!

Cheers!
Geoff

Comment #6

dmuth CreditAttribution: dmuth commented 19 December 2006 at 15:37

Cool! Thanks for the note, Geoff. That certainly makes things easier going forward.

-- Doug

Comment #7

matt@antinomia CreditAttribution: matt@antinomia commented 19 December 2006 at 18:48

Thanks Geoff, I didn't realize you were active over here at Drupal.org. Do you know why I'm unable to figure out how to set the key or key id using the class? $transfer->keyId = '##SECRET KEY##'; doesn't seem to transfer the variable.

I believe the policy is that it's supposed to be GPL in order to go into the Drupal repository. That's the reason, for example, that you must download TinyMCE from Moxiecode as part of the tinymce.module installation process. Bop me on the head if I'm wrong about this.