Not having people locked into Aegir (or the provider offering the service) is a key feature we should aim for. From the user's perspective, a site is an abstract object to which they have no fundamental access.

I think this is a two-part process:

1. provide people with HTTP (or else) access to their backups
2. provide people with a way to rebuild the hosting platform

#1 can be done by doing a hardlink between the ~/backups and ~/hostmaster-x.y/sites/aegir.example.com/files directory, with a random filename as a target. since directory listings are not allowed, this will just work. We will need to automatically delete that file after it's downloaded, not sure how that should be done.

#2 can be done by including the makefile associated with the platform (if any) or running drush make-generate on the site (?) and including that makefile. The makefile could be in the tar file...

Any other ideas? note that we're not talking about importing those in aegir, this is in #322788: generic import mechanism for external database dumps and site backups (yes, that's old).

Comments

Anonymous’s picture

I guess this is a good idea for those offering Aegir as a service, where the users don't have shell access.. and the fact that we don't really even split off backups on the server itself separate from other sites.

But I'm a little uncomfortable with putting backups in the files/ dir, random name or otherwise.

Can we introduce a 'Save backups' task for users that:

1) takes a passphrase as an option in the task form
2) Sends 'provision-backup_save $string-of-backups' like backup_delete does
3) encrypts those tarballs with GPG using that passphrase
4) drops those into the files dir
5) *maybe* becomes a good time to implement http://drupal.org/project/hosting_backup_gc and use that to also do the cleanup/deletion

I wondered if we could save ourselves the trouble and see if we can get https://github.com/computerminds/aegir_backup_storage going - another one by darthsteven, which lets people store the backups in S3 or other offsite locations, they can worry about the security themselves in that way.

Just thoughts, feel free to disagree

anarcat’s picture

So I understand the desire to make the backup service more secure, but I do not understand how GPG-encrypting the backup will make it so.

If the following are true, I believe that a file with a random name (ie. where the key is the name of the file) is secure enough:

1. the file name is known only to the user (ie. it's hidden behind HTTPS and not shared or showed on the screen)
2. the directory where the file is stored is not readable (but it's executable so you can still download from it)

What's the attack vector on the above?

I agree this would require backup garbage collection however.

Anonymous’s picture

I suggested GPG so as to provide a method with which only the user themselves that asked for the backup would have the appropriate passphrase or whatever to decrypt the file.

It strikes me that putting the user in control there is better than coming up with a random filename, putting it somewhere anyone can access, and just crossing our fingers that no-one can guess what it's called :)

So we would depend on HTTPS? That's a big ask.

You have me at 2) though. If the dir is not listable then it's not possible to be polling and fetching anything within there that is found.

Anyway, there's no reasonable attack vector if the filename can't be guessed I suppose. Will be interesting to see what you come up with ;)

anarcat’s picture

Let's say the filename is 20 characters long, with alphanumeric and some shell-agnostic characters. Heck, we could even add a microsecond timestamp here.

Let's say that the directory is not readable, so that an attacker would need to do a HEAD or GET on the guessed filename for the download to succeed.

This has two interesting property:

* unless the server is bruteforced, only the person that knows the URL of the file can access it
* bruteforce attempts can be detected by the administrator in the logfiles

GPG, on the other hand, has less seductive properties:

* bruteforce attempts can be carried offline, so on bigger machines and without the admin's knowledge
* the password is chosen by the user so it's potentially (a) weaker or (b) written down somewhere

(I assume we're using symmetric encryption here - this of course doesn't apply if we encrypt to the user's GPG key, but then we get into fun key management issues.)
My proposal is to have a /var/aegir/backups/exportable or ~sitedir/files/exportable directory with special permissions. A series of spikes could accomplish this:

* create the special directory (mode 710, aegir:www-data) somewhere during some verify (server? hostmaster site?)
* add a special flag to provision-backup to hardlink the backup to that directory
* add a form item in the frontend to mark some backups as exportable
* garbage collect exportable backups after X (configurable?) time

I am not saying GPG is a no-no, but I don't think it's *required* as a first spike. It would also add another dependency - maybe we can keep it optional, as an add-on?

Oh, and as for HTTPS - if your aegir install is not protected by TLS, you have other issues than your backups being disclosed. :P

Anonymous’s picture

Right, you win :)

I agree re: GPG bruteforce attacks can be carried offline so that's a weaker point. I don't agree that it's a problem if the user writes their password down: i don't care if the user compromises their own security, so long as I (Aegir) doesn't do it for them

I also don't think it's ok to tolerate a possible bruteforce because an admin can read logs: the same way that I don't beleive it's ok to leave SSH open to the world so long as I can tail auth.log.

But! I'm arguing semantics here and I'm not all that concerned, and I like the plan. One question: currently /var/aegir/backups is not executable by www-data meaning the web user can't get into it: will we run into problems trying to serve files from inside there, even if the subdir has correct perms, and there are hardlinks etc. (Maybe doesn't matter, we could put the dir somewhere else outside of ~/backups and do the hardlink?)

Anonymous’s picture

The more I think about it, the only real way you could provide backups to 'end users' that don't have access to servers directly, is probably:

1) provide the link for them to download it
2) email the backup to them

And since 2) probably passes through too many hands already to get to the user, I suppose 1) is really our best option

anarcat’s picture

The fact that ~/backups is not accessible by the webserver is exactly why I am suggesting hardlinks (otherwise we would have a vulnerability already and we could have used symlinks, which are more intuitive).

Hardlinks have this interesting property that the two hardlinks (pointing to the same file) can have different modes and sit in different directories.

So even if ~/backups/example.com.tgz is not accessible to the webserver, if I hardlink that to ~/hostmaster/sites/example.com/files/backup-export/87djku3887ahfdsjufdsh.tgz, that file *can* be accessible to the webserver.

It's the whole idea here.

I'm not against mailing to the user if it's more convenient (people do that kind of stuff anyways) but the problem is the files are too big...

Anonymous’s picture

Cool! I wasn't sure if hardlinks work around the permissions issue or not.

And yes: another strike against email the tarball: large tarballs :)

Anarcat 10 / mig5 0 :) Let's go with your suggestions!

omega8cc’s picture

As a side note (but still related if you think about the extended meaning of exportable/importable pair here) here is an interesting group effort: http://groups.drupal.org/site-archive-format.

bgm’s picture

Another option: put the backups in the clients/ directory and allow users to download them by sFTP (assuming they can have sFTP, but it can be an interesting complementary option, especially if the client has a large site).

anarcat’s picture

Assigned: Unassigned » anarcat

Great idea, I'll work on this right away.

anarcat’s picture

Status: Active » Fixed

Now, in 2.x:

Executing: tar cpf - . | gzip -c > '/var/aegir/backups/test2.orangeseeds.org-20110418.193309.tar.gz'
Re-cloaking database credentials after backup [0.69 sec, 14.62 MB]                                                                                                                                          [notice]
Template loaded: /usr/share/drush/commands/provision/platform/provision_drupal_settings.tpl.php [0.69 sec, 14.63 MB]                                                                                        [notice]
Changed permissions of /var/aegir/hostmaster-HEAD/sites/test2.orangeseeds.org/settings.php to 640 [0.69 sec, 14.64 MB]                                                                                     [message]
Generated config Drupal settings.php file [0.69 sec, 14.64 MB]                                                                                                                                             [message]
Changed permissions of /var/aegir/hostmaster-HEAD/sites/test2.orangeseeds.org/settings.php to 440 [0.69 sec, 14.64 MB]                                                                                     [message]
Change group ownership of /var/aegir/hostmaster-HEAD/sites/test2.orangeseeds.org/settings.php to www-data [0.69 sec, 14.64 MB]                                                                             [message]
Deleted mysql dump from sites directory [0.69 sec, 14.63 MB]                                                                                                                                               [message]
Backed up site up to /var/aegir/backups/test2.orangeseeds.org-20110418.193309.tar.gz. [0.69 sec, 14.63 MB]                                                                                               [success]
Client backup directory for anarcat path /var/aegir/clients/anarcat/backups has been created. [0.73 sec, 14.64 MB]                                                                                         [message]
Client backup directory for anarcat ownership of /var/aegir/clients/anarcat/backups has been changed to aegir. [0.73 sec, 14.64 MB]                                                                        [message]
Client backup directory for anarcat permissions of /var/aegir/clients/anarcat/backups have been changed to 750. [0.73 sec, 14.64 MB]                                                                       [message]
Client backup directory for anarcat path /var/aegir/clients/anarcat/backups is writable. [0.73 sec, 14.64 MB]                                                                                              [message]
Created symlink /var/aegir/backups/test2.orangeseeds.org-20110418.193309.tar.gz to /var/aegir/clients/anarcat/backups/test2.orangeseeds.org-20110418.193309.tar.gz [0.73 sec, 14.64 MB]                    [message]

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

ergonlogic’s picture

Version: 6.x-0.4-alpha3 » 6.x-2.x-dev
Status: Closed (fixed) » Needs work

#10 & #12 are a great solution, assuming sFTP access. Also, I hear it's been backported to 6.x-1.x. But all the discussion leading up to that point was about providing a downloadable backup from within the hosted site, no?

As for the second point in the original issue description("2. provide people with a way to rebuild the hosting platform"), I recently posted: #1372612: Keep a copy of makefiles when creating platforms, #1372614: Include platform makefile with site backups and #1372622: Import sites from backups (which I suppose is a dupe of #322788: generic import mechanism for external database dumps and site backups, and have marked as such). Also, all of this would be made more-or-less moot with #1138882: Support drush archive-dump.

helmo’s picture

Some work on downloading a backup from hostmaster is being done in a sandbox by Gerben Spil - http://drupal.org/sandbox/gerbenspil/1459830

My main concern is still the file permissions, see #1468444: File permissions on backups

Project application: #1462440: Hosting Site Backup Manager

Gerben Spil’s picture

#12 is already available in 1.6, so I can use that to fix the problem in #15.

Steven Jones’s picture

Status: Needs work » Closed (fixed)

http://drupal.org/project/hosting_site_backup_manager can provide exportable backups with a nice UI, otherwise SFTP access as per #12 is the way to go.