Important:
This document reflects drush-3.0-beta2 and later versions.

Editor's Note
At the time of this writing, the most recent drush tag is drush-3.0-beta1, which contains a different implementation of the site alias feature.

Please note that Drush has moved to Github.

Drush has introduced the concept of 'site aliases', which are small arrays containing connection details relevant to individual site instances. These are used to note information about your local dev site and a remote staging site (for example).

With this info, drush can help you move content between these locations!
Illustration of the relationship between servers using drush.
After installing drush, have a look at:
drush help site-alias
drush help rsync
drush help sql-sync
to get started.

You will first want to make a note of your local site alias information.

About the local setup

If I am working on a drupal multisite instance called demosite.drupal6.local, my working site directory may be something like /var/www/drupal6/sites/demosite.drupal6.local
I will be wanting to connect to a location called demosite.staging.remote. These are our two 'site aliases'.
As implied, the aliases can actually be shorter names, like '@dev' and '@stage'. In the context of a Drupal site, you can also use the name of the site as it appears in the 'sites' folder. In order for this to work, you must have -r / --root set in your drushrc.php file, or your current working directory must be inside the Drupal root. If you have done this, you can use the site folder name as a site alias, like so:

drush site-alias --full demosite.drupal6.local

This tells us:

$aliases['demosite.drupal6.local'] = array (
  'uri' => 'demosite.drupal6.local',
  'root' => '/var/www/drupal6',
);

Pretty simple so far. There is more to it.

Enter information about the remote target peer alias

We also need to have some information about the destination site. The remote site is (of course) not local, so the local drush knows nothing about it until we tell it. As mentioned in the help documentation drush help rsync, you can read 'example.aliases.drushrc.php' to see an example of how this can be done.

Your own aliases configuration file can be placed in several places. Since the remote site for demosite.drupal6.local is only relevant to this site, I will put it in the local site dir as
{drupal}/sites/demosite.drupal6.local/peer.alias.drushrc.php.

Edit the file (create it if needed)
{drupal}/sites/demosite.drupal6.local/peer.alias.drushrc.php
To add the remote sites information:

$aliases['peer'] = array (
  'uri' => 'demosite.staging.remote',
  'root' => '/var/www/vhosts/staging/httpdocs',
  'remote-host' => 'mystagingserver.myisp.com',
  'remote-user' => 'publisher',
);
Tip: This is documented much more in the example.aliases.drushrc.php distribution.

Tip: Creating the site-alias config array is tedious by hand. If you have a working site, you can just ask it to give you all the details like so:
Change into the site dir of a working site and run

drush site-alias --with-db --show-passwords --with-optional @self

I often go

drush site-alias --with-db --show-passwords --with-optional @self > /etc/drush/mysqit.alias.drushrc.php

and then importantly edit the resulting new files and A: add a <?php tag to the top! B: relabel it from @self to your preferred nickname - which must match the filename you used.

Those extra connection details are required for remote aliases.

If you want, you can also split out the component parts of an alias and use inheritance to construct the peer alias. For example:

$aliases['mystagingserver'] = array (
  'remote-host' => 'mystagingserver.myisp.com',
  'remote-user' => 'publisher',
);
$aliases['peer'] = array (
  'parent' => '@mystagingserver',
  'uri' => 'demosite.staging.remote',
  'root' => '/var/www/vhosts/staging/httpdocs',
);

Check that this information is now available to drush.

drush demosite.drupal6.local site-alias
# Fetch a list of known sites

drush demosite.drupal6.local site-alias --full @peer
# Show what we know about the named site

... it should reflect back the connection details we've entered. Note that because we placed the '@peer' alias inside the site folder for our demo site, this alias is only known in contexts where that site has been named or bootstrapped by drush. Had we put the alias file inside of a more global location such as $HOME/.drush, then it would be globally available. In that case, we would probably want to use an alias name that is a little less generic than 'peer'.

It may feel strange to put the alias context, demosite.drupal6.local in the previous example, on the left side of the site-alias command. A more direct way to specify the context of the @peer alias would be to use relative alias syntax:
drush site-alias --full demosite.drupal6.local/@peer

drush rsync the files

The first time we do this, it will create the remote site entirely from scratch (it didn't exist before). This uses rsync under the hood, so it will do only incremental updates and preserve permissions etc if possible.
drush rsync demosite.drupal6.local @peer

You will destroy data from publisher@mystagingserver.myisp.com:/var/www/vhosts/staging/httpdocs/ and replace with data from /var/www/drupal6/
Requirement: For drush to communicate with remote servers, you must first set up ssh authentication keys to assist automated logins. If you haven't already, go and look into this. It's pretty easy once you know how, and a huge win for productivity. greg.1.anderson has supplied a script that may help this set-up. It only has to be done once per user account per server.

.
..
... Cool. The remote site has all the current files up there now!

Gotcha: This command put up everything, even the unrelated multisites I had hanging around. There are probably tidier ways to do this.

Aside: a fix for multisite sites folder aliases

I'm not using the sites/default directory, which means my local and remote hosts will have a slightly different expectation about the sitename I will use for the site instance. This can cause trouble, but a quick, helpful way to deal with this is to just symlink the site directory with the alternative name.
On the remote site:

cd /var/www/vhosts/staging/httpdocs/sites;
ln -s demosite.drupal6.local demosite.staging.remote

... now the remote site will serve the same content and as the local one. There are other ways to work around this issue, and this approach can have side-effects, but that won't be covered here.

Sync the database

We'll also need to push up the database. It can be done in one line if everything is already set up at the other end.

The remote site will not quite use the same settings, because we'll need different database settings etc. Cleverly, the rsync has carefully chosen to --exclude="settings.php"
(see "drush help rsync")
This means we get to define the remote DB settings separately, as required.

If you've not already done so, set up your database however you normally would on the remote site. This is different between hosts, so probably not automated.
Quickest is to [CREATE DATABASE; GRANT PRIVILEGES; Copy and change the settings.php by hand to edit the $db_url], although you can probably even run the Drupal install wizard if you like.

With the remote database ready to take the content, we'll push it up. This will over-write anything that used to be there.

drush sql-sync demosite.drupal6.local @peer

This command will actually log in to the remote site, and investigate the settings found there to find out what the remote database connection string is. Pretty nifty. You can do other things remotely using this channel also.
This is easiest if drush is also available on the remote site. If you've not already set this up, you can use drush to push itself to a remote server: drush rsync demosite.drupal6.local:%drush @peer:%drush.

You can avoid installing drush remotely by noting the database connection details locally in the site-aliases array.

Gotcha: Depending on how you installed it, the commandline install of the drush program may not be found on the remote site when running a non-interactive shell session. This can be frustrating.
To work around this, the alias file allows you to define path-aliases['%drush-script'] = '/path/to/drush';.
See example.aliases.drushrc.php. NOTE the parameter '%drush-script' must link to the drush shell script file or the parameter '%drush' must link to the drush directory. Either works.

Tip: Pay attention to the advice in example.aliases.drushrc.php on creating a designated '!dump' file path. Note that %dump is a file name, not a folder, and the directory structure to it must already exist.
for mine, I set it to:
'%dump' => '/var/www/vhosts/staging/data/dump.sql',

Gotcha: drush sql-sync is not yet aware of database prefixes and may transfer a heck of a lot more than you wanted.

Done

If you got this far, and worked through any troubleshooting issues, you should now have a working remote clone of your site.
From now on it will be MUCH easier, and you can sync file and DB up and down, selectively or in bulk.

Note: Running any drush command with -y or --yes will assume 'yes' as an answer to all prompts, which is useful when scripting the synchronization or deployment of sites.

EG,
to push up some recent changes to a theme from dev to staging

drush  rsync demosite.drupal6.local/sites/all/themes/demo1/ @peer:sites/all/themes/demo1/

to fetch the latest files that may have been uploaded to the staging site

drush rsync @peer:%files/ demosite.drupal6.local:%files

It probably be a good idea to flush the site caches before transferring the db from local, or on the remote site after a db transfer. drush provides a shortcut for that.

drush cc all

(run on either local or remote system, respectively)

(there are probably more shortcuts also)

Managing remote sites with drush aliases

Once you've got remote aliases set up, you can run drush commands on them directly from your local commandline. Drush will handle to logins and relative paths needed.

drush @peer status

will

  • Check for an alias entry named 'peer'
  • Log in to the remote-site using the given credentials
  • Move to the appropriate directory, or at least use the correct remote paths
  • invoke drush there to carry out the command
  • and return the result

This should work for most drush actions, just by putting an @alias identifier before the commands. Internally, this triggers the functions drush_remote_command() drush_do_site_command() and drush_backend_invoke()

Commands like

drush @peer update
drush @peer cc

Will run DB updates or clear the cache on a remote site without you ever leaving the comfort of your local shell.

To work fully, the remote site must have drush installed also. See the note above about path-aliases['%drush-script'] = '/path/to/drush'; if you are having trouble getting that to work.

Further control over remote syncing

It is also possible to add options to site alias definitions to provide finer-grained control over how the sync operations will be preformed. See the blog post Drush aliases primer for Live->Dev syncing on emspace.

Comments

sholn’s picture

Hi,
nice guide!
just a suggestion (?)
I think you should tell that site-alias and rsync stuff are not part of current stable version but are in HEAD rel (I'd save lot of time :D)

williamn’s picture

Agree. I should read your comment first.

Also drush sql sync should be drush sql-sync

Drush is simply awesome!

dman’s picture

The command names changed in the last week. :-)
Which is one of those things about documenting something still in -dev I guess :-B
... I did not realize that these features were not stable yet.
I'll unlock the page so people can update the docs as needed!

Yoran’s picture

Perhaps this is stupid questions, but if I well understand, for a project with a dev, integration and production platforms, I should have an RC file like this

$options= array(
  'uri' => 'http://dev.my-project.com',
  'root' => '/var/www/dev/my-project',
  'site-aliases'=>array(
    'dev'=>array(
       'uri' => 'http://dev.my-project.com',
       'root' => '/var/www/dev/my-project'),
    'int'=>array(
       'uri' => 'http://int.my-project.com',
       'root' => '/var/www/int/my-project'),
    'prod'=>array(
       'uri' => 'http://prod.my-project.com',
       'root' => '/var/www/prod/my-project')
  )
);

My question is, what is the point of having dev (in this example) information duplicated ? Why not a default 'local' alias using main settings, and just adding remote platforms in alias ?

By the way, why 'alias' ? for me it sound more like 'platform' than 'alias' actually.

dman’s picture

Why do you have

$options= array(
  'uri' => 'http://dev.my-project.com',
  'root' => '/var/www/dev/my-project',
...

outside of the aliases array in the first place? I don't know where that would be used. Was the documentation misleading somewhere?

"alias" is just a word. Me, I think of an individual sites as an "instance". ... and "platform" is a much bigger thing describing what a site is built on. When using multisite, I'd have a dozen site instances on one 'platform'.
It's hard to get the right words when different folk are working on different scales.

Yoran’s picture

Hum, let say that I'm not using site alias for a moment, and that I just don't , and I don't want to enter --uri and --root parameters each time I run drush, so in my ~/.drushrc.php I have this

$options= array(
  'uri' => 'http://dev.my-project.com',
  'root' => '/var/www/dev/my-project',

This way, I fix those defaults for my local installation. When I type 'drush status', it works.

So you suggest that if I put everything in the site-aliases structure, I can do the same ? How ?

Now with site-aliases, as I understand the all thing, I have to add a 'site-aliases' record in order to declare other, let say instances ;-). That's in this way that I see duplicates.

PS: don't take me the wrong way, I just love Drush, and when I ask question... it's just questions, not a biased way to say thing are nicely done.

dman’s picture

So you suggest that if I put everything in the site-aliases structure, I can do the same ? How ?

Well, in the current -dev, yeah I think there is the expectation or need to enter both source and destination each time if working with different sites.
Adding those values as default options - which I understand you doing now sorta makes sense, but it was your idea to start that duplication that you think is unneccessary.

Are you aware that you just have to cd into the appropriate /var/www/dev/my-project/sites/sitename dir to get all the commands to work without the uri and root parameters? It detects the context just fine. See the Tips in the basic drush intro or the drush README.

Anyway, for syncing I too think it's boring to enter those URIs each time, So I've got a private extension to drush described over here that defines pairs or chains of 'peers' or relationships between sites, and an automatic detection of where 'here' is to begin with - and thus where 'there' is. Less typing.
Current code is in CVS

Yoran’s picture

Well, in the current -dev, yeah I think there is the expectation or need to enter both source and destination each time if working with different sites.

My idea is that you rarely (at least me ;-) sync two distant sites. Generally (but again this is perhaps my way of doing this) you have your local playground and you sync from/to a remote (or many) sites. But ok, you want something more general.

Are you aware that you just have to cd into the appropriate /var/www/dev/my-project/sites/sitename dir to get all the commands to work without the uri and root parameters? It detects the context just fine. See the Tips in the basic drush intro or the drush README.

Not that simple when you're have multi-site installation.

dman’s picture

Actually yeah it is that simple and is a huge benefit when managing multisite instances. Every day I go into a particular multisite instance dir and do stuff in that context. I find that easier than string the sitename as a parameter to every drush command.
So that's what I find easy. If you'd prefer longer commandlines you can do that too.

Yes, I too find it rare to want to sync two remotes, but I was providing information about how things were actually working at that time. If you have wishlist ideas, that's a different discussion. Me , I actually use a suite of wrappers to drush that always link a preferred peer so you can run commands like upload-database and the preferences know what to do
The architecture allows a synch between a and b but doesn't care if either or both are local or remote. That is the real abstraction and why the base calls available so far require you to specify both names. Improvements can be layered over that extensible base

tribe_of_dan’s picture

A video of this set up would be great!

phoang’s picture

Yeah, I hope anybody will record a screen cast...

Contact me for Drupal web development or outsource your projects...

John_Buehrer’s picture

I'm wondering if these drush SQL commands can be used on remote sites to which a local mysql can connect, but there is no ssh login access. For example, connecting from a local Unix PC to one of these cheaper shared hosting sites.

I presume the same functionality can be achieved by using the raw mysql / mysqldump commands from the command line on the local Unix PC, but it would by tidier to do these things (if possible) via drush. For example, use of consistent mysqldump parameters.

dana.elza’s picture

Thanks for your tutorial.

These work:
$ drush rsync @local @remote
$ drush sql-sync @local @remote

This works:
$ ssh my_site@xx.xxx.xx.xxx /home/my_site/public_html/drush/drush status
Enter passphrase for key '/home/User/.ssh/id_rsa':
Initialized Drupal 6.19 root directory at /home/my_site/public_html [notice]
Initialized Drupal site www.my_site.com at sites/default [notice]
Drupal version : 6.19
Site URI : http://www.my_site.com/
Database driver : mysql
Database hostname : localhost
Database username : my_site_user
Database name : my_site_drupal
Database : Connected
Drupal bootstrap : Successful
Drupal user : Anonymous
Default theme : my_site
Administration theme : garland
Distribution : Drupal
PHP configuration : /usr/local/lib/php.ini
Drush version : 4.0-dev
Drush configuration : /home/my_site/public_html/drush/drushrc.php
Drupal root : /home/my_site/public_html
Site path : sites/default
File directory path : sites/default/files

Command dispatch complete [notice]

But this does not work (with any Drush command):
$ drush @remote status
Begin redispatch via backend invoke [notice]

Enter passphrase for key '/home/User/.ssh/id_rsa':
The drush command 'my_site /home/my_site/public_html/ ←[31;40m←[1
m[error]←[0m
mysql://my_site_user:password@localhost:22/my_site_drupal
/home/my_site/public_html/drush/drush sql-sync-remote.sql status' could not be found.
Backend invoke is complete [notice]

My alias:
$aliases['remote'] = array(
'uri' => 'my_site',
'root' => '/home/my_site/public_html/',
'db-url' => 'mysql://my_site_user:password:22/my_site_drupal',
'remote-host' => 'my_hosting.com',
'remote-user' => 'my_site',
'path-aliases' => array(
'%files' => 'sites/default/files',
'%dump' => 'sql-sync-remote.sql',
'%drush-script' => '/home/my_site/public_html/drush/drush',
),
);

What am I doing wrong?

drew reece’s picture

Look at the error,

/home/my_site/public_html/drush/drush sql-sync-remote.sql status' could not be found.

It suggests that drush isn't found on the remote server, double check the script path is correct.

dana.elza’s picture

Thanks for taking the time to comment.

Drush was being found, as proven by the straight ssh call

ssh user_name@xx.xx.xx.xx /home/site_name/public_html/drush/drush cc all

which works.

Rather, in drush/includes/backend.inc

@427 $option_str .= _drush_escape_option($key, $value);

this line concatenatates site alias parameters along with commands.

Commented out line 427; et voila drush @remote dl cck views now works fine (or any command.)

Win XP wamp

Too hungry now to figure out why.

mkoistinen’s picture

Perhaps your configuration has the alias to drush (if you're using one) only in your .bash_profile and not in the .bashrc file. This caught me up too. An interactive login will source .bash_profile, but a non-interactive login (via SSH keys, in this case), will source .bashrc.

This would explain why it works when going directly, but not via a remote drush command.

Hope this helps.

Ology’s picture

I realize this is an old post but I ran into basically the same thing and found a solution.
When I ran "drush @remote --debug status" to check the connection the result was ssh ....'drush --invoke --debug --root="ROOT" --uri=URI status'

The 'drush .... won't work it needs to be 'DRUSH_PATH/drush ....

So when I copied the %drush path in my alias file to %drush-scipt and re run I get '/home/SITE/drush/drush --invoke etc.

So copy %drush and make %drush-script and it is all happy.

W.M.’s picture

Thanks for this great guide. I think that at least one particular database table should not be synchronized. It is the "comments" table. Uploading this from a local server to the remote live server would overwrite the recent comments. Correct me if I am wrong ?!

The challenge is to:

a. identify which tables should not be synchronized (local to remote).

b. finding a mechanism that can do that (effortlessly).

Any ideas will be greatly appreciated. Thanks!

Chaulky’s picture

I don't know about automating part a, but part b can be done in the alias config file. This is explained towards the end of the blog post linked to at the end of this article. You can add command specific settings to an alias. In the excerpt below, we add settings to the to the sql-sync command to specify tables to leave out of the sql-sync. You could add the comments table as well. This is all explained further in the linked blog post.

'command-specific' => array(
                'sql-sync' => array(
                    'simulate' => '0',
                    'structure-tables' => array(
                        'custom' => array(
                            'cache',
                            'cache_filter',
                            'cache_menu',
                            'cache_page',
                            'history',
                            'sessions',
                            'watchdog'),
                    ),
                ),
            ),
ice5nake’s picture

Why does Drush default to root if remote-user is empty? Is there a way to configure my alias so that Drush uses the current user?

This documentation would have been better for me if these questions were answered.

skwashd’s picture

posix_getlogin() returns the login of the current user. This should give you what you want for remote-user.

erinclerico’s picture

I have - Drupal 7.x + Drush 4.5 + a multisite config - I want to clone a site from a deployed server to my local workstation using Drush.

Apparently I can either set the 'root' property in my @remote alias like this:
'root' => 'drupal_root/sites/xyz.com' and I will get the correct site folder with 'drush rsync @local @remote' but then 'drush sql-sync' fails because that directory (sites/xyz.com) does not bootstrap

Or if I set it like 'root' => 'drupal_root' so it bootstraps correctly with sql-sync but then I get the wrong set of files for xyz.com with 'drush rsync'

So my question - is there a way to get both working from the same config or shall I just fix it with a linked file in the local sites/ directory (like local.xyz.com -> xyz.com)?

Erin Clerico

Drupal Developer

http://redjupiter.com

dman’s picture

'%root' = $drupal_root as it is labelled.

You then also set
'%site' => 'sites/xyz.com'

And drush will know what you mean.

You can see what drush needs to know when creating your own site_alias setting by going

 drush site-alias --full --with-optional --with-db @self

on the site you need info for. (cd into your sites/sitename folder, then do this)

unitedwallabies’s picture

For different sites in a multi-site setup to have their own modules (or themes), the code needs to reside under sites/sitename/modules/ . This path is stored in the database in various tables (variables, menu_router, system).

How do people deal with sitename changing during transfer? For example, when transferring from dev.example.com to qa.example.com?

We automatically replace the strings directly in the DB-dump, and it almost works except where the altered string is inside output of PHP's serialize() function. In the above dev→qa example, the change must go

from:

s:21:"sites/dev.example.com"

to:

s:20:"sites/qa.example.com"

— changing just the string is not enough in some cases, one also needs to change the string's length or else Drupal will be unable to unserialize() the value...

Automatically finding such cases in the MySQL dump is not easy at all, because mysqldump escapes special characters in the dumped strings... It does not seem like drush can do this (checked in 5.1). Does anyone have a ready solution? Thanks!

dman’s picture

What I've been doing for a few years is set up a structure like:

sites/dev.sitename.localserver
  /settings.php
sites/www.sitename.com
  /settings.php
sites/sitename
  /files/

And soon after installing I get the dev site to use an explicit (shared) files directory (either via UI or settings.php) as just sites/sitename/files
When developing on dev, images get referred to in the common folder.
On release, the live site also points to the common folder = no problems.

Earlier I used symlinks to emulate that, but now I'm in the habit of specifying the files dir because I anticipate this issue.

Also have a look at the D7 sites/sites.php config that now supports subfolder aliasing. That probably could help too, but in my experiments so far, it just means that although the image links don't break, they are in folders named a bit inappropriately, which is why I invent a common ground for both sites.

This is not a drush-migration specific problem, but one that applies to any Drupal migration.

unitedwallabies’s picture

But I need the contents of files/, modules/, and themes/ to be different between the sites... For example, when developing a new version of a custom module, I'll want the new code deployed into sites/dev.example.com/modules/, but not yet into sites/qa.example.com/modules/ -- and certainly not into sites/www.example.com/modules/

The only solution is to use a completely different document root for each site and put all everything into sites/all/ -- completely ignoring Drupal's smart ability to distinguish things based the HTTP Host-header.

But even then the value of site_name in the variable-table may be wrong.

You are right, that this is not a Drush-specific problem. However, Drush, already offering the site-transfer functionality, would've been the best place to solve it...

ADrupalUser’s picture

Why do you need sites/sitename/files? Why not just store everything in sites/www.sitename.com/files?

dman’s picture

Simply because we don't develop on the live site. We have dev and test versions. Content may be built on UAT before a first go-live, and after launch we need to pull copies of the live database down to -dev again. If WYSIWYG or textual embeds link to sites/www.sitename.com/files , that workflow means we get broken links.

But I know that a symlink approach is still not the perfect solution, it's just one way of organising our workflow into a way that the content managers and clients don;t have to worry too much about.
file portability in Drupal still has issues as long as files get stored inside a folder named after the URL AND you have different URLs for dev/test/live.

syam.mohan.vs’s picture

Hi

RSYNC is not natively supported by many shared hosting providers. But there is a way to make this work in shared hosting servers. If you can SSH in to your shared hosting server then the steps shown here - "http://oerinet.net/wordpress/synchronizing-files-with-rsync-on-godaddy-s..." should make RSYNC work for you through Drush. Instructions are for GoDaddy server but this seems to work for other shared hosting providers.

Note that the final rsync command must be issued using "--rsync-path" option provided by rsync command. Eg: drush rsync --rsync-path=~/bin/rsync sites/all/themes/demo1 @dev:sites/all/themes/demo1

Hope this helps.

roblog’s picture

If you do:
drush site-alias --with-db --show-passwords --with-optional @self >> /etc/drush/mysqit.alias.drushrc.php

instead of:
drush site-alias --with-db --show-passwords --with-optional @self > /etc/drush/mysqit.alias.drushrc.php

then it will append the configurations to the end of the file rather than overwrite it. That way you don't have to go back and add <?php. Plus you can do it lots of times for all the sites. Also you get to benefit from keeping all the extensive comments in the file.