In #1173644: Use winrs when using "drush ssh" command on Windows, we discuss adding a drush ssh command and --bastion argument to that command to allow people to execute the following flow:

  • SSH to a predefined site
  • Forward the key through a bastion or intermediary server
  • End up on the target command line

After speaking with both bjaspan and msonnabaum, we believe it would be useful to do a similar dance for drush aliases. I am willing to write this functionality. I just need a couple of pointers as to where I should begin.

The end result would be that users could run drush @site [command] --bastion=foo.bar.com (or define it in their aliases file) and have it successfully return the result of the command.

Comments

greg.1.anderson’s picture

Sounds pretty cool, but is it necessary to support as a drush feature? See http://backdrift.org/transparent-proxy-with-ssh for an example. It seems that if @site has 'remote-host' => 'myhost.com', then you could just make an entry in your .ssh/config as shown in the above link where Host myhost.com has a ProxyCommand to foo.bar.com.

It is not clear to me how to do this via commandline options to ssh (instead of a ProxyCommand in .ssh/config); however, if you provide an example of how to do this, it would be pretty easy to add the option to drush's backend invoke mechanism.

moshe weitzman’s picture

The bastion stuff is in the category of "I'll take it if it just costs a dozen lines of reasonably simple code". If it is more than that, then it isn’t worth it IMO. Quite a specialized need.

greg.1.anderson’s picture

It's probably about half a dozen lines in backend invoke if it is supported via ssh commandline args. If we have to connect to the bastion host and run netcat "by hand", then it's not worth it -- use .ssh/config.

Recipes welcome.

greg.1.anderson’s picture

From man ssh, it looks like you could drop -o "ProxyCommand ssh user@foo.bar.com nc %h %p" on the ssh commandline, and in theory that will use foo.bar.com to connect to whatever hostname you are trying to ultimately reach (e.g. myhost.com).

How do you do that? Well, you could add 'ssh-options' => '...' to your site alias, and it should work the way you expect it to.

If someone wants to try this out, then 'bastion' => '...' could be shorthand for "append a -o ProxyCommand to my ssh-options", and could be implemented in just a few lines of code.

webkenny’s picture

I'll give it a run. It's just been my experience with larger enterprises that bastions are all the rage. I agree it's kind of edge case. I have to agree with you both even though I proposed it. If it's more than a few lines of code and a test or two, it's not worth it. :)

I'll report back shortly.

webkenny’s picture

Title: Provide --bastion argument to drush for aliases » Provide recipe for connecting to remote hosts through a bastion
Component: Base system (internal API) » Documentation
Assigned: Unassigned » webkenny

Did some testing of the above scenario passing -o to the ssh-options - Got the following output which makes me believe drush is attempting to directly access the 2nd hop.

tengigabitethernet6-1:~ kennys$ drush @rh.int-stg st -vv
Load alias @rh.int-stg                                                                                                                            [notice]
Begin redispatch via backend invoke                                                                                                               [notice]
executing ssh -o "ProxyCommand  ssh root@drupal01.intranet.stage.int.foo.com nc %h %p" username@sshalias                                   [notice]
'/mnt/drupal_content/drush/drush  --uri=home.corp.stage.foo.com --root=/mnt/drupal_content/www
--db-url='\''mysqli://root@localhost/database'\'' --drush-script=/mnt/drupal_content/drush/drush --dump-dir=/tmp --v st 2>&1' 2>&1
Calling system(ssh -o "ProxyCommand  ssh root@drupal01.intranet.stage.int.foo.com nc %h %p" ksilansk@rh-phx2 '/mnt/drupal_content/drush/drush  --uri=home.corp.stage.foo.com --root=/mnt/drupal_content/www --db-url='\''mysqli://root@localhost/database'\'' --drush-script=/mnt/drupal_content/drush/drush --dump-dir=/tmp --v st 2>&1' 2>&1);
ssh: connect to host drupal01.intranet.stage.int.foo.com port 22: Operation timed out
ssh_exchange_identification: Connection closed by remote host
Backend invoke is complete                                                                                                                        [notice]
tengigabitethernet6-1:~ kennys$ 

I thought originally that perhaps the lack of -A was causing the key not to forward but the error made me think different.

Anyway, still working on this and I think it will probably end up as a recipe in the documentation as these initial passes are making me think this could be trickier than it's worth.

webkenny’s picture

Well look at that. The reason I thought it was trying to connect to the backend host first was that it was. :) The ProxyCommand should be the bastion (the proxy, duh) and the drush definition in the alias should be the backend server's hostname and user. This gets even slicker when using SSH config files.

I'll write up some cohesive documentation to this end. Where do we prefer recipes go? Does it make sense to write this as an example in the drush package's doxygen docs?

greg.1.anderson’s picture

I think that the recipe could go in examples/example.aliases.drushrc.php file.

moshe weitzman’s picture

Perhaps docs should go in example.aliases.drushrc.php.

greg.1.anderson’s picture

I was just using this today. I can easily do something like this in my .ssh/config file:

Host destination.server.ip.address
ProxyCommand ssh user@bastion.server.ip.address nc %h %p

Works great!

I can also put something like this in my drush alias file:

$aliases['destination'] = array(
  'remote-host' => 'destination.server.ip.address',
  'remote-user' => 'wwwadmin',
  'ssh-options' => ' -o "ProxyCommand  ssh user@bastion.server.ip.address nc %h %p"',
  // other stuff here...

However, I need one more feature. I'd like to put in the 'ssh-options' conditionally, so that my laptop will use the bastion server when I'm away from work, but will go straight to the server when I'm in a specified network (e.g. inside the corporate intranet).

In man ssh_config, there is this intriguing reference to a 'from' attribute in the 'pattern lists' section:

     A pattern-list is a comma-separated list of patterns.  Patterns within
     pattern-lists may be negated by preceding them with an exclamation mark
     (‘!’).  For example, to allow a key to be used from anywhere within an
     organisation except from the “dialup” pool, the following entry (in
     authorized_keys) could be used:

           from="!*.dialup.example.com,*.example.com"

I can't get this 'from' thing to work. With From="!172.?.?.?", I get the following error message:

/home/ga/.ssh/config: line 2: Bad configuration option: From

If this isn't an ssh feature after all, then I could either (a) make a wrapper script for ssh that specifies a separate config file based on my network settings, or (b) use drush features to conditionally include options in my alias records. The ssh config from option would be the best and simplest way to get this functionality, though, if it works. Can anyone confirm or deny this ssh feature?

greg.1.anderson’s picture

Status: Active » Needs review

I stopped short at looking at the source of ssh, but it seems there isn't a way to make .ssh/config react conditionally to the network environment you are operating in. So, this is what I did instead:

In my drushrc.php:

# Figure out if we are inside our company intranet by testing our gateway address against a known value
exec("route -n | grep '^0\.0\.0\.0' | awk '{ print $2; }' 2> /dev/null", $output);
if ($output[0] == '172.30.10.1') {
  drush_set_context('MY_INTRANET', TRUE);
}

In my alias file, servers.aliases.drushrc.php:

if (drush_get_context('MY_INTRANET', FALSE) === FALSE) {
  $aliases['intranet-proxy'] = array(
    'ssh-options' => ' -o "ProxyCommand  ssh username@bastion.server.ip.address nc %h %p"',
  );
}

Then, any server that needs to go through the Bastion just has an entry 'parent' => '@intranet-proxy'. This will add the Bastion proxy command to any site that needs it, but only if you are outside the intranet that it lives in.

If this technique looks reasonable to everyone, I'll write up some documentation. I'll probably add a bastion server topic to drush topic, and reference it from example.aliases.drushrc.php.

moshe weitzman’s picture

looks reasonable to me.

greg.1.anderson’s picture

Status: Needs review » Patch (to be ported)

Committed http://drupalcode.org/project/drush.git/commit/4da1d73 to master.

I think these instructions should all work in drush-4.x, so this could be backported if desired. Just needs testing on 4.x.

webkenny’s picture

Greg, this is a killer addition to the documentation! I love the network-aware piece you added. I'll test this out on a drush 4 install a little later today and we'll see if we can get this back ported (I think we can, since I started off on this road with drush 4, but I'll let you know) - If it works, I'll roll a patch for 4.x

This is why I love this community. Good ideas become great ideas in the hands of the contributors.

greg.1.anderson’s picture

Now I am using drush to manage my non-Drupal ssh connections, punching through my firewall (conditionally) with ease. Be sure to try the ssh alias in examples/example.bashrc in master. If you add your own alias alias ssh='sshd' then you can use ssh @mydrushalias to skip over to your internal machine. Must say it's quite nifty -- thanks for turning me on to ssh's bastion features.

greg.1.anderson’s picture

Thanks for your great blog post on this feature.

moshe weitzman’s picture

Issue tags: +Needs change record

Tell folks about new docs

moshe weitzman’s picture

Issue tags: -Needs change record

Added change notice: http://drupal.org/node/1407114

msonnabaum’s picture

Status: Patch (to be ported) » Fixed

Docs backported.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.