It would be lovely if aegir had a way of knowing whether a server was configured with a reverse proxy like squid or varnish. In such a case, a site might be configured to run on port 8080, but would still need to be contacted via port 80.

Until such time as aegir can handle varnish configuration, a checkbox and textbox for reverse proxy options might be helpful to make aegir generate correct login links to the sites it manages :-)

Comments

cafuego’s picture

I've added a very basic proxy_server module that implements a `proxy' service configured on a server node. Currently all it does is store a reverse proxy port number, so that I can tell Aegir that apache runs on a different port, allowing it to create the correct vhost definitions and not output incorrect links in the _hosting_site_url() function.

https://github.com/cafuego/hostmaster/commit/ddcea46a44febb9a6ea7f98b406...

I am not entirely sure which version of aegir I have modified locally, so I doubt you can pull these changes in directly. If you reckon this is a good idea, let me know and I can redo the work against HEAD on the aegir git repo.

anarcat’s picture

Status: Active » Needs work

This is what the cluster stuff is supposed to do. I understand that cluster mode is broken right now but could you clarify what's missing in there that your patch implements?

cafuego’s picture

The version I have looks to implement a site running on multiple servers, but I don't see anywhere that mentions proxying on the same server as the one that runs the http service. All cluster does is give me a radio button that removes the port and restart command form items :-)

Anonymous’s picture

Yeah I am not sure cluster is the solution here when all that's been trying to do is change the url:port combo in the Login link etc so that it doesn't point to the direct port apache is running on but under the port of the reverse proxy that apache is hiding behind.

I think what anarcat might be saying is that you could theoretically:

1) define your apache as being on port, say, 8080,
2) Make a cluster server node on port 80 and attach that port 8080 apache server to the cluster (can we even only add one server to a cluster or is there a minimum of 2 required?)
3) Create your platforms/sites against the cluster server instead of the apache server

Cafuego: took a quick look at your commit. There's a 'hosting_web_server.module' inside the proxy_server dir. Should that be hosting_proxy_server.module, and in fact should the functions inside it be hosting_proxy_server.module too? Wouldn't that create redeclared function errors with the existing hosting_web_server stuff? (it looks in fact like a copy of hosting_web_server.module entirely - it might be in there by mistake)

anarcat’s picture

@mig5 re cluster - yes, I think that's the idea behind the cluster module. If it's broken, we need to fix it or ditch it, but certainly not reinvent the wheel. :)

@cafuego - can you test what mig5 is suggesting here and report back here?

Those are the two issues I can find with the cluster mode right now in the trackers, so keep those in mind:

* #869878: In cluster servers, platforms are not getting deleted (orphaned platforms in web cluster)
* #1016890: Create Server as cluster is broken after save

The latter has a patch to test and the former should be tested for confirmation...

Besides: it looks like you're running a proxy on port 80 and apache on port 8080. Why not just configure the webserver to run on port 8080 and let Aegir ignore the proxy? What would be wrong with that?

Anonymous’s picture

Besides: it looks like you're running a proxy on port 80 and apache on port 8080. Why not just configure the webserver to run on port 8080 and let Aegir ignore the proxy? What would be wrong with that?

I think his point is that then Login links, welcome emails etc for the site will pull in the :8080 port.

What if port 8080 is only listening on an internal interface and isn't reachable from the outside world (except via the proxy on port :80)

I don't consider it a real critical problem but it I think the point is that it would be nice to have some sane handling of such things.

If Cluster works with it like this, we will really have to document the crap out of it supporting this feature, because I would *never* look at the Cluster feature to try and fix this problem (unless I read that it could be used for this).

cafuego’s picture

There's a 'hosting_web_server.module' inside the proxy_server dir. Should that be hosting_proxy_server.module, and in fact should the functions inside it be hosting_proxy_server.module too? Wouldn't that create redeclared function errors with the existing hosting_web_server stuff? (it looks in fact like a copy of hosting_web_server.module entirely - it might be in there by mistake)

@mig5: Indeed. Removed :-)

@anarcat: I've tried what mig5 suggested. The patch from #1016890: Create Server as cluster is broken after save didn't apply, so I manually made those changes. After doing that, I can create a cluster node and attach my web server node to it. The cluster node is verified successfully (which it wasn't before applying the above patch).

What if port 8080 is only listening on an internal interface and isn't reachable from the outside world (except via the proxy on port :80)

Indeed. That's is precisely my setup. Opening port 8080 up to the outside world and emailing it to users sort of defeats the point of having a reverse proxy to begin with.

I've just created a site to test via the platform attached to the cluster and the generated links are now correct. Or rather, they default to port 80 with no way to change it ;-) So I reckon that:

... we will really have to document the crap out of it supporting this feature, because I would *never* look at the Cluster feature to try and fix this problem (unless I read that it could be used for this)

would be a good thing.

Though adding this to the sites I already have in aegir is a LOT more painful than simply injecting the proxy config as per my patch.

Beanjammin’s picture

I ran into this issue in the wee hours this morning and would like to offer the following suggestion on how the solution could be structured.

I believe there are two separate components to the issue: 1) the ability to work with a web server on a non-standard port and 2) cluster support.

The reason I don't think cluster is the right place for the proxy / non-standard web server port configuration is that a single server running a caching proxy and web server is more likely to run into the non-standard port issue than a cluster as there's no technical reason why a cluster would need to run apache on a non-standard port whereas in a single server set-up there is no other way.

anarcat’s picture

I think I understand the problem better...

I am not sure how to cleanly fix that issue. I feel unconfortable in creating yet another server type for this, because that's exactly the kind of thing that the cluster stuff should be fixing.

Maybe we want to have a different behavior when a webserver node is associated with a cluster node? Instead of checking if proxy == true, maybe check if we have a parent cluster instead?

SqyD’s picture

I am researching to code tight Varnish integration into Aegir, including varnish configuration with provision. So far I am just diving into drush provision. Good I found this issue. The functionality described here would be a good foundation for the hostmaster part of things. This is the functionality I am aiming for in the long run:

- A proxy service type
- A port setting.
- A proxy type setting. This can be a generic proxy without further configuration, just the functionality as described in this thread so far.
- Can also be Varnish type that enables provision to generate varnish.vcl configurations and reload the varnish config.
- Allow for a cluster of reverse proxies.
- Allow to configure sites to be aware of the proxy server(s) through varnish and purge module settings.

Just a cluster type won't do for this. There are more possible proxy sub classes possible then just Varnish like pound or nginx. This basic service type sounds a good foundation.

I didn't test it yet but will do so over the weekend, thanks!

SqyD’s picture

Update on my digging on this issue so far:

I would like to support to following use cases:
- Unmanaged proxies: All traffic passes through unchanged to a single (cluster of) webservice(s). No provisioning/configuring needed.
- Managed proxies: Reverse proxy servers that can be managed and may choose their backends from more webservices and change traffic depending on the platform.
- SSL enabled proxy (clusters).
- Single web server with a straight reverse proxy in front of it.
- Multiple web servers with multiple proxies, including multiple proxies on one server, with or without a local web service.
- Make platforms that support it aware of the proxy service. Provisioned Varnish/Purge settings etc.

I see two ways of handling this:
1: Create the proxy service as proposed.
Pro:
- Straight forward approach when use cases are simple.
Con:
- Increased complexity when use cases become more complex.
- A lot of code duplication from the http services.
- Other parts of Aegir (dns, need to know about the proxy settings too.

2: Make reverse proxy a type of http service and make derivatives of http(ssl) and cluster types.
Pro:
- Will be less messy once complexity increases.
- Less code duplication.
- Less changes to other parts of Aegir
Con:
- To be practically useful it will depend on the possibility to configure multiple services of the same class on a single server instance. I think this feature request #881962: Use Cases: One server with multiple service instances on different ports, although much more of an edge case, has that same need. I will post some more thoughts on this over there.

Since starting on approach 1 and learned more about the http code the more I am convinced approach 2 is the way to go. I will get some code together and post it here once it's concept working.

SqyD’s picture

Another update:

So far I've created very rough, hardly functioning versions of most of the components needed for proper reverse proxy support.

Hostmaster features that provide:
- A basic proxy service. It extends the http service class. It has a port and has a setting to select 1 backend http service. It assumes it's configuration is not managed by aegir and cannot distinguish between backends and just sends it to one.
- A proxy server setting on the platform node form. It allows per platform selection of proxy servers. It validates so only dynamic and proxy services that have selected the webserver of the platform can be selected.
- A varnish proxy service . It extends the basic proxy service. It also has a restart and a "dynamic" backend setting that allows varnish to choose backends based on it's provisioned configuration files.
- I think I will also create a simple apache proxy service as a reference implementation. nginx, pound, etc should be doable after.

Provision
- A varnish http service type.
- It manages a conf.d file structure like the parent and glues it together before reload since Varnish has no wildcard includes.
- Templates for varnish that allow for
- Redirects of aliases
- Backend selection
- Platform and site specific includes
- Configuration reloads without downtime
- Future ideas: web_cluster aware load balancing, health checking, ssl-offloading, clustering...

This all still is nowhere near functional and needs a lot of debugging but I believe 80% of the needed code is there. The remaining 20% has a few tough ones.

What I still need to figure out, and could use some help with to speed up my effort:
- How to get the proxy setting in the drushrc files of the platforms.
I think I need to also make a provision service for the basic proxy class, even though that does not need configuration files, right? Where to stick the proxy setting on the hostmaster side?
- How to trigger provision to generate/manage the proxy config files?
- How to make the selection proxy form do aha/ajax filtering so only valid proxy server selections are displayed?
- What would be the cleanest way to make other parts of aegir aware of a proxy server? At least dns and the frontend that generates the url need to be in the loop. Am I missing any?

I would appreciate any help, pointers to other sources and also general comments on my approach. Does this sound like a good idea to all of you or am I doing something completely insane here? (I sometimes feel it's the last ;-) If we manage to pull this one off Aegir can make a big leap in performance, scalability and thus cost effectiveness.

lloydpearsoniv’s picture

subscribe

izmeez’s picture

subscribing

helmo’s picture

subscribe

Dane Powell’s picture

I'm having similar problems - Apache runs on 8080, and Varnish on 80. In order to get the login links that Aegir generates to work, I have to allow access on port 8080. This is not ideal, but it's better than nothing. Unfortunately, many networks block port 8080, so depending on where I'm working, I still can't use the login links.

I'm also having another problem that I think is related- I'm not able to use SSL on sites. Because the Aegir master server (/ Varnish) is on 127.0.0.1, the vhost file for SSL-enabled sites has the line <VirtualHost 127.0.0.1:443>, which I think should really be <VirtualHost *:443> or <VirtualHost [external ip]:443>. Can anyone confirm that this problem is related, or is a misconfiguration on my part?

Dane Powell’s picture

Version: 6.x-0.4-alpha3 » 6.x-1.2

Updating version...

anarcat’s picture

@spyd - this sounds like a great idea! I would love to see the code you come up with, and I'm sure all the people here would love to test it for you. :)

cafuego’s picture

The simple patch in comment #1 would solve the incorrect link problem you're having. It doesn't address the vhost definitions, though.

The problems are related in the sense that the IP you specified for the web server is the local IP, so that's what it'll use :-) I don't know how easy it would be to allow for a single server definition with multiple IP addresses specified in it; 127.x and the external IP.

A sneaky fix would be to portforward [external ip]:443 to 127.0.0.1:443 via say iptables on Linux (if you're using that).

iptables -t nat -A PREROUTING -i ethX -p tcp --dport 80 -j DNAT --to 127.0.0.1:8080
Dane Powell’s picture

Very clever "fix" cafuego- I'll have to try that!

SqyD’s picture

I got completely stuck on this issue and have abandonded my previous approach.

During the Aegir Dev BOF session at Drupalcon London we discussed this issue. The idea that came up was to create "subservices". I created a separate issue for this. #1260844: Refactor the web service to implement subservices

Steven Jones’s picture

Version: 6.x-1.2 » 6.x-2.x-dev
mccrodp’s picture

Where are we with this issue with regard to track 1.x and will it be a feature in 2.x? I can't see it in any of the 2.x or 3.x notes on the aegir community site. Let me know if I can help with this feature.

I followed mig5's steps in #4 and my cluster server at 127.0.0.1 was created and verified successfully for me without using any patches. When installing however the apache server port 8008 seems to lead to the following error lines.

parse_url(http://test12.domain.dev:8008user/2): Unable to parse URL boost.module:472
WD varnish: Authentication to server failed!
WD varnish: Recieved status code 101 running purge req.http.host ~ test12.domain.dev && req.url ~ "^/http://test12.domain.dev:8008user/2$". Full response text: Unknown request. Type 'help' for more info.

The http://domain.dev/node/1052/goto_site link and the site created email link, i.e. - the link associated to the !login_url token in the email string still includes the url http://test12.domain.dev:8008 including the apache port rather than the cluster/varnish server port that the platform is associated to. However after the Verify task is run after the Install task the http://domain.dev/node/1052/goto_site link in Aegir then correctly points to the cluster/varnish server and does not display the port (port 80) number as desired.

I would prefer not to patch Aegir as in #1, so I am wondering if there is a hook or recommended method to solve this before hopefully it is included in an upcoming release. i.e. - even a hack to make the site created emails be sent without the port number for now

Thanks,
Paul.

anarcat’s picture

Quite frankly, I think that if you're looking at running varnish on the same server as the webserver, you should consider running nginx, which has caching built in, instead of Apache. It would make everything much simpler.

ergonlogic’s picture

Version: 6.x-2.x-dev » 7.x-3.x-dev
Issue summary: View changes

New features need to be implemented in Aegir 3.x, then we can consider back-porting to Aegir 2.x.

Jon Pugh’s picture

Just making a note of a hosting_varnish tool that's been getting some attention recently:

https://github.com/PraxisLabs/hosting_varnish

leandrops70’s picture

Hello,

I have a Drupal 7 site on a (godaddy) shared server.

It's based on IIS 7 and it means (as far as I know) IIS ignores setting.php and we have to use web.config

My goal: I need to get the (real) IP address of the client machine.

The functions I tested (including ip_address()) are returning me the value of "REMOTE_ADDR" when in my scenario the real IP address of the user is in the the X-Forwarded-For header

The solution in case of using setting.php is this:

// Tell Drupal that we are behind a reverse proxy server $conf['reverse_proxy'] = TRUE; // List of trusted IPs (IP numbers of our reverse proxies) $conf['reverse_proxy_addresses'] = array( '127.0.0.1', );

Then my question is: How can we make these changes/settings using IIS/web.config file?

I tried to create a rule (web.config) to rewrite the value of REMOTE_ADDR with the X-Forwarded-Fo but It doesn't work for me. REMOTE_ADDR is not taking the real IP (X-Forwarded-Fo) of the user. Maybe I need to add something to override this value, before "run" this rule?

I really appreciate any help with this.

Regards,

Leo.

Steven Jones’s picture

Full Varnish support aside, if I manually set the port for Apache to be say 8080, then on a new site install, and if I use the 'Goto' link then the port is wrong, because they are searching for a hardcoded default of 80 to strip out.

I suppose either one could change it so that you could change the hardcoded '80' to be some configurable port, so that for the purposes of checking if we should remove the port when setting a link, we check for 8080 instead, or we could add another variable to the server and use that for the port in various places and that could be empty, and thus we don't need to check to see if it's 80, we just won't be adding it.

Steven Jones’s picture

I think that adding a 'URL port' setting (needs a better name) to the service would be the way to go here, then with an update hook, for each server it can set it to nothing if the port is currently 80, and copy over the current value if it's been changed, then for people who want it, they can just go and clear that value out from the server config in hosting.

For people, like me, that want to create configs with port 8080, but URLs with no port in the URL, I'd go and change the setting to reflect that.

We should probably still check and hide port 80 from http and port 443 from https links, because that's baked in to HTTP clients, right!?

SocialNicheGuru’s picture

when using varnish/pound I can no longer rebuild permissions. I get access denied

url:port/admin/reports/status/rebuild works
but

url/admin/reports/status/rebuild does not

the site at url works fine otherwise

colan’s picture

Running behind an HTTPS proxy, sticking this in /var/aegir/config/includes/global.inc worked for me.

<?php # global settings.php

/**
 * Tell all Drupal sites that we're running behind an HTTPS proxy.
 */

// Drupal 7 configuration.
if (explode('.', VERSION)[0] == 7) {
  $conf['reverse_proxy'] = TRUE;

  // Work around for https://www.drupal.org/project/drupal/issues/2466417.
  // Once that gets in, simply follow the Drupal 8 format below.
  $proxies = array();
  for ($count = 0; $count <= 255; $count++) {
    $proxies[$count] = "192.168.177.$count";
  }
  $conf['reverse_proxy_addresses'] = $proxies;

  // Force the protocol provided by the proxy. This isn't always done
  // automatically in Drupal 7. Otherwise, you'll get mixed content warnings
  // and/or some assets will be blocked by the browser.
  $base_url = $_SERVER['HTTP_X_FORWARDED_PROTO'] . '://' . $_SERVER['SERVER_NAME'];
}
// Drupal 8 configuration.
else {
  $settings['reverse_proxy'] = TRUE;
  $settings['reverse_proxy_addresses'] = array('192.168.177.0/24');
}