I am starting to use Aegir in containers with the NGINX-Proxy plus LetsEncrypt companion container.

This setup is similar to using load balancers with HTTPS termination: The request for HTTPS hits the NGINX proxy, then it's HTTP only requests through to the Drupal site.

The problem is that Drupal detects an HTTP request, and then renders all links to content like CSS and javascript to HTTP, causing problems.

I learned that Drupal looks for $_SERVER['HTTPS'] = 'on' to determine what scheme to use, so all we have to do is detect the HTTP_X_FORWARDED_PROTO and set "HTTPS" = on to automatically let Drupal handle the different scheme.

Let's put this directly in the settings.php templates:


  /**
   * If external request was HTTPS but internal request is HTTP, set $_SERVER['HTTPS'] so Drupal detects the right scheme.
   */
  if (isset($_SERVER['HTTP_X_FORWARDED_PROTO']) && $_SERVER['HTTP_X_FORWARDED_PROTO'] == 'https' && $_SERVER["REQUEST_SCHEME"] == 'http') {
    $_SERVER['HTTPS'] = 'on';
  }
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Jon Pugh created an issue. See original summary.

Jon Pugh’s picture

Jon Pugh’s picture

Title: Detect HTTPS requests and set HTTPS variable. » Detect HTTPS requests and set HTTPS server variable so Drupal respects original request scheme.

  • Jon Pugh committed cd5f690 on 3040646-settings-https
    Issue #3040646: Detect HTTPS requests and set HTTPS variable
    

  • Jon Pugh committed cd5f690 on 3040646-settings-https
    Issue #3040646: Detect HTTPS requests and set HTTPS variable
    
Jon Pugh’s picture

Issue summary: View changes
colan’s picture

I'm not in a good position to test this right now, but I did something similar with an external (to Aegir) HTTPS proxy so that Drupal sites know about it. See the example I added to Using a load balancer or reverse proxy.

Not sure about D6, but for D7 I set the $base_url. In D8, I didn't even have to do that. In both cases, I didn't need to explicitly enable HTTPS so maybe it figured this out from the proxy? So you probably just have to set that.

ergonlogic’s picture

Shouldn't we set this in the vhost templates, rather than in settings.php?

Jon Pugh’s picture

This method seems much simpler, Colan.

Drupal 6 7 and 8 all look at that $_SERVER['HTTPS'] variable.

Since all 3 set the base_url dynamically based on $_SERVER['HTTPS'], we simply change that to "on" and Drupal handles the rest.

ergonlogic: I don't see the benefit. settings.php method will work for NGINX also, and it only kicks in if the original request is in HTTPS. Perhaps it could be done in Apache Vhosts with Regex? but I'm sure it is simpler and less code to maintain with these 4 lines in settings.php

colan’s picture

But Drupal still needs to know the proxy IP addresses. Where are you setting those?

Jon Pugh’s picture

In my testing, it isn't a requirement for it to work, but I suppose if we can, we should.

REMOTE_ADDR should probably be set too.

ergonlogic’s picture

It'd be good to remain consistent. We already set that param here, along with lots of other headers.

Both Nginx and Apache provide simple conditional syntax:

helmo’s picture

+1 for setting the var

@ergonlogic, adding it in the apache/nginx templates looks more complex .. and note that with the use of an external SSL termination the ssl variant of these templates is often not used, ss it would have to land in both ssl and non-ssl variants.

Jon Pugh’s picture

ergonlogic: The "HTTPS" param is only set where you mention when the server is using hosting_ssl or hosting_https. What I am talking about is when those modules are not be used at all.

We'd have to see a patch to the NGINX and Apache templates to do a direct comparison, but on initial count, 6 files would have to be changed to implement on the vhost config level: (core vhost.tpl.php, vhost_ssl.tpl.php and vhost_https.tpl.php for both nginx and Apache)

Keeping this logic in PHP is less code, less files, less obfuscation (a simple if statement versus regex).

It's also more clear to Aegir users. Settings.php is the first place Aegir users would look, and a local.settings.php file is the easiest way to override this behavior.

Considering no one has marked it "Needs Work", I assume yall don't feel to strongly against this. Let's move forward with this simple change?

memtkmcc’s picture

The $_SERVER['HTTPS'] belongs in settings.php (PHP level) not in the vhosts (web server level) and is pretty harmless addition as it affects only proxied requests with external HTTPS termination.

Both Jon and helmo are correct that it can't even be done on the vhost level, because vhost is not aware of external HTTPS termination, only aware of local request which is plain HTTP.

Also, the ssl on; Nginx directive is not the same as $_SERVER['HTTPS'] and is already deprecated in newer Nginx releases, plus, even if Nginx offers the if() logic (which should be avoided like plague btw), many/most directives can't be put within if() anyway.

memtkmcc’s picture

I should also mention that we are using similar logic in the global.inc in BOA to make HTTPS support more predictable for many years already.

memtkmcc’s picture

Status: Needs review » Reviewed & tested by the community
ergonlogic’s picture

I respectfully disagree. PHP expects $_SERVER to be set by the web server: http://php.net/manual/en/reserved.variables.server.php

No regex or other complex logic should be required in the vhost to accomplish this. Setting a header in response to another header's value is common practice.

While it would have to be set in the non-SSL vhosts selectively, it should be set by default in the SSL ones.

As it stands, we're making changes to how we pass this into each version of Drupal. We'll presumably need to hack this into Drupal 9+ as well.

Since we're doing this in PHP, other apps (such as JS running on hosted sites) won't have access to these modified headers.

Anyway, I'll stand aside. I still disagree with the implementation, but I won't block it.

memtkmcc’s picture

If you can propose a working modification to support sending proper variable to PHP (FPM and Apache) level (Drupal) from the web server level, based on received (not set) HTTP headers, why not?

ergonlogic’s picture

Assigned: Jon Pugh » ergonlogic
Status: Reviewed & tested by the community » Needs work

okay, I'll take a crack at this.

Jon Pugh’s picture

FWIW I do think we do need a much better way to set generic server variables per site. I've gotten feature requests for devshop to do just that. If that existed this feature could leverage that.

Also worth noting that drushrc.php sets $_SERVER right now. Provision should have an easy and generic way to set the environment for both the web server and the CLI on a per site basis.

Provision 4 does this for CLI users with a special Shell command that sets the Drupal environment vars in a hard coded way so that drush will work: https://github.com/provisionCLI/provision/blob/4.x/src/Provision/Command...

It's still very manual.

We should look at something like the load.environment.php in the drupal-composer project, which leverages Dotenv library to automatically load .env files.

https://github.com/drupal-composer/drupal-project/blob/8.x/load.environm...

ergonlogic’s picture

Status: Needs work » Needs review
FileSize
3.66 KB

Here's a patch for the Nginx piece. It's about 4x longer than it should be, IMO, but subdirs.tpl.php reproduces the fastcgi_param blocks multiple times for some reason.

  • llamech committed b41f714 on dev/3040646-https-proxy authored by ergonlogic
    Issue #3040646: Detect proxied HTTPS requests.
    
  • llamech committed ed6ef6a on dev/3040646-https-proxy
    Issue #3040646 by ergonlogic, llamech: Detect proxied HTTPS requests in...
llamech’s picture

Fixes pushed to dev/3040646-https-proxy

Test procedure:

  1. Deploy provision on the 3040646-https-proxy branch.
  2. Verify the web server.
  3. Install a site.
  4. Add this debugging snippet to the site's local.settings.php:
    ksort($_SERVER);
    
    print "<div style=\"background-color: white;\">";
    foreach ($_SERVER as $key => $value) {
      print($key . " => " . $value . "</br>");
    }
    print "</div>";
    
  5. Install ModHeader browser extension.
  6. Visit the site; confirm that HTTPS header is being set to "off".
  7. Set an X-Forwarded-Proto header in ModHeader to "foo" and refresh the site; confirm HTTPS header is still not being set to "on".
  8. Set the header to "https" and refresh the site; confirm that HTTPS header is now being set to "on".
  9. If it's working, css should stop working (because the browser will then try to load css via https which, presumably, isn't set up).
llamech’s picture

For ease of review, here's a patch.

  • llamech committed b41f714 on 3040646-https-proxy authored by ergonlogic
    Issue #3040646: Detect proxied HTTPS requests.
    
  • llamech committed ed6ef6a on 3040646-https-proxy
    Issue #3040646 by ergonlogic, llamech: Detect proxied HTTPS requests in...

  • colan committed ec43f61 on 3040646-https-proxy
    Issue #3040646: Merge branch '7.x-3.x' into 3040646-https-proxy
    
ergonlogic’s picture

FWIW, this is currently only matching "https". If a proxy is adding a X-Forwarded-Proto header with "HTTPS" or "Https" (for example) it won't match. I don't know whether this is likely. We can always match against an array instead.

ergonlogic’s picture

Deploying this requires re-verifying Nginx servers, since it sets a variable in the server vhost that's then used in site vhosts. I suppose a hook_update_N() will be required to trigger this automatically. Should that be in hosting_nginx? I don't think it's required on Apache.

colan’s picture

Status: Needs review » Needs work

Branch works in my tests, but we still need that hook update.

colan’s picture

Project: Provision » Hostmaster (Aegir)

Moving projects as the hook needs to be in Hosting. We can track both from Hostmaster.

ergonlogic’s picture

Status: Needs work » Needs review

I pushed a commit to a new `3040646-https-proxy` branch in Hosting that adds a hook_update_N(), to regenerate the Nginx server-wide vhosts.

colan’s picture

With a clean dev VM, and pulling the branch here, a verify task doesn't get created from the command line with:

drush @hm updb

..but it does work from the front-end by hitting /update.php.

Can someone else confirm?

Jon Pugh’s picture

Status: Needs review » Fixed

So now there are 3 separate branches for this issue...

  • 3040646-https-proxy
  • dev/3040646-https-proxy
  • 3040646-settings-https (My branch, settings.php method.)

Keeping this in settings.php is the best for long term maintenance. The other branches require maintaining logic flow structures in both Apache configs and Nginx configs, while settings.php method is a single if statement regardless of host.

I don't want this issue to be derailed. Comment #17 approved this change, so I am merging.

Jon Pugh’s picture

Project: Hostmaster (Aegir) » Provision

  • Jon Pugh committed cd5f690 on 7.x-3.x
    Issue #3040646: Detect HTTPS requests and set HTTPS variable
    

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

Jon Pugh’s picture

Related followup:

Aegir Ansible module provides Apache servers with HTTPS only. It must work without HTTPS as well.