Add support for proxy servers

Did this ever get taken care of in 4.5.2. I can't seem to get any feed to work from behind a firewall. Getting error....

Failed to parse RSS feed Microsoft Security Info: 10060 A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond..

At home this same feed works fine, so I know it's not the feed.

Thanks,

SlackNet

+1 on having a facility to modify proxy settings...

here's a patch of /includes/common.inc for drupal-4.4.2 version

+1 on having a facility to modify proxy settings...

here's a patch of /modules/system.module for drupal-4.4.2 version

patch for /includes/common.inc 4.5.2 version to facilitate rss feeds behind firewall

patch of /modules/system.module 4.5.2 version to set proxy settings

hi, tso small issues. one is described at: http://drupal.org/node/9706
The other is that the code is not "drupal compliant". Drupal developrs are (luckily) very picky about spaces, indentation, etc.

i forgot to remove my hardcoded proxy name.. sorry

Ber, which is not drupal compliant?

1) New features only get added to CVS/HEAD, not to the 4.5.x branch.

2) 'Blind' variables that cannot be changed through an admin interface are not a good idea.

3) Your proxy codepath ignores the $defaults array completely. At this point in the code, this doesn't just contain default headers but also any extra headers passed to drupal_http_request(). This is not good as they are simply not used anymore.

4) Code style:
//use proxy settings
There should be a space between '//' and text. MInd capitalization and punctuation.

$request = "$method ".$uri['scheme']."://".$uri['host'].$path." HTTP/1.1\r\n";
Rules for string concatenation: never a space between . and a quote, always a space otherwise.

Tab usage: we never use tabs due to editor differences. Your patch contains several.

String quote usage: we prefer single quotes in Drupal to double quotes, except in the case where single quotes would be too hard to read (e.g. backslash escapes should be avoided as well).

In a very tight setup, users must authenticate to access the proxy, to prevent unauthorised usage, or to track authorised usage.

Can the proxy support also be modified to allow username/password to be supplied to the proxy?

Thanks Steven.

I hope this patch works and be accepted... This will allow proxy variables to be modified using the admin interface.

compliant patch for proxy suport, thanks Steven.

-1 on this. Please do not introduce more config options. IT makes no sense, since you only set that option once.

Also: you solve a single case of fopen, please look for an overal solution that will solve all drupa fopens, rather than hack into the feed code to fix that specific issue.

* either make the code smarter so it can detect proxy errors (http://drupal.org/node/9706) OR
* add this to conf.php

I'd also like to avoid YAS (yet another setting). Can't we include proxies settings.php?

I tried to apply the patch and from the fsockopen(): unable to connect to error i get the Warning: Unknown: A session is active. You cannot change the session module's ini settings at this time. in Unknown on line 0 error.
I am on the version v4.6.3 on windows with
php
PHP 5.0.5 (cli) (built: Sep 22 2005 10:49:43)
Copyright (c) 1997-2004 The PHP Group
Zend Engine v2.0.5, Copyright (c) 1998-2004 Zend Technologies
with Zend Extension Manager v1.0.8, Copyright (c) 2003-2005, by Zend Technol
ogies
with Zend Optimizer v2.5.10, Copyright (c) 1998-2005, by Zend Technologies
apache 2.0.55 for windows and mysql as backend.
Any help or pointers in the right direction much appreciated.
my commons.inc file version is 1.443.3.10
Regards,
Rajesh

As dlr noted before (http://drupal.org/node/34340), wouldn't it be cleaner to add the proxy support in the core system? Have I missunderstood, but don't I have to patch (http://drupal.org/node/7881) common.inc every time after an update? If it were possible to define the proxy-configuration as an option, it wouldn't be neccesery to do the patch all over again after each update.

I bet there are lots of other corporate users (in addition to dlr and me) behind firewall that aren't able to get aggregated content without proxy support.

I would also love to see the possibility to configure a proxy server as part of the drupal distribution.

Until that happens I have created a patch with the "won't make it into a proper release" hacks from above that applies cleanly to 4.7.2.

1. Features only go to HEAD
2. We must avoid the cluttering of the "admin" settings page for "system" variables like this one, that are defined only once. (Perhaps we should create a special page with all variables in the table, if someone does not like to play with settings.php)

Proxy server should be a core based setting. For instance, here at Cingular Wireless all the servers are sitting behind a proxy server as well as the users. So anything that is hosted on this network is forced to access external links through a proxy server.

I need this functionality too. I have just finished dealing with support at my hosting company that have implemented a proxy server for security. This is only going to become a bigger issue as more hosting companies implement similar techniques to increase security of thier servers. Please, can we have a generalised solution - and may it not take 2 years to implement? Unfortunately, I'm not a coder so can't do it myself. I agree that it needs to be generalised not just for RSS feeds, but for any external communication by Drupal.

Made the changes described to a fresh install of Drupal 5.1 and it worked like a charm. Thanks ;-)

Would be better if it was just a config settings tho.

@#24:

What did you do to get it working with 5.1? I've tried it with my fresh install here and it doesn't work, it fails with the following error:
The feed from BBC News: Education seems to be broken, due to " ".

I've attached the changes I made to my common.inc file.

To get drupal_http_request() working with a proxy server which requires authentication, you need to add the username and password to the request, like this:

+1 to this feature.

One of the greatest pushes for Drupal that I've seen has been behind proxies, using Drupal as a knowledge base or as an intranet. Some form of proxy awareness would be a godsend. In my instance, I am currently behind a NTLM proxy (which makes life real interesting on Linux) although I've managed to get NTLMaps set up so apt-get (etc, etc) can talk to the outside world. Some method to allow proxy details to be passed to Drupal (whether it's a pointer to the NTLMaps application, or to the main proxy itself) would allow us to restore a serious slab of functionally back to the installation.

I might also point out, that with Drupal6, I now have a permanent alert on my admin page, saying "One or more problems were detected with your Drupal installation. Check the status report for more information." ... why?

"Update notifications are not enabled. It is highly recommended that you enable the update status module from the module administration page in order to stay up-to-date on new releases. For more information please read the Update status handbook page."

So now, it's a security issue.

I feel that this issue is starting to grow in significance.

All issues are now fixed in 7.x, then backported if applicable.

I've attached a patch to common.inc to make drupal_http_request work through a proxy. I've allows the proxy host, port, username and password to be specified.

I don't have a proxy that required authentication so authentication has not been fully tested.

This is against Drupal CVS HEAD.

Perhaps, instead of amending drupal_http_request(), we should make it plugable/replaceable?

We already do that with drupal_mail().

Every once in a while somebody discovers another deficiency in drupal_http_request() and suggests to improve it. I myself had to forgo drupal_http_request in a certain project, in favor of CURL, because I discovered it doesn't behave well in strained environments.

+1 on proxy settings.

I'd really like a place to edit in common.inc or through the Drupal interface.

I've been trying to demonstrate Drupal 6.0 inside the corporate network. We have a proxy firewall, so I'm unable to complete the setup with OpenID, Update Status, and RSS. I need to specify (unauthenticated) proxy access to [ip address]:8080.

Check out the post from agentrickard. He provided a link to an fsockopen example

Since the Update Status module is now in the core, and Drupal nags if it isn't on, I do not consider this a feature request. This is a bug. OpenID is now in the core and cannot work for many Intranet sites without an area to specify a proxy. RSS does work under the same environment as well. These individual module developers should not have to create their own proxy module, either.

All changes get made to 7.x first.

Added configuration options to the UI under Site Information. Probably needs to be tweaked some for wording and such.

Probably be changing this tomorrow so that it adds a new link called Proxy Settings rather than throwing under Site Information, but it's a start.

okay, i know nada about php, but i found this commentary in a discussion of this specific issue from a web host blocking http requests for security reasons and they offer these ideas:

1) Using the PHP environment variable $_SERVER['DOCUMENT_ROOT'], which returns the absolute path to the web root directory.
2) cURL is another method that could be used.

would option 1 be possible in the current common.inc file as a manual edit? and if so, how?

I've added changed the patch in 39 to include:
* Validation of the proxy port to be a number and in the range of valid tcp ports
* Use a valid port by default in the settings page
* Fix a bug in my original patch with the setting of $auth_string.

@zxombie good catch.

Opinions on creating a new menu item under Site Configuration called "Proxy Settings"? Also, I'm thinking there should be an on/off option.

@zilla the fsockopen method used in this patch is much better than either of those two alternatives.

I'm thinking we should also allow the option of a persistent connection (via a checkbox), and use pfsockopen rather than fsockopen. On a dedicated server, it would make sense to use a persistent connection, which is probably the majority of sites that are stuck inside of an intranet.

@boydjd, I've updated the patch to create a "Proxy Server" page that allows setting the 4 proxy settings. I've added to the description a note that when the proxy host name is empty it will use a direct connection to the internet.

I've applied this patch in a 6.1 test site on my school webserver which is behind a firewall requiring proxy authentication. There were no errors reported.

The Proxy Settings form appears in Administer-->Site Configuration-->Proxy Server and accepts input.

But I'm still getting "HTTP request status Fails" from Status Report and the Aggregator module reports "" The server can't issue HTTP requests".

I appreciate the work that has gone into this, but I am also amazed that this is even still an issue. There must be countless educational/corporate intranet/webservers that must authenticate through an upstream proxy service.

As someone observed, this is not a feature request, it is a design flaw.

I hope this can be resolved sometime soon and built into core. 6.2 perhaps?

Cheers

@Slim Pickens, what method of authentication does the proxy server expect? The patch only implements basic authentication.

If you need this environment to support NTLM (which continues to be the bane of many of our existence), you will need to potentially review a solution that uses http://ntlmaps.sourceforge.net/ ... I use this on my Ubuntu machine at work to handle the NTLM Proxy so apt-get (etc) works.

#53 zxombie - the proxy server expects a username and password. I thought the common.inc patch was creating that capability?

Hey guys -- I haven't tested this yet, but it looks reasonable to me. A couple of suggestions:

1. We don't capitalize each word in a sentence or title. Only the first word gets a capital letter. So things like 'Forward Proxy Settings' should become 'Forward proxy settings'.

2. Proxy settings are quite technical. It would be good if there was a little bit of help on the settings page. If my mom reads it, she should probably be able to determine that this is not a page she should make changes to.

3. What is an 'empty Drupal site'?

4. Add an entry to CHANGELOG.txt please.

Given some additional polish, it should be good to go.

I've re-rolled the patch so that it applies cleanly against head. Working on implementing Dries' changes.

Thanks, it works!

'Forward Proxy Settings' should be 'Forward proxy settings'.

'the Internet' should be 'the internet'.

'Proxy Server' should be 'Proxy server'.

Etc.

Actually "Internet" is a proper noun, and should therefore be capitalized.

I'm attaching a tarball of a module + core patch which enables the use of a http proxy in drupal 5.7. It's based almost entirely on proxy_11.patch at #57 above.

One exception though, TCP port 0 is reserved so I don't allow it.

Thank you for the module/core patch.... does this allow for the option to have only secure https requests to go through a proxy server? That feature would be awesome as it would allow Drupal to run on shared hosting plans (such as GoDaddy). GoDaddy requires https calls to run through s proxy server but not http.

Edit: While my initial post referenced the Drupal 5 patch, I think it would be of great value to separate the http and https calls through proxy options in Drupal 7 for the same reason. Some hosts require only secure https calls to be done through a proxy server.

I edited my files with the proxy_11.patch above for version 6.2. It is currently working and solved my problems completely...THANK YOU to everyone who worked on this and provided this solution.

I have attached my common.inc, system.admin.inc, and system.module patched files in a zip for those who are unable to patch their files. Hopefully, this will work for you as well.

Thanks again.

Any system that does not support NTLM/Digest authentication is just not ready for the Enterprise! Proxies have been around for along time. How come Mozilla Firefox gets it right?

Suggestion for Proxies that require Digest/NTLM authentication.
If the proxy you are authenticating against requires than try NTLMAPS from sourceforge.
NTLMAPS works in Linux as well as Windows or any OS for that matter as it require Python.
I have used it in Ubuntu and Windows XP and it works well.

This is not a support forum. This needs to go into D7 first.

Subscribing....

Hi all,

I'm new here so sorry if I tread on toes - just wanted to note that the patch in #57 *almost* works behind our basic-authentication proxy at the University of Newcastle for Drupal 6.2. Common.inc is not quite right though and needs to be modified slightly as given in the attached patch. I now have proper update and RSS functionality through the proxy as far as I can tell, though more testing is obviously required.

I also can't confirm that this works for 7.x-dev as I'm not actively involved with HEAD.

Very cool, I'd love to test it, but I haven't encountered the need of a proxy server. New features have to go into Drupal 7, and then back ported to Drupal 6. It's wonderful to get as many patches and tests out there as possible though, so thanks! If I get the chance, I'll try porting this to HEAD.

@boydjd:

I think I found following problem with the patch from your comment#57: your modification of drupal_http_request would by itself work fine in my site (which is behing an ISA proxy with basic authentification) if there wasn't the call for system_check_http_request at the beginning. Here Drupal checks if it is able to load a local page per HTTP request. The relative url for this check is 'admin/reports/request-test' and gets completed to an absolute url (like 'http://www.example.com/admin/reports/request-test' if 'http://www.example.com' is the url of the Drupal Site). This check is now done over the modified drupal_http_request function and it fails in my site because it tries to make the request over the proxy I configured before in the new proxy settings. And because this check fails, drupal_http_request returns with an error and does not try further requests.

So the self_test (system_check_http_request) has to be a direct request on my site and I think on every site behind a proxy to be successful.

To achieve this behaviour I added the parameter No proxy for to the proxy settings. It works like the same parameter in Firefox. If you define something like 'example.com' as "No proxy for", then every request to a site which has the string 'example.com' in it's url is requestet directly, not over the proxy.

With the additions in the patch (against Drupal 7 head) I provide here I could use update status and the feed aggregator behind the proxy.

Hi asilva,

here is the patch from comment#83 above for D6.3 and D6.4. It works for my site behind an ISA proxy server with authentification. On another drupal site I had still troubles with this patch. There I added some code to switch off Drupal's HTTP self test to make the patch work. Tell me if the patch here works for you. If it doesn't I will provide the other patch with the disabled HTTP self test.

To apply the patch for D6.3 please copy the file drupal-6.3-proxy.patch into your Drupal base directory and apply it with the command: patch -p0 < drupal-6.3-proxy.patch.

The 2nd file drupal-6.4-proxy.patch is the same patch as patch file for D6.4

Four things in response to the patch at #83:

1. I'm not quite sure how I feel about a is_in_no_proxy_list() function as it isn't really that descriptive. drupal_in_proxy_list() maybe?
2. I'm also not sure how I feel about foreaching through the list whenever the drupal_http_request function is called, but can't really think of a cleaner alternative.
3. $output .= '<li>'. t('basic configuration options for .... The space before and after the period is part of Drupal 7 coding standards ;-) .
4. Is there anywhere else that the proxy settings could live? They seem pretty small for just one page.

Hi Rob and thanks for your response.

I'm not quite sure how I feel about a is_in_no_proxy_list() function as it isn't really that descriptive. drupal_in_proxy_list() maybe?

I suggest to rename the function is_in_no_proxy_list() to drupal_is_proxy_exception(). But I'm not sure if it's good to use the suffix drupal_ for this internal helper function. Wouldn't it be better to name it _is_proxy_exception() instead?

2. I'm also not sure how I feel about foreaching through the list whenever the drupal_http_request function is called, but can't really think of a cleaner alternative.

I agree with your concern. Maybe it would be a better alternative to use an additional variable proxy_enabled which would be represented by a new checkbox labeled "Use proxy server" in the proxy settings page? Then the foreaching through the list would only be necessary when "Use proxy server" is checked.

3. $output .= '<li>'. t('basic configuration options for .... The space before and after the period is part of Drupal 7 coding standards ;-) .

Shame on me, I used ancient coding convention for string concatenations (Drupal 4 I think...). I'll adjust the syntax in my next patch file.

4. Is there anywhere else that the proxy settings could live? They seem pretty small for just one page.

You are right, but looking at D7's site configuration menu I find some other small settings pages like "Clean URLs", "File system", "Image toolkit" or "Error reporting". On the other hand I don't feel that the proxy settings would match with any of them from a topical point of view. So I suggest to keep the proxy settings on their own page for now. Maybe in the future someone will collect all those small pages into one or more bigger pages?

Please tell me what you think of my suggestions, so I will be able to modify my patch.

Hi rizamp,

here is a patch for D6.4 with the disabled self test. But in the meantime I found a better replacement for the problem I had on one of my sites behind a proxy server. I had to enable the reverse_proxy setting in settings.php:

With this setting the http self test workes fine for me. please try to enable the reverse_proxy setting, before you apply the new patch and disable the http self test (by checking the checkbox "Skip HTTP self test" at the end of the proxy settings page).

If the proxy patch still does not work on your site and your proxy server regards authentification: be sure that your proxy server accepts "basic authentification method". If he does not allow basic authentification but only allows other methods like Digest, Radius or NTLM the proxy patch won't work. It works only for basic authentification (or if no proxy authentification is required at all).

Another thing we have to think about is testing. Is there anyway to create an effective SimpleTest for this for self-testing?

• Removed the is_in_no_proxy_list entirely for a stripos call
• Fixed some strings that didn't seem that helpful
• Added hook_help text
• Fixed some of the minor logic which looked kind of weird (variable scope issues)
• Replaced != '' with calls to empty()
• Removed the fieldset in the settings, following what they have for Clean URLs, and other settings with just straight textfields
• Logic in the validation wasn't optimal

To test:

1. Apply patch, clear cache
2. Visit admin/settings/proxy and enter your proxy settings
3. Make a PHP call using drupal_http_request

... And I'd consider this a feature.

Hi,

This is interesting feature, however I believe the word "proxy" used here might be confusing with different kinds of proxies. For instance, default/settings.php use "reverse_proxy" variables for something different.

I would suggest to be a bit more explicit and use "forward_proxy" or something similar as prefix for variables, etc.

Hi Rob,

your fixes look good and make the code cleaner. But after applying your patch from comment#91 I get following error when I access my site:

The reason for this is that empty() only checks variables as anything else will result in a parse error. In other words, the following will not work: empty(trim($name)) or - like it is done in common.inc - !empty(variable_get('proxy_server', '')) and !empty(variable_get('proxy_username', '')).

Hi subru77, unfortunately this can't be implemented through a module because it requires modifying the drupal_http_request function. We could implement a hook in drupal_http_request to expose how HTTP requests are made. Then we could create add on module that could make different kinds of requests, like HTTP requests, HTTPS requestions, FTP requests, etc. But that would have to be thought about, and still wouldn't work if we required a proxy in from of the request..... Thoughts?

Thanks for taking a look at it, brahms. Is that the only issue you saw with it?

Hi Rob,

I'm not shure if the two places in code with empty(variable_get()) are the only issues. I just tested the code (with fixes for the empty issue) on my site behind an ISA proxy. First it didn't work because the variable drupal_http_request_fails had been set to 1 before. This caused a call to the self test (function system_check_http_request) and the self test always seems to fail on my site. The reason for this may be the special network configuration I have to use to reach my customer's site. I am connecting to the web server over an SSH tunnel and there is a network address translation configured between the client I use and the web server.

If I set the variable system_check_http_request) to 0 or delete the variable everything works fine. Starting the update status check I am able to connect from my site to the drupal.org update server.

So I think the patch itself works fine. But I am not sure if Drupal's http self test is working from behind a proxy server. (This is why I tried a piece of code which turns of the self test as I provided in my comment#89) In the next days I will visit my customer with the affected Drupal site. There I'll try to check the self test in a network environment without the SSH tunnel and without the NAT.

EDIT:
I just returned from a visit at my customer. I had enough time there to test the code and everything worked fine including the http self test. For me - and much more: for Drupal - it would make sense if your code from comment#91 will make it's way into core after the empty issue has been fixed.

Yep, the !empty() is broken. This patch looks close though. :)

A workaround for those not comfortable patching Drupal is to use cntlm (http://cntlm.sourceforge.net/) and then using apache to rewrite the URL to point to the cntlm proxy. You then need to alter the URL of what you are looking for to point to your local box. I have posted an example of configuring update module to authenticate with an IIS proxy at http://drupal.org/node/172708#comment-1003511.

subscribing

same as #91 backported to Drupal 6.4

• corrects the diff context respect Drupal 6.4
• corrects the empty calls to variables
• adds one line to make a callback be valid

I applied the patch provided in post #102 to a fresh drupal-6.4 installation without issue (thank you for that). After configuring my proxy server I still can't check for updates. I am now getting the HTTP Request Self-Test error. I followed the instructions in post #89 to enable reverse_proxy, that didn't work for me either. I'm going to attempted to patch drupal with the patch from post #89 (drupal-6.4-proxy-skip_selftest.patch) but I have to mess with it first. In the mean time can someone answer these questions:

- Since I'm getting the Self-Test error should my update checks still be failing?

- Does someone have another solution to the Self-Test problem.

- It seems like most people applied the patch (#91 or #102) like I did and were able to get updates. Is this true or did the rest of you have to do something more to get it to work? The reason I ask is I have been hitting my head on this for 2 weeks and can't get it to work.

Thanks

I'm having second thoughts on this. Instead of adding this straight into core, would we be able to create some kind of hook that would manage altering the request so that if we wanted a proxy contrib module to handle this, it could?

Anyway, here's a patch with the !empty() fix along with an added user permission (administer proxy configuration) because you don't want to give your administrator user team permission to see your proxy server password......

is this for D5/6/7?

I'm having second thoughts on this. Instead of adding this straight into core, would we be able to create some kind of hook that would manage altering the request so that if we wanted a proxy contrib module to handle this, it could?

+1 of course!!!

I'm having second thoughts on this. Instead of adding this straight into core, would we be able to create some kind of hook that would manage altering the request so that if we wanted a proxy contrib module to handle this, it could?

+1

I suggested this in comment #35. On the other hand, I see people are feeling very strongly that this proxy thing should be in core.

====

It will be interesting to see how this plug/hook mechanism will be implemented. It will require only few lines of code, but nevertheless we should "standardize" the way we "plug" custom code into Drupal. I'll have to do the same for format_date() (to enhance i18n support). drupal_mail() too has a "plug" mechanism and it could be replaced with the one we finally settle on.

about the mail there is some work already done http://drupal.org/project/smtp, but it doesn't seems to be proxy aware

I'd prefer to have this in core, but I can be convinced (as always).

If someone can confirm that this solution works, I'm ready to commit it to core.

We should probably add a CHANGELOG.txt entry though.

I can confirm that the most recent patch from Rob in comment#104 works in following environments:

• Drupal 6.4 Site behind a proxy server without authentification
• Drupal 6.4 Site behind a proxy server which allows basic authentification

(I tested those Drupal 6.4 Sites with a back port of the patch, because I didn't have the chance to install Drupal 7 in those environments. I include this backported patch here as g-zipped file for Drupal 6.4).

Though there are proxy configurations where the solution will not work (if there is another authentification method than basic authentification required) I think having the given solution here in core is an important step for bringing Drupal to enterprise environments. Nevertheless we should continue to work on solutions for the other authentification methods, but this should go into a new issue.

I tested #91 backported to D6 in #102 on Drupal 6.4 behind a proxy server without authentification

I vote for inclusion in core. I work in public education. Our State department has more than 1600 schools - all of them are obliged to use a proxy server for internet access. The other 5 Australian States education departments would operate similarly.

Surely a proxy server is the norm for corporate environments? Proxy setup should be a part of the site configuration through core. If you don't need it, don't configure it. We're talking a few lines of code here. No biggie.

Cheers

Include in core.

A major wish and focus of mine is to get Drupal better recognised by large corporations and government. This is not going to happen without "thinking corporate" right out of the box.

The patch at #104 still applies.

Did someone test #104?

On a quick code style and string review, there are several concatenation operators that aren't set off with spaces, plus several code comments not ending in a period. Also, one of the things we need to clean up (in another patch) is the variety of ways we show examples. This patch uses "eg.". Elsewhere in code we have "e.g.," and "Example: xxxx" and "Example: xxx". As long as this patch is internally consistent and hits one of the existing styles it should be fine, especially since there are other issues in the queue to conform all these to a single standard.

What about:

The host name of the proxy server. Leave empty if your site has an existing connection to the Internet or does not use a proxy server. Example: localhost.

The password used to connect to the proxy server, if necessary. Warning: This password is stored as plain text.

"comma seperated" should be "comma-separated".

These could just be "Configure proxy server settings."

I can roll some of these changes into a new patch tomorrow if nobody beats me to it.

Patch #104 modified according comment #117.

Doesn't seem to apply anymore for me ...

The attached patch works on my installation of Drupal 6.5. Please review. Looks like 2 hrs, 33 minutes. A new record! Hurraaahhh!

patch #123 patch without errors for my 6.5, but in status report HTTP request status fails and check manually Drupal core update status also fails. Proxy settings are correct without auth.

patch #123 removes slowing down from update-status module behind a proxy.

Has someone other infos?

@Gos77:

Just some things you can try to make the proxy settings work on your site:

reverse proxy settings

I had to enable the reverse_proxy setting in settings.php to make HTTP connections work on my site.

Drupals HTTP Self Test and the variable drupal_http_request_fails

When the HTTP self test in Drupal fails, Drupal sets the variable drupal_http_request_fails to 1. This is the intended Drupal behaviour and it was the case on my site before I applied the proxy patch. Then I applied the patch but though the proxy patch itself would work, update status still failed because the self test did not reset the variable to 0. I had to reset the variable by SQL (setting the column value to b:0; in the variable table) to make the proxy connection work. An alternative way is to delete the variable drupal_http_request_fails from the table. (This is no bad hack because Drupal itself writes this variable into the table once the self test fails. If Drupal's self test on your site never had a problem, there is also no row for the variable drupal_http_request_fails in the table.)

As in #125 me tends to add the remark to think about stuff like this: if the self-test request fails, drupals internal logic will ZAP any further http request. Adding a proxy in this whole drupal-http logic is not only imlementing it the technical way. The whole dataflow needs to be thought of. I would love to see proxies out of drupal therefore, it makes configuration too complex. A reverse proxy transparently configured somewhere else can do the job as well.

Instead of this, an own connection and transport class for php5 based druapl version(s) might lead to a more and propper way to solve such things. in this case, you could switch to a simple http proxy transport class instead of the default http one.

works for my blank 6.5 with patch from #123 and hint #125 .. thx

Just reviewd the patch. Looks OK for me so far (6.5 patch). Right now I still have no Idea to solve the problem with a sleaner integration if the http self test fails. I had self test fails with DNS problems but they recovered automatically. Why doesn't this happen here. Any Idea brahms, Gos77?

subscribe

This is a feature request, and as such, goes to HEAD, not 6.x.

Looks like the patch doesn't apply to HEAD cleanly anymore, I'll re-roll when I find a spare moment. :)

Changing to CNW, since it doesn't apply anymore.

No, new features go only against the dev version, regardless of how long they've been requested. Sometimes people maintain backport patches of them, but they do not go into core. Of course, this has to get committed to Drupal 7 first.

Why not use cURL instead of implementing it ourselves? As a bonus, we'll get its 'timeout' feature (it's good for Aggregator).

For Drupal 7, we could define a Drupal::HTTP interface. The default implementation will be the current one. A 'cURL' module, when enabled, will plug in its own implementation.

without getting into the politics of whether or not to backport here is my patch for 6.6 - it works for me, others may find it useful as i found there patches extremely helpful.

Now I am back at work I have attached a patch file for Drupal 6.8 - I hope this helps anyone who needs it.

Nitin - have you tried anything else that requires proxy such as checking for module updates - does that work?

There have been some changes in 6.9 that affect the patch. Try this version, it works for me.

I can confirm the 6.9 proxy patch to work fine with D6.9. Thank you silid!

If we got this into Drupal 7, a patch for Drupal 6 would be very easy. As a side note, #64866: Pluggable architecture for drupal_http_request() would be helpful to stick proxy support in contrib.

Hi everyone!

The patch outlined in the original post didn't work on our proxy. We needed to leave this bit that the author advised to comment out:
$request .= implode("\r\n", $defaults);
With this line commente dproxy complained about missing content-length header. With the line in it works.
Cheers,
Michal

Subscribing

Here is a patch for 6.10

I haven't tested this so I don't know that it works. Please give feedback

patch from #158 working fine for me for D6.10

only getting (Stripping trailing CRs from patch.) for all three patched files while patching.

Feature additions do not get committed to stable branches, unless it is necessary to close a security hole. Getting this into Drupal 7, however, is something that really does need to happen. Can we focus on that? (Even if it were to go into D6, it won't until it's landed in D7. Focus your efforts there.)

Crell, I know that this is the rule - but there is no rule without exceptions. For example, see http://drupal.org/node/243524 and http://drupal.org/node/276174 that now went into 6.10 without being a security fix.

This issue is in fact a security fix because the lack of proxy support breaks the Update status module on servers behind a proxy, with the consequence that it no longer warns you about essential security updates. It's not an option that some may want to enable because it's a fancy feature - it's essential to work behind a proxy. Without proxy support, all those servers are locked away from using Drupal 6 at all. So I don't see this patch as a new feature, but rather as a critical bug fix for the Update status, Aggregator and other modules to work.

The D6 patch is working excellent - what's so hard in porting it to D7?

This patch has been made

The last submitted patch failed testing.

Missing the is_in_no_proxy_list() function?

Patch for 6.10 works great for me for http requests. But it appears that https requests are not proxified so any module that uses https, such as OpenID, does not work - request times out.

Hmm. That appears to be true. I must say that I have never needed to access https by proxy and so didn't realise, but looking at the patch it doesn't do anything to https requests at all.

So in other words, it isn't broken it just hasn't ever done it. I will have a look at the patch when I get time but feel free to submit your suggestion.

Si

I applied the 6.11 patch in #176 and I have the 'available updates" report working for the very first time on a PC and Drupal servers in our company. Thanks for all the great work.

However our team is debating whether we should start the precedence of patching the core. If we do, when we upgrade to future 6.x releases, we will have to keep patching (either relying on someone releasing a patch or we create the patch ourselves).

Proxy support is essential in a corporate environment. D7 supports proxy but it is still a long way away.

Not being able to check "available updates" is a security risk. Including this patch into the next 6.12 release as a security fix will make the life of many corporate Drupal users easier and is another step towards getting Drupal wider acceptance as an enterprise CMS.

Thanks.

This is 6.11 patch updated for 6.12.

Attached file contains the three files to add proxy support to D 6.12

I am closing this issue and deleting every "I tried to patch" comment from it and then reopen.

Please focus on getting a working patch for HEAD (http://drupal.org/node/320). Also, please refrain from posting questions regarding 'how to apply patches' and 'why this is not in Drupal N". Those who do will be rewarded with a two week ban.

I've done my best to ignore the horrible // PROXY-HACK comments, so here is a review of the patch in #181.

Drupal doesn't use value-as-first-component comparisons. You can simply rewrite that as $proxy_not_required && ($proxy_server = variable_get('proxy_server', ''))

No support for HTTPS (via the CONNECT proxy command)? It seems like this could be useful in several use cases I can think about.

This indentation is wrong, and same remark as above for the condition.

Same remark.

The function is poorly named and it is not documented.

The condition (that uses type-agnostic comparison on the result of strstr()) is severely broken: it would allow only partial matches that do not start at the first characters (ocalhost will match localhost, but localhost will not match localhost).

We probably want this match to be either on the hostname or on the IP address. This is not possible right now.

This form should use the new autodefault feature of system_settings_form() in D7.

$form_state['values']['proxy_server'] != '' should be strlen($form_state['values']['proxy_server']) > 0, or "0" will not match. The two error messages should be collapsed in one.

Could anyone give me a hint where I have to enter the settings of the proxy server (IP, Port, maybe authentication) with proxy6.12.patch? I searched the issue with no idea on that...

Administer -> Site configuration -> Proxy Server (admin/settings/proxy)

Sorry to ask here, but does the 6.x patch also work with Drupal 6.13?

Hi roball .. patch from #181 works fine for my 6.13 installation.

As requested here is a patch for 6.13 - I haven't reviewed the suggestions or made any amendments to the 6.11 patch I last uploaded except to apply it to 6.13.

As mentioned previously I haven't written the code included here but have produced the patch from other code submitted in the thread to help other users.

The 'horrible // PROXY-HACK comments' are to make it easier for me to roll the patch from one version to the next.
It doesn't use any D7 feature because it is a patch for D6.

I haven't yet upgraded my site to 6.13 so this is untested but I see no reason why it shouldn't work. When I get some time I may try to take all feedback into consideration but that won't be soon.

Thanks a lot silid, your patch works fine with 6.13 :-)

As far as I understand, nobody wants this feature to go into D7. Marking as won't fix accordingly.

If you do want this to go into Drupal 7, please reroll a patch taking into account my comments in #187 and previous remarks by others.

It will get the work when there is time. This thread is full of people that want this feature.