As reported at http://drupal.org/node/168828#comment-643863 it seems not unheard of that people are running Drupal on sites where httpd is configured to not allow outbound network connections. :( This obviously renders update.module useless, since it can't "phone home". Seems like we need to:

A) Document this in the "OPTIONAL REQUIREMENTS" section of INSTALL.txt.

B) If drupal_http_request() fails, notice it, log an error to watchdog(), and if we're doing a manual check, drupal_set_message() about it, too.

C) Figure out if it's possible to test for this problem (without WSOD or triggering a hang/timeout/etc) on the status report and inform the site admin that their httpd configuration prevents update.module from working, then disable attempts to check for updates.

Given that this has string-freeze implications, we should try to clean this up before the RC1 release, so I'm marking this critical. However, I'm not able to work on this immediately, so if anyone else can get this started, please assign it to yourself.

Thanks,
-Derek

Comments

sun’s picture

Status: Active » Needs work
StatusFileSize
new801 bytes

A completed, B + C missing.

gábor hojtsy’s picture

How can you distinguish in C between a regular timeout and blocked requests? It this is not possible, I'd not suggest blocking further tries on the updates if we cannot distinguish. Otherwise all sounds reasonable.

dww’s picture

I don't know how to distinguish yet, that's part of why I posted this issue, in case others know. It really depends on how this apache config knob is implemented. I'm guessing there's no way to tell just from the errno of the connect() failing, in which case we're kinda screwed. I'm hoping there's some way to directly ask httpd about this configuration knob, so we can test that, instead of just trying the connection and failing.

However, either way, if the http request fails, we should log it, and ideally print a nice message on the status report and update status pages about it, instead of a blank screen...

ChrisKennedy’s picture

Should we log all failed drupal_http_requests() or just those related to update.module?

chx’s picture

You can request the clean URL page -- if that fails, that's surely not a timeout.

chx’s picture

Status: Needs work » Needs review
StatusFileSize
new2.66 KB

B is impossible. Failure can have so many faces... you might even get a webpage telling you that stuff is denied. There are so many broken configs I would not try to figure out in drupal_get_request whether a request has failed. I will let every module decide for itself whether the result it got resembles what it expects.

chx’s picture

StatusFileSize
new2.82 KB

Much better comments. (I was kind of satisfied that menu_execute_active_handler can run with any path :) otherwise I'd needed to patch that.) So we first get the JSON for the clean URL check -- why that? Because it's the simplest page I know. And then drupal_http_request it. The two should match.

dww’s picture

Status: Needs review » Needs work

Applied the patch to a clean HEAD install, and added a "return NULL" at the top of function drupal_http_request() to simulate error. When I try to check for updates from the status report page, I see:
{ "status": true }
printed directly to the screen. When I view the available updates page, I see that before the page headers, so I see a screen full of HTML, not the rendered page. :(

dww’s picture

Oh, and after reloading the status report once the error's been hit, I get a bunch of E_ALL warnings:

notice: Undefined index: title in /Users/wright/drupal/cvs/6/core/modules/system/system.module on line 974.
notice: Undefined index: title in /Users/wright/drupal/cvs/6/core/modules/system/system.module on line 974.
notice: Undefined index: title in /Users/wright/drupal/cvs/6/core/modules/system/system.module on line 974.
notice: Undefined index: title in /Users/wright/drupal/cvs/6/core/modules/system/system.module on line 974.
notice: Undefined index: title in /Users/wright/drupal/cvs/6/core/modules/system/system.admin.inc on line 1895.
chx’s picture

Status: Needs work » Needs review
StatusFileSize
new2.88 KB
chx’s picture

StatusFileSize
new2.93 KB

Added title, too.

chx’s picture

StatusFileSize
new3.45 KB

Thou shall not hack.

dww’s picture

StatusFileSize
new609 bytes

After the reviews and testing with chx that led to the above patches, the functionality is now working fine. Thanks, chx! Here's a re-roll that attempts to clean up some of the user-facing text.

I'm still not sure if this menu handler and requirement check belong in system.module (under the premise that they might be shared by other modules), or if they should just go in update.module (the only part of core that's using them). Thoughts on this welcome. For now, I left them in system, as with chx's versions of the patch, though I think I lean towards making this stuff internal to update.module.

dww’s picture

StatusFileSize
new4.34 KB

Doh, wrong patch. ;) Try this one. However, the previous patch is useful if you want to test this one. ;)

chx’s picture

Keep it in system. I had good reasons to put it there -- others might want to use it and it's not like it's a big bloat or somethin'

chx’s picture

Component: update.module » base system
StatusFileSize
new8.18 KB

There are many places that issues these requests. And yet we can't add simply to drupal_http_request because as I said above, failure can have many faces. We are adding this check after the reply does not parse.

chx’s picture

StatusFileSize
new9.38 KB

Added a way to get the variable cleared.

gábor hojtsy’s picture

- So why was calling of the clean URL check page considered hackish? This 'Nothing' menu items looks much more hackish to me. We can rename the clean URL check menu item to something more general to apply to both cases, if this is the only problem.

- It looks like that some code populates $result->error with an English message, while another part adds a t()-ed string (with a strange space at the start of the string). How is this supposed to work?

chx’s picture

The clean menu URL issues a JS content type header which neatly fscks up the forthcoming plain HTML page. I am not aware of any way to remove an issued header and another content type header to mask the JS one, now that's a hack if I have ever seen one. The nothing entry is nice and compact :) The result error space is not significant now, I'll remove it later, the $result->error usually comes from fsockopen so it's usually not translated, so this special case is actually a bit better than other error strings.

gábor hojtsy’s picture

Still you can name it "request-test" or whatever, not "nothing"...

keith.smith’s picture

As a suggestion, how about this:

+          'description' => $t('Your system or network configuration does not allow Drupal to access web pages, resulting in reduced functionality. This could be due to your webserver configuration or PHP settings, and should be resolved in order to download information about available updates, fetch aggregator feeds, sign in via OpenID, or use other network-dependent services.'),

in the place of this:

+          'description' => $t('For some reason (webserver configuration or PHP setting) Drupal can not fetch web pages. Because of this, it is impossible to download information about available updates, fetch aggregator feeds, ping other sites, OpenID does not work etc.'),

Not worth holding up the patch, but just thought I'd suggest in in case of a reroll for some other reason.

chx’s picture

StatusFileSize
new9.69 KB

Renamed to request-test, added keith.smith 's text (thanks), changed deliberate no break to Deliberate because it's a sentence, removed the starting white space in aggregator.

dww’s picture

Assigned: Unassigned » dww
Status: Needs review » Needs work

This is looking great, chx, you rock! There are a few typos and formatting problems in various comments in the code, which I'm re-rolling now. Also, I'm fixing the INSTALL.txt text to be more general (it's not just about update.module anymore), and wrapping the hard-coded error message in t() -- granted, most of the errors aren't translatable, but this one is, so I don't see the harm... New patch coming soon, stay tuned.

dww’s picture

Assigned: dww » Unassigned
Status: Needs work » Needs review
StatusFileSize
new11.45 KB

Re-rolled for the following:

  1. Fixed INSTALL.txt.
  2. Cleaned up a number of code comments.
  3. Fixed two bugs in system_check_http_request():
    • misplaced parens resulted in drupal_http_request() being called with a relative url(), causing it to fail and throw PHP errors. I'm not sure if we should fix drupal_http_request() to more gracefully recover from errors like this, but that should be a separate issue I think. For now, I just fixed the parens around url() and friends, so that we send an absolute URL and things actually work.
    • The $works variable was initialized via !isset(), but that's wrong. If we have data and the data is what we expect, it works.
dww’s picture

Assigned: Unassigned » dww
Status: Needs review » Needs work

No longer applies cleanly. :( Guess, I'll re-roll it.

dww’s picture

Assigned: dww » Unassigned
Status: Needs work » Needs review
StatusFileSize
new10.6 KB
snufkin’s picture

I tested by adding a return; in the first line of function drupal_http_request to simulate the block. When trying to run the status update it returns the blocked report, and it is also indicated correctly on the status report page. After removing the return and running the update the warning disappears, also from the status report. I read the INSTALL.txt modification, it looks good to me.

snufkin’s picture

Status: Needs review » Reviewed & tested by the community

hence i mark it RTBC

gábor hojtsy’s picture

Status: Reviewed & tested by the community » Fixed

Hm, committed this one, thanks.

Anonymous’s picture

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.

Battlehard’s picture

sorry this might be a really stupid question But how do i us this patch ?? sorry total newbie.. but I'm suffering the same problem.