Hi,

We are having to write a module to work around the issue where if a web service endpoint disappears for whatever reason (usually nothing to do with Drupal - could be network latency, a temporary problem with the remote service, etc.) then the drupal_http_request_fails variable is set to TRUE and *all* our web service method calls, even to services that still work, fail.

This needs to be smarter. It needs to note that it could not access a *specific* URL, not blanket block all URLs. Perhaps the variable should store the URLs that failed in an array instead of just TRUE or FALSE - then you could stop HTTP requests only to a resource which was not available, not all resources.

Finally, there should be a watchdog entry warning the URL was not accessible as well.

Comments

greg.harvey’s picture

greg.harvey
Primary language English
CreditAttribution: greg.harvey commented
Title: system_check_http_request() function sets drupal_http_request_fails to TRUE when HTTP access is still available. » system_check_http_request() function sets drupal_http_request_fails is being set to TRUE when HTTP access is still available.

To be 100% clear, the reason this is a "bug report" and not a "feature request" is this:

drupal_http_request_fails is being set to TRUE and blocking all HTTP requests when HTTP access is still available.

Undeniably a bug. And a big problem for anyone using or proposing to use Drupal in an SOA environment.

I have proposed a possible "feature" that would negate the bug above.

See below.

greg.harvey’s picture

greg.harvey
Primary language English
CreditAttribution: greg.harvey commented
Title: system_check_http_request() function is overzealous! » system_check_http_request() function sets drupal_http_request_fails to TRUE when HTTP access is still available.
greg.harvey’s picture

greg.harvey
Primary language English
CreditAttribution: greg.harvey commented
Title: system_check_http_request() function sets drupal_http_request_fails is being set to TRUE when HTTP access is still available. » system_check_http_request() test is unreliable.

Actually, I think my problem is a config issue, but a bug there still remains. It just has a different name:

The HTTP request check does not reliably confirm HTTP access is available.

I looked at the function more closely and realised it is not checking the requested URL, but actually a local URL that ought to work. Drupal can still access external web services without issue, therefore HTTP access is fine, but the local request test fails. I suspect this is down to my computer having a hosts file determining the domain name (in dev) rather than proper DNS, but it is slightly annoying to have to reset this all the time.

I don't know what would constitute a reliable check - perhaps half a dozen popular websites, and make sure you get a 200 response from at least one of them?

Another option would be to provide some way to over-ride this check for development purposes as well as a means of resetting the variable.

greg.harvey’s picture

greg.harvey
Primary language English
CreditAttribution: greg.harvey commented
Title: system_check_http_request() test is unreliable. » system_check_http_request() test is not consistent and I don't know why
Version: 7.x-dev » 6.6
Category: bug » support

Actually, I'm not understanding this at all... =(

Every time I call one of my web services, I invoke the xmlrpc() function in common.inc, which in turn invokes _xmlrpc() in xmlrpc.inc, which in turn calls drupal_http_request(), which does a system check using system_check_http_request(). This all works fine. I know it works fine, because I get results from my web service and the drupal_http_request_fails variable remains unmolested.

But if I call system_check_http_request() somewhere else, say in a nodeapi implementation, or a menu callback function, it fails and sets drupal_http_request_fails to TRUE. Why???

And why does it set drupal_http_request_fails to TRUE when *one* of my web service endpoints fails, even though all the others still work fine and as soon as I manually reset the variable everything is good again?

Please help. I'm totally lost as to whether this is a bug report, a config issue or a problem with code... I cannot work this out, because it makes no sense that system_check_http_request() works sometimes and not others. =(

dx9s’s picture

dx9s CreditAttribution: dx9s commented

system_check_http_request() is broken because of the recursive nature between that function and drupal_http_request(...)

YES there is a "static" $self_test .. but that appears to not be working... disabling the code near the top:

includes/common.inc:

 [...]
  /*
  if (!$self_test && variable_get('drupal_http_request_fails', FALSE)) {
    $self_test = TRUE;
    $works = module_invoke('system', 'check_http_request');
    $self_test = FALSE;
    if (!$works) {
      // Do not bother with further operations if we already know that we
      // have no chance.
      $result->error = t("The server can't issue HTTP requests");
      return $result;
    }
  }
  */
 [...]

or

modules/system/system.module:

function system_check_http_request() {
  // Check whether we can do any request at all. First get the results for
  // a very simple page which has access TRUE set via the menu system. Then,
  // try to drupal_http_request() the same page and compare.
  ob_start();
  $path = 'admin/reports/request-test';
  menu_execute_active_handler($path);
  $nothing = ob_get_contents();
  ob_end_clean();
  $result = drupal_http_request(url($path, array('absolute' => TRUE)));
  $works = isset($result->data) && $result->data == $nothing;
  $works = TRUE; // this is dx9s's a hack as this function is not reliable!
  variable_set('drupal_http_request_fails', !$works);
  return $works;
}

are work arounds...

I detected this problem when I was working with OpenID and discovered that a person can enter in a bogus OpenID URI and cause the boolean variable in the database to get set (name='drupal_http_request_fails' under {variables})...

and the site remains unable to use drupal_http_request for anything (OpenID, RSS feeds, checking for code updates, etc.)

This is not good.. all a user needs to do is put in bad RSS feed or try with bogus OpenID URL (such as "x" which has no http:// in front) and wham the system is locked out of using drupal_http_request(...)

This is not good... and my solution is to do what you see in the system module.

Hope this can be fixed as it is a form of a mild DoS! (aka can take out the ability to log into a system via OpenID -- and it's pretty easy DoS for OpenID enabled sites)

--Doug

Damien Tournoud’s picture

Damien Tournoud CreditAttribution: Damien Tournoud commented

Simply not true. It probably means that the self-test in system_check_http_request() fails for some reason on your system, while actual HTTP connections work. This means that your server can't connect to itself (generally because of configuration issues in the DNS).

greg.harvey’s picture

greg.harvey
Primary language English
CreditAttribution: greg.harvey commented

Hi Damien,

That's what I figured, but I can wget the request test page that system_check_http_request() on that server, using the full URI, so the machine can "see" itself. Which is really weird. Have you any suggestions as to what checks I might do to find out what's going on/where the point of failure is?

dx9s’s picture

dx9s CreditAttribution: dx9s commented

It doesn't make sense...

Why why WHY does the function + test work fine ( both drupal_http_request(...) and system_check_http_request() ) and the moment a bad URL for RSS feed or bad URL for OpenID is entered it never recovers?

The test works fine up until a point. ONCE a bad URL is handed to drupal_http_request(...) and the boolean gets set -- it NEVER recoveres (the system_check_http_request() always fails)... If I clear the boolean and cache and all that, it continues to work.

I really dislike having to reset that testing logic to fix a problem that the test is (in theory) to already test for.

Just doesn't make sense and I've come to the conclusion that system_check_http_request() is bad..

BTW one way to clear is to put " $works = TRUE; // this is dx9s's a hack as this function is not reliable!" in the function and visit reports page and have it test and then remove that line (or uncomment) and the test will continue to work and the website continue to function until another bad URL (via RSS/OpenID) is processed.

How can I duplicate / isolate the code so I can prove myself wrong?

The machine has it's own /etc/hosts file so no DNS is required (are you suggesting that the code doesn't make use of the system /etc/resolv.conf sub-system?) -- however there are DNS entries as well.

I'd perform DNS test to verify all that is working... but there is something screwy someplace and I'd love to be proven wrong and figure out where the problem is.

--Doug

greg.harvey’s picture

greg.harvey
Primary language English
CreditAttribution: greg.harvey commented

Doug, I feel your pain - we are in exactly the same situation. If it is the server set-up, I'll be damned if I know what it is, and we have half a dozen decent system administrators here looking at the problem and they can't work out what it is either. This is becoming a critical issue for us.

I'm still not convinced this isn't a bug, but until we can really work out what the cause is we can't say either way. Very frustrating. I would like to dedicate some time to trying this further, but at the moment that isn't possible.

I created a "Reset HTTP Status" button in the admin panel for my module, but, just like you report, a single bad URL takes all HTTP requests down, and if you don't notice for a few hours then that can be very bad. We may be forced to hardwire the system check like you have done if this is not resolved by our sysadmins and no one can suggest an actual fix/investigate it as a potential bug. Damien seems to have disappeared after his initial input, which is a shame, as we could really do with some pointers here on server config, if he's so sure there's no bug here. =(

(are you suggesting that the code doesn't make use of the system /etc/resolv.conf sub-system?)

I can't think of any other rational explanation, as we are in the same place, but this is weird:

however there are DNS entries as well.

Hmmm, we don't have this. Are the definitely correct and match the hosts entries?

I find it hard to believe the former would be true anyway. That would seem like a crazy thing to do!

Damien Tournoud’s picture

Damien Tournoud CreditAttribution: Damien Tournoud commented

Well, you will have to debug that on your installations. I can't reproduce any of the behavior you are describing.

One way of debugging is to check the access.log for requests to admin/reports/request-test, verify that the server do receive requests to that page, and that the size of the response is 12 and the response code 200.

greg.harvey’s picture

greg.harvey
Primary language English
CreditAttribution: greg.harvey commented

Fair enough. If you can't reproduce it, then you can't help. =(
Will delve further.

Doug, you're not using the Single (Shared) Sign-On module are you? I have a lot of this, as though Drupal is trying to login to hit the test page:

192.168.0.231 - - [16/Nov/2008:04:28:27 +0000] "GET /admin/reports/request-test HTTP/1.0" 302 - "-" "Drupal (+http://drupal.org/)"
greg.harvey’s picture

greg.harvey
Primary language English
CreditAttribution: greg.harvey commented
Status: Postponed (maintainer needs more info) » Active

I will leave this post open for Doug, but in my case this is resolved. Damien, your tip to check the access_log was vital. On my local machine I noticed the line in the previous post appearing a lot. I realised that the Shared Sign-On module was hijacking the HTTP check and trying to bounce it via a login check, which was stopping the system check from working.

Fix was relatively simple. In the shared sign on settings (admin/settings/singlesignon) do this:

Target URL

Add this line to that box, so tell Shared Sign-On to let requests to this page go by:

\/admin\/reports\/request-test$

I will raise the addition of the request-test line to the Shared Sign-On default config as a feature request over there.

Doug, please close this issue if this fixes it for you as well. Thanks!

Edit: updated this to reflect the real fix. Post was out of date.

ainigma32’s picture

ainigma32 CreditAttribution: ainigma32 commented
Status: Active » Postponed (maintainer needs more info)

@dx9s: Did you manage to find time to test greg.harvey's solution?

- Arie

ainigma32’s picture

ainigma32 CreditAttribution: ainigma32 commented
Status: Active » Fixed

Looks like dx9s won't be posting any feedback so I'm setting this to fixed.

Feel free to reopen if you think that is wrong.

- Arie

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.