function httprl_admin_settings_form() {

  $headers = array();
  $headers['User-Agent'] = 'User-Agent: Drupal (+http://drupal.org/)';

  // Add in the headers and enable blocking mode.
  $options = array(
    'blocking' => TRUE,
    'headers' => $headers,
    'method' => 'HEAD',
    'max_redirects' => 1,
  );

  $urls = array(
    'http://www4.karlsruhe.de/kultur/ausstellungen',
    'http://www.egyptianmuseum.gov.eg/news.asp',
    'http://www.museum.gov.my/english/home.htm',
  );

  drupal_set_time_limit(240);
  foreach ($urls as $url) {
    // Queue up the requests.
    httprl_request($url, $options);
    krumo(drupal_http_request($url, $options));
  }

  // Execute requests.
  $responses = httprl_send_request();
  foreach ($responses as $response) {
    krumo($response);
  }

Sorry, German error message only...
2012-02-11_143621.png

CommentFileSizeAuthor
2012-02-11_143621.png15.57 KBhass

Comments

mikeytown2’s picture

mikeytown2 commented
Status: Active » Closed (duplicate)

This has been fixed in #1427966: fwrite(): XXX is not a valid stream resource in httprl_send_request()
Your not going to get the same error message because the connection is made async; if your version of php is one where STREAM_CLIENT_ASYNC_CONNECT doesn't work and these are https connections then you would get the -10060 error.

hass’s picture

hass commented

Do you have a fixed PHP version for testing? I have no idea why I should receive other status codes with async? It's a network error that says something like "-10060 Can't connect to remote server".

Currently I get "-1 request timed out" for all above links. Something seems to be wrong as the core error message have shown real PHP network and mostly firewall issues on the host what was *extremely* helpful to solve issues. Do you know why are you not receiving the same error with async mode? Aside you said - with the broken PHP we should still get "-10060" isn't it? This is not the case...

Just as note, WGET also receive a "Connection timed out".

hass’s picture

hass commented

This code seems not logic to me... $flags is set outside the condition, than inside you look for isset() what will be TRUE after httprl_request() called first and than it end with ELSE $https_flags = STREAM_CLIENT_ASYNC_CONNECT|STREAM_CLIENT_CONNECT; for sub-sequenced calls. At least it must fail if httprl_request() is called more than once what can be expected.

I don't get the idea behind this confusing code... I guess it's because you don't like to run version_compare() more than once?

  $flags = STREAM_CLIENT_ASYNC_CONNECT|STREAM_CLIENT_CONNECT;
  // Workaround for PHP bug with STREAM_CLIENT_ASYNC_CONNECT and SSL
  // https://bugs.php.net/bug.php?id=48182 - Fixed in PHP 5.2.11 and 5.3.1
  if ($uri['scheme'] == 'https') {
    if (!isset($https_flags) && (version_compare(PHP_VERSION, '5.2.11', '<') || version_compare(PHP_VERSION, '5.3.0', '='))) {
      $https_flags = STREAM_CLIENT_CONNECT;
    }
    else {
      $https_flags = STREAM_CLIENT_ASYNC_CONNECT|STREAM_CLIENT_CONNECT;
    }
    $flags = $https_flags;
  }

I believe it must be:


  $flags = STREAM_CLIENT_ASYNC_CONNECT|STREAM_CLIENT_CONNECT;
  // Workaround for PHP bug with STREAM_CLIENT_ASYNC_CONNECT and SSL
  // https://bugs.php.net/bug.php?id=48182 - Fixed in PHP 5.2.11 and 5.3.1
  if ($uri['scheme'] == 'https') {
    if (!isset($https_flags) && (version_compare(PHP_VERSION, '5.2.11', '<') || version_compare(PHP_VERSION, '5.3.0', '='))) {
      $https_flags = STREAM_CLIENT_CONNECT;
    }
    elseif (!isset($https_flags)) {
      $https_flags = $flags;
    }
    $flags = $https_flags;
  }

or more readable:


  // Workaround for PHP bug with STREAM_CLIENT_ASYNC_CONNECT and SSL
  // https://bugs.php.net/bug.php?id=48182 - Fixed in PHP 5.2.11 and 5.3.1
  if ($uri['scheme'] == 'https') {
    if (!isset($https_flags) && (version_compare(PHP_VERSION, '5.2.11', '<') || version_compare(PHP_VERSION, '5.3.0', '='))) {
      $https_flags = STREAM_CLIENT_CONNECT;
    }
    elseif (!isset($https_flags)) {
      $https_flags = STREAM_CLIENT_ASYNC_CONNECT|STREAM_CLIENT_CONNECT;
    }
    $flags = $https_flags;
  }
  else {
    $flags = STREAM_CLIENT_ASYNC_CONNECT|STREAM_CLIENT_CONNECT;
  }
  // @todo: add inline documenatation
  if (empty($options['context'])) {
hass’s picture

hass commented
Status: Closed (duplicate) » Needs work

But it still does not return "-10060 Can't connect to remote server" to me...

mikeytown2’s picture

mikeytown2 commented
Status: Needs work » Closed (works as designed)

It doesn't know the state of the stream. In order to get an error like that I need to block and connect synchronously to the server. If you have 10 connections that can't connect; why not open 10 connections at once? That's what the asynchronous flag does; the tradoff is we don't get an exact error message. The recent fix I did in #1446662: stream_select timings mainly the code that uses stream_socket_get_name() will now tell you that the connection was refused or dropped instead of a timeout.

Closing this as works as designed. I'm not going to switch HTTPRL to use synchronous connections; defeats the purpose of this module. Making that an option, I would be open to; but the only advantage I can see at this point is the error returned; they both error out, which is the desired behavior.

As you can see almost no one uses the stream_socket_get_name() function; this is on the bleeding edge of what PHP can do.
stackoverflow.com [php] stream_socket_get_name
stackoverflow.com [php] stream_select

hass’s picture

hass commented

I opened 8 async connections, but why are they not able to provide valid status codes???

mikeytown2’s picture

mikeytown2 commented

Assuming the code pattern from the top, can I get the 8 urls to test against? Let me know the expected outcome. If it's only having the wrong error code close this issue or the other one or start a new issue. Having things spread across multiple issues makes following "the issue" difficult for others to track. Thanks for your interest in this code :)

hass’s picture

hass commented

Sorry, tooo many cases and days I wasted to investigate all the httprl bugs... in this specific test case it was only 3 test urls.

However it does not change the question, why you don't get a status code in async mode like in non-async mode. If we fetch an url it must return a valid code. If the local system cannot connect to remove host it should give 10060 as one example. from my understanding, async is only in parallel and tons of requests fetched in parallel. It is not allowed to change any status codes - it just fetches more links at the same time.

Note non-blocking link checks does not mean your non-blocking mode - where we do not wait for an answer from the server. It means the async without blocking the fetching of more urls where we have a blocking logic with core http_request and we can only fetch one by one link and not 1000 in parallel (in httprl blocking mode)... may be confusing...

mikeytown2’s picture

mikeytown2 commented

http://api.drupal.org/api/drupal/includes!common.inc/function/drupal_htt...

This is where the negative error codes come from in core.

  if (empty($options['context'])) {
    $fp = @stream_socket_client($socket, $errno, $errstr, $options['timeout']);
  }
  else {
    // Create a stream with context. Allows verification of a SSL certificate.
    $fp = @stream_socket_client($socket, $errno, $errstr, $options['timeout'], STREAM_CLIENT_CONNECT, $options['context']);
  }

  // Make sure the socket opened properly.
  if (!$fp) {
    // When a network error occurs, we use a negative number so it does not
    // clash with the HTTP status codes.
    $result->code = -$errno; // ***THIS LINE***
    $result->error = trim($errstr) ? trim($errstr) : t('Error opening socket @socket', array('@socket' => $socket));

    // Mark that this request failed. This will trigger a check of the web
    // server's ability to make outgoing HTTP requests the next time that
    // requirements checking is performed.
    // See system_requirements().
    variable_set('drupal_http_request_fails', TRUE);

    return $result;
  }

It's an error created by stream_socket_client(). Because of the ASYNC flag, this no longer returns an error. I need to detect the error my self. Issue your bringing up has to do with how async vs sync works in the php C library code. async flag makes the connection happen in the background; PHP doesn't block here, thus it doesn't know if the connection failed.

Let me know if I didn't make this clear.

hass’s picture

hass commented

I know here the negative value comes from, but I do not understand why PHP does not get the same info from the operating system. It still issues HTTP requests and must get the same result from winsock. It must not matter if there are 1 or 100000 requests are running at the same time as every connection is using a port and it's own socket and the operating system must return the same. Something must be wrong or are you trying to tell me PHP is no longer using winsock to open it's sockets if async mode has been enabled?

mikeytown2’s picture

mikeytown2 commented

The main difference between my code and whats in core in this section is the async flag. I'm under the impression this is working correctly. When using the async flag the only thing it blocks on is the DNS lookup (from my understanding) (hence this idea #1325662: Usage and/or caching of gethostbyname). If you believe this isn't working correctly then this is a php bug. Modify HTTPRL and remove the async flags; you should get the error from stream_socket_client(). I'm in IRC if you want to talk the finer details of this
http://drupal.org/irc
irc://irc.freenode.net/drupal-contribute

STREAM_CLIENT_ASYNC_CONNECT
http://stackoverflow.com/search?q=STREAM_CLIENT_ASYNC_CONNECT

hass’s picture

hass commented

Yes, it sounds like a PHP bug if you don't get a winsock error or there is any secret how the can be grabbed. I'm not familiar enough to discuss the inner workings of stream_socket_client or stream_socket_client.

mikeytown2’s picture

mikeytown2 commented

It can be detected; which is what I do now. I can't find any documentation on how to grab the winsock error from the stream when ASYNC is used.