Got a lot of Connection timed out in my very broken test links. :-) Compared to wget I have not found any inconsistencies yet. So far so good.

If you believe this is a false error, turn off async_connect in the httprl options array and try again. Write.

This sounds like an impossible TODO for HTTPRL admins/users, isn't it? It's not a configuration option (I'm aware of) that users are able to change. Should we make this an optional setting in HTTPRL settings or a hidden settings.php setting or just remove the confusing text? Working around such issues is currently up to all maintainers using HTTPRL and no effected site can change this without patching modules.

I'm just asking for a usability discussion how we should allow users to change this on their own, but I would prefer to have a setting in HTTPRL as this setting sounds like a global setting that should affect all modules integrating with HTTPRL. I would also remove the confusing message users cannot recover from. Not sure why this message is there... maybe it's for the async PHP bug that we identified in 5.2.x ?

Comments

StatusFileSize
new926 bytes

In the options array that is passed into httprl_request() set 'async_connect' => FALSE and try again.

This patch has been committed. It only displays the extra text if the socket name is empty and the async_connect option is set to TRUE. Let me know if you get the extra text now :)

If you do, try increasing the global connections limit as that will reduce the occurrences of this. Explanation of why increasing the global limit will help:

$empty_runs is a counter that gets incremented if stream_select returns empty. On success $empty_runs is reset to zero. stream_select() returns the number of stream resources contained in the modified arrays, which may be zero if the timeout expires before anything interesting happens. If this timeout has been hit after 25ms*32 ~ 800ms + 5ms*32 = 160ms Grand Total of about 1 second (at the very start of the loop the timeout can be 2 seconds); then HTTPRL checks to see if any of the sockets haven't made a connection yet VIA stream_socket_get_name. If this function returns FALSE that means a connection to the endpoint server has not been made yet; it's a way to see when a stream is ready if an asynchronous connection was made.

All of the above only happens if ALL streams return back in a not ready state. Thus the more streams that are in the connection pool, the chance of having all of them take a long time to complete a TCP connection is reduced. If all connections are dead/slow then it would be best to remove the dead/slow ones so other connections can be made and not timeout the function.

<?php
...
 
$stream_select_timeout = 1;
...
  while (!empty(
$responses)) {
...
        if (
$empty_runs > 32) {
         
// If nothing has happened after 32 runs, see if the connection has
          // been made.
         
$socket_name = stream_socket_get_name($result->fp, TRUE);
        }
       
// Connection was dropped or connection timed out.
       
if ($timeout <= 0 || empty($socket_name)) {
         
$result->error = $t('Connection timed out. ');
          if (empty(
$socket_name) && $result->async_connect) {
           
$result->error .= $t('If you believe this is a false error, set async_connect to false in the options array that is passed into httprl_request() and try again.');
          }
...
   
// Set the read and write vars to the streams var.
   
$read = $write = $this_run;
   
$except = array();
   
// Do some voodoo and open all streams at once. Wait 25ms for streams to
    // respond.
   
$n = stream_select($read, $write, $except, $stream_select_timeout, 25000);
   
$stream_select_timeout = 0;
   
// We have some streams to read/write to.
   
$rw_done = FALSE;
    if (!empty(
$n)) {
     
$empty_runs = 0;
...
    }
    else {
     
$empty_runs++;
    }
...
    if (!
$rw_done) {
     
// Wait 5ms for data buffers.
     
usleep(5000);
    }
...
  }
...
?>

StatusFileSize
new1.2 KB

Also committed this

Please remove the trailing space from the translatable strings. Translators may not see them. Just write . ' ' . to glue both strings together, please. I give dev a new try later.

However it was not about me, who is able o set the async option... It's our users who need to patch other maintainers modules just to try out... This sounds tooo difficult to me.

StatusFileSize
new931 bytes

Committed this patch for the white space issue.

StatusFileSize
new672 bytes

Following patch has been committed.

I think I need more than 1 timeout setting. wget has multiple timeouts
http://www.gnu.org/software/wget/manual/html_node/Download-Options.html#...

I've listed where the equivalent is in HTTPRL

When async_connect is TRUE
--dns-timeout is done in stream_socket_client()
--connect-timeout is done in httprl_send_request() via stream_socket_get_name()
--read-timeout is done in httprl_send_request() via microtime

When async_connect is FALSE
--dns-timeout is done in stream_socket_client()
--connect-timeout is done in stream_socket_client()
--read-timeout is done in httprl_send_request() via microtime

Would you be in favor of multiple timeout settings?

Until now I had no need for this yet...

Have you seen this with the 900 seconds read timeout? Sounds a bit high...

‘--timeout=seconds’
Set the network timeout to seconds seconds. This is equivalent to specifying ‘--dns-timeout’, ‘--connect-timeout’, and ‘--read-timeout’, all at the same time.
When interacting with the network, Wget can check for timeout and abort the operation if it takes too long. This prevents anomalies like hanging reads and infinite connects. The only timeout enabled by default is a 900-second read timeout. Setting a timeout to 0 disables it altogether. Unless you know what you are doing, it is best not to change the default timeout settings.

All timeout-related options accept decimal values, as well as subsecond values. For example, ‘0.1’ seconds is a legal (though unwise) choice of timeout. Subsecond timeouts are useful for checking server response times or for testing network latency.

dns-timeout + connect-timeout. PHP default is about 20 seconds.

read-timeout. Drupal uses 30 seconds from the start of the function call which is effectively (dns-timeout + connect-timeout) - 30 = How much time is left for the read-timeout.

If HTTPRL had different timeouts for each of these I would feel confident in removing the async_connect message, as one could adjust the connect-timeout setting. I would make the defaults for these timeouts user configurable then.

Proposed Error codes for said timeouts:
dns-timeout 11001 "Host not found. No such host is known." It's the best match I could find.
connect-timeout 10060 "Connection timed out. A connection attempt failed because the connected party did not properly respond after a period of time, or the established connection failed because the connected host has failed to respond."
read-timeout 10053 "Software caused connection abort. An established connection was aborted by the software in your host machine, possibly due to a data transmission time-out or protocol error. "

Sounds all good to me.

Proposed defaults
5 seconds for DNS
5 seconds for connect
20 seconds for first byte
30 seconds for the whole thing (what the current timeout setting controls)

All 4 timeouts will be configurable from GUI.

Status:Active» Needs review
StatusFileSize
new13.26 KB

Following patch has not been committed. GUI options are not in yet. Minimal testing done.

New thing added is to the object is time_to_first_byte.

Status:Needs review» Fixed
StatusFileSize
new22.35 KB

Attached patch has the displayed default values come from the defined constants. It has been applied to 6.x & 7.x

Status:Fixed» Needs work

Just for the usability I would name the fields like "DNS timeout (dns_timeout)" and move the "Maximum number of seconds a DNS lookup request may take." to the description. Additionally I would end the sentence with a period. I would also use '#suffix' = t('seconds') :-)

How about using http://api.drupal.org/api/drupal/includes%21common.inc/function/format_i... for formatting the seconds in t()?

Had some time to do a test. Looks great here. Found ~40 Connection timed out. TCP Connect Timeout. and one Connection timed out. Time to First Byte Timeout. with http://www.nonprofit.de. http://www.nonprofit.de seems to be an extremely slow site, but gives me a 200 OK with wget.

These will be overridden if a different timeout is set in the options array.

Uhhh... shouldn't the max() value win?

Status:Needs work» Needs review
StatusFileSize
new5.89 KB

#15 is not clear to me. How should a user override a wrong value? I think it should be max() between this setting and the setting a maintainer may have set in options array (or what the defaults are).

Patch attached fixes code style issues and usability issues. I left the interval formatting out.

Missed a closing bracket

Status:Needs review» Fixed

Committed #17. Thanks for the patch :)

#16
The defaults get added into the options array via +. Same thing as overriding GET (the default) with HEAD. In terms of max(), that comes into play with the global timeout and connection (global & per domain) limits. All other timeouts are per connection. So in link checkers case, it can run for at least 180 seconds because the global timeout has been set to 180 and 120 is less than that.

What we could do in link checker is

<?php
if (variable_get('httprl_global_timeout', HTTPRL_GLOBAL_TIMEOUT) == HTTPRL_GLOBAL_TIMEOUT) {
 
// use 180
}
else {
 
// use httprl_global_timeout setting because it is different from the default.
}
?>

Attached is a patch that will adjust the TTFB timeout if timeout is larger than the default and ttfb is still the default. It has been committed to 6.x & 7.x

Status:Fixed» Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

Issue summary:View changes

Updated issue summary.