Support from Acquia helps fund testing for Drupal Acquia logo

Comments

mikeytown2’s picture

Implement this as a hook that can modify the request before a connection is made; and as a sub module

hass’s picture

The local host operating system already caches the DNS lookups!? Sounds useless.

mikeytown2’s picture

Priority: Normal » Minor
mikeytown2’s picture

Found a good use case for this. http://eumc.eu.int/ times out on the DNS lookup. stream_socket_client & gethostbyname both use a lot of seconds to timeout on the DNS lookup. No way around it using native php. You can do this though
http://stackoverflow.com/questions/4305604/get-ip-from-dns-without-using...

// I can use "localhost" as a way to test if httprl_getipbyhost works.

$url = 'http://eumc.eu.int/';
$uri = @parse_url($url);
timer_start('gethostbyname' . $url);
$x = gethostbyname($uri['host']);
$timeout = timer_read('gethostbyname' . $url) / 1000;
echo 'gethostbyname: ' . $x . ' ' . $timeout . ' ' . $uri['host'] . "<br>\n";

$url = 'http://localhost/';
$uri = @parse_url($url);
timer_start('gethostbyname' . $url);
$x = gethostbyname($uri['host']);
$timeout = timer_read('gethostbyname' . $url) / 1000;
echo 'gethostbyname: ' . $x . ' ' . $timeout . ' ' . $uri['host'] . "<br>\n";

$url = 'http://localhost/';
$uri = @parse_url($url);
timer_start('get_addr_by_host' . $url);
$x = httprl_getipbyhost($uri['host']);
$timeout = timer_read('get_addr_by_host' . $url) / 1000;
echo 'httprl_getipbyhost: ' . $x . ' ' . $timeout . ' ' . $uri['host'] . "<br>\n";

$url = 'http://eumc.eu.int/';
$uri = @parse_url($url);
timer_start('get_addr_by_host' . $url);
$x = httprl_getipbyhost($uri['host']);
$timeout = timer_read('get_addr_by_host' . $url) / 1000;
echo 'httprl_getipbyhost: ' . $x . ' ' . $timeout . ' ' . $uri['host'] . "<br>\n";

function httprl_getipbyhost($host, $timeout = 1) {
  $query = `nslookup -timeout=$timeout -retry=1 $host`;
  if (preg_match('/\nAddress: (.*)\n/', $query, $matches)) {
    return trim($matches[1]);
  }
  return $host;
}

Output:

gethostbyname: eumc.eu.int 15.0005 eumc.eu.int
gethostbyname: 127.0.0.1 0.00009 localhost
httprl_getipbyhost: 127.0.0.1 0.01014 localhost
httprl_getipbyhost: eumc.eu.int 2.01326 eumc.eu.int
hass’s picture

This extremly risky and unreliable. If you set a timeout to 1s you will miss very many slow answers. I know that dns requests often timeout for no reason. I see this so often if I'm running manual nslookup tests... One second after the first failed request, I get the correct answer. Caching this answers based on first result may lead to tons of new issues. Strongly against doing this. We have so many layers of possible troublemaker sources... To find out I already need to clear my local dns server (that caches) and my local machine that also caches... All need to be flushed. I believe there are very good reasons why DNS servers are not caching these failed requests! Before commiting any troublemaker like this we should find out.

I don't see a need for caching these edge cases. That's why we are running parallel requests... For not getting blockings like this :-)

mikeytown2’s picture

stream_socket_client will block on the DNS lookup. On my box, its 30 seconds. We may not want to cache it, but using an alt lookup (that takes into account the timeout option) might be a nice option.

mikeytown2’s picture

Title: Usage and caching of gethostbyname » Usage and/or caching of gethostbyname
hass’s picture

One hanging DNS lookup will block all other link fetch operations or only the one that hangs?

mikeytown2’s picture

DNS lookups happen in httprl_request so it will stall the rest. DNS lookups are not parallel and/or async in PHP.

hass’s picture

OK, this is not acceptable and explains me something I've experienced, but not understood.

mikeytown2’s picture

in PHP DNS look ups happen in stream_socket_client(); which is located in httprl_request(). This is a loader function, it generates the file pointers so stream_select() can do it's magic later on in httprl_send_request(). httprl_request() only operates on a single url thus if the DNS lookup stalls here, the whole script is put on pause and nothing else happens. httprl_send_request() operates on multiple urls so if one of them is stalled it will skip it and move on to the next one. In short, DNS lookups stall the script; nothing else happens when this is going on.

Thinking about this, there is a way to make DNS lookups parallel in PHP by using proc_open and steram_select() issuing multiple nslookup commands at once. This idea would take a lot of work to do as it needs similar logic to the event loop I have for urls currently.

First step is to create the option to support timeouts by using nslookup.

mikeytown2’s picture

Status: Active » Needs review
FileSize
3.33 KB
hass’s picture

Sounds a bit like:

11001
WSAHOST_NOT_FOUND
Host not found. No such host is known. The name is not an official host name or alias, or it cannot be found in the database(s) being queried. This error may also be returned for protocol and service queries, and means that the specified name could not be found in the relevant database.

Aside... A few days ago I found a dns/nslookup library for php... Missed to post the link... Maybe much better than running shell commands... I don't expect any win from this except much more troubles. Troubles like this shound stay optional.

mikeytown2’s picture

mikeytown2’s picture

Status: Needs review » Active
FileSize
6.28 KB

This patch has been committed. It allows for hook_httprl_request_alter(). Caching of gethostbyname is still not committed.

mikeytown2’s picture

Prevent the hook from running if not at the correct bootstrap level. Patch has also been committed.

mikeytown2’s picture

Status: Active » Needs work
FileSize
16.63 KB

Pulled code into its own functions so that said functions can be re-ran in hook_httprl_request_alter a lot easier. Still need to document new functions. Patch below for this change.

hass’s picture

Great cleanup. Makes code much more easier to read!

mikeytown2’s picture

Status: Needs work » Active
FileSize
21.01 KB

Patch attached below has been committed.

mikeytown2’s picture

mikeytown2’s picture

Status: Active » Closed (works as designed)

Not going to implement this. Open to patches though.

mikeytown2’s picture

Just a heads up that this is a lot simpler to do thanks to this going in #1874764: Connection timed out message may confuse end users

delacosta456’s picture

Issue summary: View changes

hi brothers.
Please, is it the reason why when the module is enable , accessing config page or admin page in Drupal takes a lot of time and something send you to 500 server error? (When the module is disable everything works perfectly).

thanks