Comments

Implement this as a hook that can modify the request before a connection is made; and as a sub module

The local host operating system already caches the DNS lookups!? Sounds useless.

Priority:Normal» Minor

Found a good use case for this. http://eumc.eu.int/ times out on the DNS lookup. stream_socket_client & gethostbyname both use a lot of seconds to timeout on the DNS lookup. No way around it using native php. You can do this though
http://stackoverflow.com/questions/4305604/get-ip-from-dns-without-using...

<?php
// I can use "localhost" as a way to test if httprl_getipbyhost works.
$url = 'http://eumc.eu.int/';
$uri = @parse_url($url);
timer_start('gethostbyname' . $url);
$x = gethostbyname($uri['host']);
$timeout = timer_read('gethostbyname' . $url) / 1000;
echo
'gethostbyname: ' . $x . ' ' . $timeout . ' ' . $uri['host'] . "<br>\n";
$url = 'http://localhost/';
$uri = @parse_url($url);
timer_start('gethostbyname' . $url);
$x = gethostbyname($uri['host']);
$timeout = timer_read('gethostbyname' . $url) / 1000;
echo
'gethostbyname: ' . $x . ' ' . $timeout . ' ' . $uri['host'] . "<br>\n";
$url = 'http://localhost/';
$uri = @parse_url($url);
timer_start('get_addr_by_host' . $url);
$x = httprl_getipbyhost($uri['host']);
$timeout = timer_read('get_addr_by_host' . $url) / 1000;
echo
'httprl_getipbyhost: ' . $x . ' ' . $timeout . ' ' . $uri['host'] . "<br>\n";
$url = 'http://eumc.eu.int/';
$uri = @parse_url($url);
timer_start('get_addr_by_host' . $url);
$x = httprl_getipbyhost($uri['host']);
$timeout = timer_read('get_addr_by_host' . $url) / 1000;
echo
'httprl_getipbyhost: ' . $x . ' ' . $timeout . ' ' . $uri['host'] . "<br>\n";
function
httprl_getipbyhost($host, $timeout = 1) {
 
$query = `nslookup -timeout=$timeout -retry=1 $host`;
  if (
preg_match('/\nAddress: (.*)\n/', $query, $matches)) {
    return
trim($matches[1]);
  }
  return
$host;
}
?>

Output:
gethostbyname: eumc.eu.int 15.0005 eumc.eu.int
gethostbyname: 127.0.0.1 0.00009 localhost
httprl_getipbyhost: 127.0.0.1 0.01014 localhost
httprl_getipbyhost: eumc.eu.int 2.01326 eumc.eu.int

This extremly risky and unreliable. If you set a timeout to 1s you will miss very many slow answers. I know that dns requests often timeout for no reason. I see this so often if I'm running manual nslookup tests... One second after the first failed request, I get the correct answer. Caching this answers based on first result may lead to tons of new issues. Strongly against doing this. We have so many layers of possible troublemaker sources... To find out I already need to clear my local dns server (that caches) and my local machine that also caches... All need to be flushed. I believe there are very good reasons why DNS servers are not caching these failed requests! Before commiting any troublemaker like this we should find out.

I don't see a need for caching these edge cases. That's why we are running parallel requests... For not getting blockings like this :-)

stream_socket_client will block on the DNS lookup. On my box, its 30 seconds. We may not want to cache it, but using an alt lookup (that takes into account the timeout option) might be a nice option.

Title:Usage and caching of gethostbynameUsage and/or caching of gethostbyname

One hanging DNS lookup will block all other link fetch operations or only the one that hangs?

DNS lookups happen in httprl_request so it will stall the rest. DNS lookups are not parallel and/or async in PHP.

OK, this is not acceptable and explains me something I've experienced, but not understood.

in PHP DNS look ups happen in stream_socket_client(); which is located in httprl_request(). This is a loader function, it generates the file pointers so stream_select() can do it's magic later on in httprl_send_request(). httprl_request() only operates on a single url thus if the DNS lookup stalls here, the whole script is put on pause and nothing else happens. httprl_send_request() operates on multiple urls so if one of them is stalled it will skip it and move on to the next one. In short, DNS lookups stall the script; nothing else happens when this is going on.

Thinking about this, there is a way to make DNS lookups parallel in PHP by using proc_open and steram_select() issuing multiple nslookup commands at once. This idea would take a lot of work to do as it needs similar logic to the event loop I have for urls currently.

First step is to create the option to support timeouts by using nslookup.

Status:Active» Needs review
StatusFileSize
new3.33 KB

Sounds a bit like:

11001
WSAHOST_NOT_FOUND
Host not found. No such host is known. The name is not an official host name or alias, or it cannot be found in the database(s) being queried. This error may also be returned for protocol and service queries, and means that the specified name could not be found in the relevant database.

Aside... A few days ago I found a dns/nslookup library for php... Missed to post the link... Maybe much better than running shell commands... I don't expect any win from this except much more troubles. Troubles like this shound stay optional.

Status:Needs review» Active
StatusFileSize
new6.28 KB

This patch has been committed. It allows for hook_httprl_request_alter(). Caching of gethostbyname is still not committed.

Prevent the hook from running if not at the correct bootstrap level. Patch has also been committed.

Status:Active» Needs work
StatusFileSize
new16.63 KB

Pulled code into its own functions so that said functions can be re-ran in hook_httprl_request_alter a lot easier. Still need to document new functions. Patch below for this change.

Great cleanup. Makes code much more easier to read!

Status:Needs work» Active
StatusFileSize
new21.01 KB

Patch attached below has been committed.

Status:Active» Closed (works as designed)

Not going to implement this. Open to patches though.

Just a heads up that this is a lot simpler to do thanks to this going in #1874764: Connection timed out message may confuse end users