Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.
DNS lookups are slow; option to do this on cron and cache it might be a good idea?
http://php.net/gethostbynamel
http://php.net/gethostbyname
http://stackoverflow.com/questions/5715424/make-stream-socket-client-fas...
Comments
Comment #1
mikeytown2 CreditAttribution: mikeytown2 commentedImplement this as a hook that can modify the request before a connection is made; and as a sub module
Comment #2
hass CreditAttribution: hass commentedThe local host operating system already caches the DNS lookups!? Sounds useless.
Comment #3
mikeytown2 CreditAttribution: mikeytown2 commentedComment #4
mikeytown2 CreditAttribution: mikeytown2 commentedFound a good use case for this.
http://eumc.eu.int/
times out on the DNS lookup. stream_socket_client & gethostbyname both use a lot of seconds to timeout on the DNS lookup. No way around it using native php. You can do this thoughhttp://stackoverflow.com/questions/4305604/get-ip-from-dns-without-using...
Output:
Comment #5
hass CreditAttribution: hass commentedThis extremly risky and unreliable. If you set a timeout to 1s you will miss very many slow answers. I know that dns requests often timeout for no reason. I see this so often if I'm running manual nslookup tests... One second after the first failed request, I get the correct answer. Caching this answers based on first result may lead to tons of new issues. Strongly against doing this. We have so many layers of possible troublemaker sources... To find out I already need to clear my local dns server (that caches) and my local machine that also caches... All need to be flushed. I believe there are very good reasons why DNS servers are not caching these failed requests! Before commiting any troublemaker like this we should find out.
I don't see a need for caching these edge cases. That's why we are running parallel requests... For not getting blockings like this :-)
Comment #6
mikeytown2 CreditAttribution: mikeytown2 commentedstream_socket_client will block on the DNS lookup. On my box, its 30 seconds. We may not want to cache it, but using an alt lookup (that takes into account the timeout option) might be a nice option.
Comment #7
mikeytown2 CreditAttribution: mikeytown2 commentedComment #8
hass CreditAttribution: hass commentedOne hanging DNS lookup will block all other link fetch operations or only the one that hangs?
Comment #9
mikeytown2 CreditAttribution: mikeytown2 commentedDNS lookups happen in httprl_request so it will stall the rest. DNS lookups are not parallel and/or async in PHP.
Comment #10
hass CreditAttribution: hass commentedOK, this is not acceptable and explains me something I've experienced, but not understood.
Comment #11
mikeytown2 CreditAttribution: mikeytown2 commentedin PHP DNS look ups happen in stream_socket_client(); which is located in httprl_request(). This is a loader function, it generates the file pointers so stream_select() can do it's magic later on in httprl_send_request(). httprl_request() only operates on a single url thus if the DNS lookup stalls here, the whole script is put on pause and nothing else happens. httprl_send_request() operates on multiple urls so if one of them is stalled it will skip it and move on to the next one. In short, DNS lookups stall the script; nothing else happens when this is going on.
Thinking about this, there is a way to make DNS lookups parallel in PHP by using proc_open and steram_select() issuing multiple nslookup commands at once. This idea would take a lot of work to do as it needs similar logic to the event loop I have for urls currently.
First step is to create the option to support timeouts by using nslookup.
Comment #12
mikeytown2 CreditAttribution: mikeytown2 commentedinstall nslookup if missing
http://www.ehowstuff.com/how-to-fix-nslookup-host-dig-bash-command-not-f...
http://www.cyberciti.biz/faq/install-dns-client-tools-on-debian-linux/
Will make this a HTTPRL sub module; inline patch for now.
Comment #13
hass CreditAttribution: hass commentedSounds a bit like:
11001
WSAHOST_NOT_FOUND
Host not found. No such host is known. The name is not an official host name or alias, or it cannot be found in the database(s) being queried. This error may also be returned for protocol and service queries, and means that the specified name could not be found in the relevant database.
Aside... A few days ago I found a dns/nslookup library for php... Missed to post the link... Maybe much better than running shell commands... I don't expect any win from this except much more troubles. Troubles like this shound stay optional.
Comment #14
mikeytown2 CreditAttribution: mikeytown2 commentedThanks for reminding me to check http://msdn.microsoft.com/en-us/library/aa924071.aspx
This it?
http://pear.php.net/package/Net_DNS2
http://code.google.com/p/netdns2/
Comment #15
mikeytown2 CreditAttribution: mikeytown2 commentedThis patch has been committed. It allows for hook_httprl_request_alter(). Caching of gethostbyname is still not committed.
Comment #16
mikeytown2 CreditAttribution: mikeytown2 commentedPrevent the hook from running if not at the correct bootstrap level. Patch has also been committed.
Comment #17
mikeytown2 CreditAttribution: mikeytown2 commentedPulled code into its own functions so that said functions can be re-ran in hook_httprl_request_alter a lot easier. Still need to document new functions. Patch below for this change.
Comment #18
hass CreditAttribution: hass commentedGreat cleanup. Makes code much more easier to read!
Comment #19
mikeytown2 CreditAttribution: mikeytown2 commentedPatch attached below has been committed.
Comment #20
mikeytown2 CreditAttribution: mikeytown2 commentedInteresting reads about DNS lookups: http://www.igvita.com/2012/06/04/chrome-networking-dns-prefetch-and-tcp-... https://plus.google.com/103382935642834907366/posts/FKot8mghkok In short this is a hard problem to solve.
Comment #21
mikeytown2 CreditAttribution: mikeytown2 commentedNot going to implement this. Open to patches though.
Comment #22
mikeytown2 CreditAttribution: mikeytown2 commentedJust a heads up that this is a lot simpler to do thanks to this going in #1874764: Connection timed out message may confuse end users
Comment #23
delacosta456 CreditAttribution: delacosta456 commentedhi brothers.
Please, is it the reason why when the module is enable , accessing config page or admin page in Drupal takes a lot of time and something send you to 500 server error? (When the module is disable everything works perfectly).
thanks