I make a GET call to a service that responds with a huge data payload. Everything looks correct, except the returned data value is empty. And the script doesn't run for the full time out. It ends in just a few seconds.
I've attempted to set different options than the default for blocking and async_connect but that does not fix the issue.
I know that the resource is working fine because I've called it from another client on a .net machine.
What options do I need to set to return a huge data payload via GET?
Do you need more info?
Thanks!
| Comment | File | Size | Author |
|---|---|---|---|
| #5 | httprl-1983624-5-fix-utf-8.patch | 4.08 KB | mikeytown2 |
Comments
Comment #1
mikeytown2 commentedI would play around with the various timeouts
(default value) - option_key - short description.
(5) - dns_timeout - maximum number of seconds the DNS portion of a request may take.
(5) - connect_timeout - maximum number of seconds establishing the TCP connection of a request may take.
(20) - ttfb_timeout - maximum number of seconds a connection may take to download the first byte.
Also check
$result->error&$result->codefor any indication of what might be the issue.Also noted is the returned data payload will be stored in PHP's memory. How many megabytes of data are we talking about? You might need to adjust the php memory limit
ini_set('memory_limit','1024M');Also noted is chunk_size_read. Default is 32768 (32K). Increasing this *might* speed up the download as this is the maximum number of bytes fread() will get off of the stream buffer while in the stream_select loop.
Comment #2
matthandThanks for the info. I've increased all of the timeouts, the PHP memory limit and chunk_size_read and still do not see any data. And I really jacked up the limits to ridiculous levels that should cover it.
The code returned is 200 for success.
I am seeing this warning after processing:
Warning: strlen() expects parameter 1 to be string, resource given in httprl_print_empty() (line 2442 of /path/to/sites/all/modules/httprl/httprl.module).
Any further ideas? THANKS!!!!
Comment #3
matthandOddly enough, I switched over to Curl HTTP Request (http://drupal.org/project/chr) and it received the data fine, no errors.
The data string had 6081316 characters. Pretty big!
Comment #4
mikeytown2 commentedFunction in question
http://drupalcode.org/project/httprl.git/blob/e71c38b81c3555b1161097332d...
Things I might need to use in order to fix the warning
http://php.net/types.type-juggling (cast to string)
http://php.net/is-resource
http://php.net/is-string
As for the GET request, can you try drupal_http_request() & file_get_contents() and see if either one of these work?
Comment #5
mikeytown2 commentedThe error that was causing the issue is
iconv(): Detected an illegal character in input string. PHP functions I had to use and redo: mb_convert_encoding(), iconv(), & htmlentities() (If the input string contains an invalid code unit sequence within the given encoding an empty string will be returned, unless either the ENT_IGNORE or ENT_SUBSTITUTE flags are set.).The following patch has been committed.