When using drupal_http_request to fetch the full browscap.ini file from browser.garykeith.com, the request times out on my server. Increasing the chunk size from 1024 to 8192 resolves the problem, at least on my server. Probably for 8.x we ought to designate an option for setting the chunk size, but I'll wait for a working 8.x testbot before submitting that issue.

Comments

pillarsdotnet’s picture

Issue tags: +Performance

tagging.

podarok’s picture

pillarsdotnet’s picture

Category: feature » bug

@podarok -- Dunno how you're gonna test for something like this. It's a judgment call, not a pass/fail. The only testable difference this patch can make is the amount of time it takes to fetch a remote file, and even that will differ depending on the configuration and load of the server.

Removing tag.

pillarsdotnet’s picture

Category: bug » feature
pillarsdotnet’s picture

Title: Configurable chunk size in drupal_http_request() » Configurable chunk size drupal_http_request()
StatusFileSize
new2.02 KB

Changed from hardcoded to configurable chunksize, and added entry to settings.php:

/**
 * HTTP Request Buffer
 *
 * When drupal requests a file from another webserver, it periodically checks
 * for end-of-file and server timeouts.  The default is to read a maximum of
 * 1024 bytes between each check.  Increasing this limit may help performance
 * when reading large files from the network.
 */
# $conf['http_request_buffer'] = 1024;
pillarsdotnet’s picture

Title: Increase chunk size in drupal_http_request() » Configurable chunk size in drupal_http_request()
Category: bug » feature
pillarsdotnet’s picture

Title: Configurable chunk size drupal_http_request() » Configurable chunk size in drupal_http_request()
StatusFileSize
new2.02 KB

Thanks, testbot! Trying again...

pillarsdotnet’s picture

Version: 7.x-dev » 8.x-dev

Bumping version, even though I'd *really* like to get this into 7.x as well.

pillarsdotnet’s picture

Issue tags: -Performance

#7: http_request_buffer.patch queued for re-testing.

(Previous result from 7.x testbot passed. Do we have an 8.x testbot yet?)

Status: Needs review » Needs work
Issue tags: +Performance

The last submitted patch, http_request_buffer.patch, failed testing.

pillarsdotnet’s picture

Status: Needs work » Needs review
Issue tags: +Needs backport to D7
StatusFileSize
new2.58 KB

Trying again...

pillarsdotnet’s picture

pillarsdotnet’s picture

pillarsdotnet’s picture

mikeytown2’s picture

Version: 8.x-dev » 7.x-dev

D8 has guzzle
Moving this issue to D7.
Note that httprl supports different chunk sizes for reading and writing.

mgifford’s picture

Issue summary: View changes
StatusFileSize
new1.96 KB

just a reroll.

podarok’s picture

Status: Needs review » Needs work

'buffer' => variable_get('http_request_buffer',1024),

Please, use whitespace after the comma

mgifford’s picture

Status: Needs work » Needs review
StatusFileSize
new1.96 KB

missed that.

podarok’s picture

Status: Needs review » Reviewed & tested by the community

Looks good for me.
RTBC

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 18: drupal_http_request-chunksize_option-1055276-18.patch, failed testing.

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 18: drupal_http_request-chunksize_option-1055276-18.patch, failed testing.

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 18: drupal_http_request-chunksize_option-1055276-18.patch, failed testing.

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 18: drupal_http_request-chunksize_option-1055276-18.patch, failed testing.

Status: Needs work » Needs review
David_Rothstein’s picture

Status: Needs review » Needs work

Seems reasonable to let this be changed, but does it really need to be both an option and a variable? None of the other options work like that...

If it is a variable, does it really rise to the level that it needs to be documented in default.settings.php? (Also a couple minor typos there, such as "drupal' => "Drupal".)

As an option to drupal_http_request(), it should be documented in the PHPDoc at the top of the function.

David_Rothstein’s picture

As an option to drupal_http_request(), it should be documented in the PHPDoc at the top of the function.

Oops... I missed that it already is :) However, the documentation there seems much less useful than the settings.php documentation (which actually explains why the parameter might be used).

Status: Needs work » Closed (outdated)

Automatically closed because Drupal 7 security and bugfix support has ended as of 5 January 2025. If the issue verifiably applies to later versions, please reopen with details and update the version.