suggestions: use an honest user agent, and maybe don't accept gzip [#1363248]

In my stand-alone script that does the same thing I found the need to let the server know not to accept gzipped responses. Although your use of CURL may be taking care of that for you, it may not be a problem.

I would prefer that you use a user-agent string that doesn't pretend to be a browser. If you are going to send any User-Agent at all, you may as well say who you are. I really dislike bots pretending to be browsers, it's just disinformation. Totally tiny issue though.

Comment	File	Size	Author
#2	stage_file_proxy-honest_user_agent-1363248-2.patch	1.4 KB	StephenBrown

Comments

Comment #1

StephenBrown commented 23 April 2012 at 20:03

I second this request, for both 6.x and 7.x: Using a User Agent that does not mimic a browser would definitely help in many things, for example access statistics and reports, as well as troubleshooting.

I just discovered that someone had accidentally misconfigured stage_file_proxy on the production server and pointed it to itself, causing our Apache install to perform infinite loops looking for files it already had. This may be something that can be abated in the stage_file_proxy code itself, but the process took much longer than it could have, had a "Drupal: Stage File Proxy" user agent shown up repeatedly in the logs. As it was, I didn't find out until turning on the RewriteLog that a single file was being requested once (by a search bot), and then repeatedly internally redirecting to itself in an attempt to find the file. I thought it was already the case that if the site already has the file, it will not go looking for it from stage_file_proxy_origin. This is to be examined.

All that to say, please change the User Agent to something more accurate. Pretending to be Firefox doesn't help anything but fooling the browser, when all we need is just the file anyways.

Comment #2

StephenBrown commented 24 April 2012 at 21:18

Status	File	Size
new	stage_file_proxy-honest_user_agent-1363248-2.patch	1.4 KB

Here's a patch to change the user agent string.

I also added a helper function in the style of the 'Helper to retrieve the file directory' to grab the version info for the User Agent string.

Please test and make sure it works for you too, and give comments as well, as this patch is currently untested, though I don't see anything that should break it.

I don't believe cURL uses gzip by default, as that is an option added in cURL 7.10 that needs to be set via CURLOPT_ENCODING:
i.e.: curl_setopt($ch, CURLOPT_ENCODING, "gzip,deflate");
and if the empty string ("") is set, a header containing all supported encoding types is sent.
ref: http://www.php.net/manual/en/function.curl-setopt.php