HTTP Requests not getting through because wrong filename [#2049819]

Advagg seems to be saving file names in a different pattern than it is trying to look them up:

Files named on server in format as follows:
/sites/mysite.com/files/advagg_js/js__--89fShq-buDnCJQb1xulJKvP52YtTfvbYCI6RwpEKE__uT-LQV3FKZGzw6NOhz3ROMs_84O3947Ls0Tvy6xVClU__Q2gzmvgVIfV6wxRJ_hluAYyL-XMU48-pmdEODFelYRs.js

Files called by Advagg in following format:
/sites/mysite.com/files/advagg_js/js1374710206.js

Error Message on Status Page

Adv CSS/JS Agg - HTTP Request	HTTP requests to advagg for js files are not getting through.
Raw request info:
stdClass Object
(
    [request] => GET /sites/mysite.com/files/advagg_js/js1374710206.js HTTP/1.0
User-Agent: Drupal (+http://drupal.org/)
Host: www.assetinsightinc.com


    [data] => 
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
<title>404 Not Found</title>
</head>
</script>
<body>
<div id="body-content">
<h1>404 Not Found</h1>
<p>The server can not find the requested page:</p>
<blockquote>
mysite.com/sites/mysite.com/files/advagg_js/js1374710206.js (port 8082)
</blockquote>
</body>
</html>

    [protocol] => HTTP/1.1
    [status_message] => Not Found
    [headers] => Array
        (
            [server] => nginx
            [date] => Wed, 24 Jul 2013 23:56:46 GMT
            [content-type] => text/html
            [connection] => close
            [x-cacheable] => YES
            [x-varnish] => 484645393
            [via] => 1.1 varnish
            [age] => 0
            [x-cache] => MISS
            [cf-cache-status] => MISS
            [set-cookie] => __cfduid=dd73dedd5248617e60e4b3e02806c60ab1374710206; expires=Mon, 23-Dec-2019 23:50:00 GMT; path=/; domain=.mysite.com
            [cf-ray] => 9348c88698104fb
        )

    [code] => 404
    [error] => Not Found

Not sure where to even start with this one?

Comments

Comment #1

mikeytown2 CreditAttribution: mikeytown2 commented 25 July 2013 at 00:39

Category:

bug

» support

AdvAgg has a fast 404 handler that will set a header and leave a html comment. This checks to see if a bad request to the advagg dir (mysite.com/sites/mysite.com/files/advagg_js/) results in a generic 404 or a 404 generated by AdvAgg. In your case the 404 is not being handled by AdvAgg. Something else is intercepting the request before AdvAgg can handle it.

Do you have a fast 404 rule in nginx or varnish?

Comment #2

JMOmandown CreditAttribution: JMOmandown commented 25 July 2013 at 00:51

I do. However in cleaning up the output to display only relevant information I removed what you are looking for, the 404 is handled upstream by the CDN provider. Below is the full output:

Adv CSS/JS Agg - HTTP Request	HTTP requests to advagg for css files are not getting through.
Raw request info:
stdClass Object
(
    [request] => GET /sites/mysite.com/files/advagg_css/css1374713239.css HTTP/1.0
User-Agent: Drupal (+http://drupal.org/)
Host: www.mysite.com


    [data] => 
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
<title>404 Not Found</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
<script type="text/javascript" src="//ajax.cloudflare.com/cdn-cgi/nexp/abv=547027026/appsh.min.js"></script><script type="text/javascript">__CF.AJS.inith();</script><style type="text/css">body{font-family:Verdana,Arial,Helvetica,sans-serif;font-size:12px;background-color:#367E8E;scrollbar-base-color:#005B70;scrollbar-arrow-color:#F3960B;scrollbar-DarkShadow-Color:#000000;color:#FFFFFF;margin:0;}a{color:#021f25;text-decoration:none}h1{font-size:18px;color:#FB9802;padding-bottom:10px;background-image:url(sys_cpanel/images/bottombody.jpg);background-repeat:repeat-x;padding:5px 0 10px 15px;margin:0;}#body-content p{padding-left:25px;padding-right:25px;line-height:18px;padding-top:5px;padding-bottom:5px;}h2{font-size:14px;font-weight:bold;color:#FF9900;padding-left:15px;}</style>
</head>
<body><script type="text/javascript">
//<![CDATA[
new Image().src = "/cdn-cgi/ping?cf%5blocation%5d=404&cf%5bjs%5d=1";
//]]>
</script>
<noscript>
<img src="/cdn-cgi/ping?cf%5blocation%5d=404&amp;cf%5bjs%5d=0" alt=""/>
</noscript>
<div id="body-content">
 
 
<h1>404 Not Found</h1>
<p>The server can not find the requested page:</p>
<blockquote>
mysite.com:8082/sites/mysite.com/files/advagg_css/css1374713239.css (port 8082)
</blockquote>
<p>
Please forward this error screen to mysite.com
<a href="http://www.cloudflare.com/email-protection#81ebeeede8efe6e4f3c1e0f2f2e4f5e8eff2e8e6e9f5e8efe2afe2eeecbef2f4e3ebe4e2f5bcc4f3f3eef3a1ece4f2f2e0e6e4a1dab5b1b5dca1b5b1b5a1cfeef5a1XFound for mysite.com:8082/sites/mysite.com/files/advagg_css/css1374713239.css port 8082 on Wednesday, 24-Jul-2013 19:47:20 CDT">
WebMaster</a>.
</p>
<hr/>
 
</div></body>
</html>

    [protocol] => HTTP/1.1
    [status_message] => Not Found
    [headers] => Array
        (
            [server] => cloudflare-nginx
            [date] => Thu, 25 Jul 2013 00:47:20 GMT
            [content-type] => text/html
            [connection] => close
            [x-cacheable] => YES
            [x-varnish] => 484645432
            [via] => 1.1 varnish
            [age] => 0
            [x-cache] => MISS
            [cf-cache-status] => MISS
            [set-cookie] => __cfduid=d5363dba625b48017213987b58065ba691374713240; expires=Mon, 23-Dec-2019 23:50:00 GMT; path=/; domain=.mysite.com
            [cf-ray] => 934d696d68204fb
        )

    [code] => 404
    [error] => Not Found
)

Comment #3

mikeytown2 CreditAttribution: mikeytown2 commented 25 July 2013 at 01:01

How do you handle image styles (imagecache)?

Comment #4

JMOmandown CreditAttribution: JMOmandown commented 25 July 2013 at 01:07

This is also handled at the CDN level upstream with Mirage 2. As a precaution on the Drupal side we use CSS Embedded Images, pushing images inline that are smaller than 32kb which may by an off chance be missed by the CDN.

The CDN compress, caches, and resizes all images upstream to each request instance.

Comment #5

mikeytown2 CreditAttribution: mikeytown2 commented 25 July 2013 at 02:03

Quick info about Mirage 2: http://blog.cloudflare.com/mirage2-solving-mobile-speed

So you don't use core's image module (config location: admin/config/media/image-styles). I just committed another patch will allow you to skip this check #2046299-3: Add in support to bypass far future mode in the CDN module

Apply the patch in the linked issue and In your settings.php file set $conf['advagg_skip_404_check'] = TRUE;

I'm assuming that cloudflare is still passing the requests back to drupal, just that it creates its own 404 instead of using the one given from AdvAgg.

Comment #6

JMOmandown CreditAttribution: JMOmandown commented 25 July 2013 at 03:20

Got rid of the warning, but can we be sure it is indeed working correctly?

Comment #7

mikeytown2 CreditAttribution: mikeytown2 commented 25 July 2013 at 06:48

Delete a file in the advagg dir and then request it. If the file re-appears then 404 generation is working correctly.

Comment #8

mikeytown2 CreditAttribution: mikeytown2 commented 26 July 2013 at 18:37

If the file still isn't showing up try adding a query string to the url.

Step 1: rename this file and the .gz in the web server; add a .html to the end of it.
.../files/advagg_css/css__UsGeyENJc9v9vLJrLTPt1M0EC3RZjnBRTThaD237zuk__MUDsE72pgevWSD9v3sPXlRUe99Vaw2_6Efh42qar48k__R3tR2fxCoPdJQpExYIIU1HrAHCUzZptW2o946hfwS8w.css

Step 2: Issue a request for this file from a web browser using a query parameter to bypass the CDN cache.
http://.../files/advagg_css/css__UsGeyENJc9v9vLJrLTPt1M0EC3RZjnBRTThaD237zuk__MUDsE72pgevWSD9v3sPXlRUe99Vaw2_6Efh42qar48k__R3tR2fxCoPdJQpExYIIU1HrAHCUzZptW2o946hfwS8w.css?bypass=1

Step 3: See if the file was generated. Does this exist now?
.../files/advagg_css/css__UsGeyENJc9v9vLJrLTPt1M0EC3RZjnBRTThaD237zuk__MUDsE72pgevWSD9v3sPXlRUe99Vaw2_6Efh42qar48k__R3tR2fxCoPdJQpExYIIU1HrAHCUzZptW2o946hfwS8w.css

If the file does exist, great! AdvAgg404 is working; go ahead and delete the files from step 1. If the file doesn't exist then odds are AdvAgg 404 is not working. Rename the files from step 1 back to the original filename (by removing .html from the end of them).

Let me know if this answers your question :)