I have not seen support for advagg module in the code?

EG: 

if (!strpos($_SERVER['QUERY_STRING'], 'imagecache') && !strpos($_SERVER['QUERY_STRING'], '/advagg_')) {
CommentFileSizeAuthor
#13 fast_404-advagg-support-1166622-13-D6.patch4.12 KBmikeytown2
#11 fast_404-advagg-support-1166622-10-D6.patch4.13 KBmikeytown2

Comments

mikeytown2’s picture

mikeytown2 commented

Rather than hard coding for imagecache & other modules (advagg) I query the database. See the files_proxy module's files_proxy_remote() function to see how I do it. I do have special imagecache handling in files_proxy, but that's so files_proxy downloads the source file instead of all the imagecache generated files.

Peter Bowey’s picture

Peter Bowey commented

You would 'recommend' [fast404] module over the method outlined in the advagg 'readme'?
eg:

FAST 404
--------

Assuming that this guide was followed:
http://2bits.com/drupal-planet/reducing-server-resource-utilization-busy...
and your having issues with Advanced CSS/JS Aggregation. Advagg works similar
to imagecache, thus we need to add in an exceptions for the directories that
advagg uses. Replace this if statement

     if (!strpos($_SERVER['QUERY_STRING'], 'imagecache')) {

 with one like this:

     if (!strpos($_SERVER['QUERY_STRING'], 'imagecache') && !strpos($_SERVER['QUERY_STRING'], '/advagg_')) {

This will most likely be in your settings.php file.

I am still 'in-study' with the various methods - particularly in reference to Nginx!
Current nginx method handling (sample):

        location ~* /(?:external|system|files/imagecache|files/styles)/ {
            access_log         off;
            expires            30d;
            # fix common problems with old paths after import from standalone to Aegir multisite
            rewrite  ^/sites/(.*)/files/imagecache/(.*)/sites/default/files/(.*)$  /sites/$host/files/imagecache/$2/$3 last;
            rewrite  ^/files/imagecache/(.*)$                                      /sites/$host/files/imagecache/$1 last;
            rewrite  ^/files/styles/(.*)$                                          /sites/$host/files/styles/$1 last;
            add_header X-Header "IC Generator 1.0";
            try_files $uri @drupal;
        }

similar logic applies to nginx /advagg handling...

mikeytown2’s picture

mikeytown2 commented

The settings.php method (2bits article) is about as fast as one can get (at the php level) without hacking core. A module is obviously more portable, and it could be more robust. The algorithm I have in the files proxy module is generic; no need to hardcode for imagecache or advagg or the next module that plays with the files directory. I'm here to point this out to soyarma when ever he sees this issue; there is a better way then having to always patch this module to take into account other modules.

I haven't tried out this module so I can't officially comment on it.

Peter Bowey’s picture

Peter Bowey commented

Thanks Mike,

Having studied http://drupal.org/files/issues/404.diff (patch that I don't think made it to D7), and looking at how nginx (in place of using apache) can handle the need for a selective [fast 404], I submit that PHP methods are not necessary for handling static assets with drupal 'handled' 404's - be it imagecache or advagg.

I am sure you accept nginx is not a Drupal module (I refer to #3)?

Testing Mode: Seems to me that advagg refuses to validate fast404's for anything but Drupal methods.
In testing this advagg method, I find if I send 404's to Drupal that advagg expects the issue - but if I use nginx [front-end webserver] to selectively fast 404 - as per the code logic in

The settings.php method (2bits article)

that advagg reports

Check to see if you have fast 404s, if so create an exception for this module. The readme file explains what needs to be changed. You can try flushing the menu cache as well.

Hence, I could be forced to hack my D6 core with http://drupal.org/files/issues/404.diff.

See http://drupal.org/node/76824

and

http://drupal.org/node/76824#comment-4050706

mikeytown2’s picture

mikeytown2 commented

advagg tests to see if a call to the sites/default/files/advagg_css/ dir will get through to the page callback for that path (see advagg_menu()). If it doesn't then there is something up-stream preventing it from happening and you won't [mafia voice]get the 404 protection for CSS/JS aggregated files that advagg offers[/mafia voice]. What prevents it from hitting the page callback can be nearly anything; server configuration, settings.php tricks, Drupal issue patches, modules, or something else. Point is, just about every configuration has imagecache white listed, advagg showed up late into the game so it has not been taken into account. In order to prevent the next module that uses the files directory as hook_menu item from having issues, I've come up with a generic way to handle this; code is in the files_proxy module. This implies a Drupal solution for this issue. A server configuration (apache/nginx) solution will always need to chase any new hook_menu items that work in the files directory; but it will be a lot faster because it doesn't involve php.

The modules I'm talking about in #3 referrer to Drupal modules :)

Peter Bowey’s picture

Peter Bowey commented

Thanks Mike,

OK, so I need to look / study at the suggested http://drupal.org/project/files_proxy module before I commit 'code'. 'Secrets ' within the D6 'kitten'.....modules...(files_proxy).

Related part of the 'files_proxy' appears to be this:

/**
 * Send out a fast 404 and exit.
 *
 * @param $msg
 *   send this message in the header.
 */
function files_proxy_missing_fast404($msg = '') {
  global $base_path;
  if (!headers_sent()) {
    header($_SERVER['SERVER_PROTOCOL'] . ' 404 Not Found');
    header('X-FilesProxy: ' . $msg);
  }
  print '<html>';
  print '<head><title>404 Not Found</title></head>';
  print '<body><h1>Not Found</h1>';
  print '<p>The requested URL was not found on this server.</p>';
  print '<p><a href="' . $base_path . '">Home</a></p>';
  print '</body></html>';
  exit();
}

Seems to me, at a glance, that we have a 'mixed collection' of 'proposed' D6 'fast404' methods.
Need to think and compare....

Peter Bowey’s picture

Peter Bowey commented

Ref #5

Thanks Mike, when I did apply the full Drupal method you outlined for 404's. I got slow and skewered response times for most static assets pulled via advagg's aggregation through a CDN. The given Drupal 404 method certainly made the advagg 'report' status happy - but real life benchmarks indicated I had 40% slower response to static assets.

I went back to do handling this the previous Nginx way (fast selected 404's that Drupal only will only ever see for for 2 chosen locations -> imagecache + advagg + more as required). Benchmarking testing indicates that the 'lost' 40% performance is restored. Yet, 'poor' advagg complains like a Apache piglet - about my setup not being so 'cool'.

So after days spent on code + code for this aggregation testing, I am about to hack out (grin) all advagg's 'report' reasoning on 404 drupal handling. I think it was meant for Apache logic. For despite all the D6 report 'complaints' issued by advagg reasoning - it darn works fast.

Have you thought about some serious Nginx compat?

soyarma’s picture

soyarma commented

I'll add in support for advagg and perhaps add a variable to set allowed paths more dynamically.

Fast_404 can't check the db, because in some configs it fires before the db connection is made.

alladdin’s picture

alladdin commented

How about support of advagg by fast 404 module? Still not implemented?
These 2 modules are great for high load sites.

mikeytown2’s picture

mikeytown2 commented
mikeytown2’s picture

mikeytown2 commented
Category: support » feature
Status: Active » Needs review
StatusFileSize
new fast_404-advagg-support-1166622-10-D6.patch4.13 KB

Taking the same code used for imagecache, here is a patch against the master branch for advagg; patch also takes care of some white space issues. Code that's been added in fast_404_ext_check() in fast_404.inc.

    // Check to see if the URL is an advagg URL. Those are handled via Drupal
    if (strpos($_SERVER['QUERY_STRING'], '/advagg_')) {
        // We're allowing anyone to hit non-existing advagg URLs (default behavior)
        return TRUE;
      }
    }
Renee S’s picture

Renee S commented

Is that last bracket supposed to be there? :)

mikeytown2’s picture

mikeytown2 commented
StatusFileSize
new fast_404-advagg-support-1166622-13-D6.patch4.12 KB

@Reinette
Thanks for the heads up!

soyarma’s picture

soyarma commented

Will the same fix work against advagg in D7?

mikeytown2’s picture

mikeytown2 commented

Yes it will, but there is no D7 version of AdvAgg currently #1171546: AdvAgg - D7 Port/Re-write

mikeytown2’s picture

mikeytown2 commented
Title: What about support for the advagg aggregator? » AdvAgg, Fast 404: Support for AdvAgg
Renee S’s picture

Renee S commented

The patch in #13 worked for me. Thanks :)

mikeytown2’s picture

mikeytown2 commented
Status: Needs review » Reviewed & tested by the community
soyarma’s picture

soyarma commented

I've committed this to the dev branch, going to get a few other things in there as well and cut 1.4

soyarma’s picture

soyarma commented
Status: Reviewed & tested by the community » Fixed

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.