Here's an interesting idea: Prevent drupal from nuking the cache. Lets say you want to do an action that normally nukes the cache. You want to delay the cache being cleared from happening. What's needed is a backup of the cache, css, js dir's. Make a copy of those dir's, do the drupal action that kills the cache, restore those dir's. It would likely be a manual operation from the drupal admin menu. Next cron still kills the cache; its only a delay mechanism. Thoughts?

Comments

xn2001’s picture

@mikeytown2: I don't quite understand why you want to "prevent Drupal from clearing the cache"? I was direct here from my request to regenerate page only if there are changes(http://drupal.org/node/326515).

Do you imply that every time Drupal clears its cache, Boost regenerates all the pages?

mikeytown2’s picture

Whenever cron is run & css/js aggregation is turned on, the static html files reference non existent *.js & *.css files and get a 404. If we prevent the deletion of those js & css files when cron runs, then the html files don't need to be deleted as well. Boost already clears the node's cache if it's changed or if a comment is added to it. So if you have css/js aggregation turned off then boosts cache will not be cleared thanks to #374067: Pages expires long before their lifetime, which is what your requesting, along with this #326683: Increasing cache lifetime beyond presets; which btw doesn't matter a whole lot.

mikeytown2’s picture

Function calls boost needs to intercept
http://api.drupal.org/api/function/_drupal_flush_css_js
http://api.drupal.org/api/function/drupal_clear_css_cache
http://api.drupal.org/api/function/drupal_clear_js_cache

Found a new one...
http://api.drupal.org/api/function/system_themes_form
#276615: If users goes to admin/build/themes and does not submit, anon users see cached pages with references to non-existent CSS
Viewing your theme page clears the css cache... WHY!?

alt to this is to copy the aggregated css/js files, put them somewhere in the cache dir, rewrite the html with new js/css names & use some magic to point these requests to the correct dir. If we go this direction, boost can be used as a ghetto html exporter.

dropchew’s picture

I was wondering why was it necessary to clear/rebuild the aggregated css and js during cron runs? Maybe hacking the core to prevent clearing of the aggregated css/js will be a better idea if it doesn't serve any purpose.. Hope someone can guide me to do the changes in core. Thanks.

EDIT: Just saw that Mikey had provided the api, thanks. Commenting out those lines and running some tests now...

mikeytown2’s picture

mikeytown2’s picture

Currently running a hacked core with these changes in order to detect the calls made to clear js/css

/**
 * Delete all cached JS files.
 */
function drupal_clear_js_cache() {
  watchdog('clear cache', 'clear_js_cache', array(), WATCHDOG_ERROR);
    ob_start();
    debug_print_backtrace();
    $trace = ob_get_contents();
    ob_end_clean(); 
  file_put_contents("debug_js.txt", $trace);
  
  file_scan_directory(file_create_path('js'), '.*', array('.', '..', 'CVS'), 'file_delete', TRUE);
  variable_set('javascript_parsed', array());
}
/**
 * Delete all cached CSS files.
 */
function drupal_clear_css_cache() {
  watchdog('clear cache', 'clear_css_cache', array(), WATCHDOG_ERROR);
    ob_start();
    debug_print_backtrace();
    $trace = ob_get_contents();
    ob_end_clean(); 
  file_put_contents("debug_css.txt", $trace);
  
  file_scan_directory(file_create_path('css'), '.*', array('.', '..', 'CVS'), 'file_delete', TRUE);
}

After running cron, I get Nothing! From what I can tell this issue #355529: boost_cache_clear_all does not delete files was wrong. Cron doesn't clear the js/css cache. Thinking part of the issue was #276615: If users goes to admin/build/themes and does not submit, anon users see cached pages with references to non-existent CSS but because its a bug, most people including me would put it off as cron clearing the cache. hook_flush_caches() needs to be run for update.php; but it also gets run on cron (BAD, not documented). I need to detect a way and check that update is running & not cron when the function gets called.

This is the output I get from my hacked version of boost

/*
 * Implementation of hook_flush_caches(). Deletes all static files.
 */
function boost_flush_caches() {
  watchdog('clear cache', 'clear_boost_cache', array(), WATCHDOG_ERROR);
    ob_start();
    debug_print_backtrace();
    $trace = ob_get_contents();
    ob_end_clean(); 
  file_put_contents("debug_html.txt", $trace);
  
  if (variable_get('preprocess_css', FALSE)==TRUE || variable_get('preprocess_js', FALSE)==TRUE) {
    boost_cache_clear_all();
  }
  return;
}

cron.php - Wrong... static cache still good

#0  boost_flush_caches()
#1  call_user_func_array(boost_flush_caches, Array ()) called at [/home/content/m/c/a/mcarper/html/includes/module.inc:471]
#2  module_invoke_all(flush_caches) called at [/home/content/m/c/a/mcarper/html/modules/system/system.module:1239]
#3  system_cron()
#4  call_user_func_array(system_cron, Array ()) called at [/home/content/m/c/a/mcarper/html/includes/module.inc:471]
#5  module_invoke_all(cron) called at [/home/content/m/c/a/mcarper/html/includes/common.inc:2661]
#6  drupal_cron_run() called at [/home/content/m/c/a/mcarper/html/cron.php:11]

update.php - Correct... static cache needs to be flushed

#0  boost_flush_caches()
#1  call_user_func_array(boost_flush_caches, Array ()) called at [/home/content/m/c/a/mcarper/html/includes/module.inc:471]
#2  module_invoke_all(flush_caches) called at [/home/content/m/c/a/mcarper/html/includes/common.inc:3668]
#3  drupal_flush_all_caches() called at [/home/content/m/c/a/mcarper/html/update.php:288]
#4  update_finished(1, Array (), Array ()) called at [/home/content/m/c/a/mcarper/html/includes/batch.inc:301]
#5  _batch_finished() called at [/home/content/m/c/a/mcarper/html/includes/batch.inc:43]
#6  _batch_page() called at [/home/content/m/c/a/mcarper/html/update.php:654]
mikeytown2’s picture

StatusFileSize
new1.48 KB
mikeytown2’s picture

Status: Active » Needs review

Above patch should allow boost to not flush the cache when cron is run. Flushes cache on performance page submit, since that wasn't happening, & css/js files where getting flushed.

mikeytown2’s picture

committed patch from #7; still looking for creative ways of intercepting calls to drupal_clear_js_cache() & drupal_clear_css_cache().

mikeytown2’s picture

Status: Needs review » Active
mikeytown2’s picture

Title: Prevent Drupal from clearing cache » Prevent Drupal from clearing js/css file cache
mikeytown2’s picture

Once I figure out a sane way to do this, having the option to put boost into a fully static mode would be nice. Make boost ignore all calls to flush it's own cache & make sure it doesn't overwrite a file if it already exists. Site would have to be fully indexed for this to be effective, but it would essentially allow for one to operate in an "off line" manner with the site still being up to the outside world for the most part. POST and non indexed urls (url variables, ect...) would still be sent to Drupal, and I don't see an effective way to prevent it other then the site off line msg or a 307, 403, 404 or 503 msg; or allowing pages not in the boost cache to still hit Drupal. Things like xml sitemap and imagecache would need some sort of special handling since they serve the file once it is requested if sending a status code & preventing requests from hitting Drupal. Login page would be the exception of course.
http://en.wikipedia.org/wiki/List_of_HTTP_status_codes

mikeytown2’s picture

If aggregated css/js is on; save them in cache/example.com/sites/all/files/*; get files via boost_preprocess_page(&$variables) -> $variables['styles'] & $variables['scripts']. Grab all files that have the same name, but different extensions (.gz, .min.js, ect...) in a later stage like boost_exit(). Add in rewrite rules to handle js/css files, normal & gzipped. mystyle.css will be named mystyle.css_.css.gz & mystyle.css_.css

  <FilesMatch "\.css\.gz$">
    ForceType text/css
  </FilesMatch>

...

  RewriteCond %{DOCUMENT_ROOT}/june/cache/gz/%{SERVER_NAME}%{REQUEST_URI}_\.css\.gz -s
  RewriteRule .* cache/gz/%{SERVER_NAME}%{REQUEST_URI}_\.css\.gz [L,QSA,T=text/css]

eventually boost will handle 4 content types
html
xml
css
js

mikeytown2’s picture

Title: Prevent Drupal from clearing js/css file cache » Cache js/css files, allow for a "static" site
Status: Active » Needs review
StatusFileSize
new27.54 KB

This Patch will cache css & js files; if page compression is enabled it will compress them too. Compatible with Javascript Aggregator & CSS Gzip, but they are not needed in order to gzip css/js files.
New option to make boost ignore page flushes. Means you can put the site into an offline mode, run update.php and to the outside world site will appear to still be up! Because of the CSS/JS caching your won't lose much in terms of presentation. Pages that are not in the cache will still get a site offline message though, so this is not a cure all. POST's will not work as well, when this is on and your site is in the off line mode.

Also in here are some bug fixes.

mikeytown2’s picture

StatusFileSize
new27.74 KB

minor fixes from above patch

mikeytown2’s picture

Status: Needs review » Fixed

committed

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.