I'm working on a site with thousands of pages that takes several minutes to do a full cache clear with boost_cache_clear_all. So let's say I update some global CSS/JS files, which Drupal repackages into a new aggregated CSS/JS file, which then boost much clear the entire cache so that pages are pointing to the new aggregated files. What's happening is that Drupal first deletes the old aggregated CSS/JS files (when doing a cache clear on admin/settings/performance), so that while boost is clearing the cache, pages that haven't been cleared yet are pointing to a dead CSS/JS file.

I mitigated this problem somewhat by enabling the boost cache for CSS and JS files. With that in place, the boost cache clear doesn't cause any problems until it clears those cached CSS/JS files in the /files directory, after which pages that haven't been cleared yet are again pointing to dead files that are no longer cached by boost. This is a high traffic site, so this small period of broken pages is not acceptable, and I'd rather not take down the site for maintenance for every minor CSS/JS tweak.

It seems to me that the best solution for this would be to clear out the cached CSS/JS files at the very end of the boost_cache_clear_all -- that way, any pages that haven't been cleared yet will still be pointing to valid cached files. And by the time it finishes and the old CSS/JS cached files are cleared, there are no more cached pages pointing to the old aggregated files. Alternatively, Drupal core could be tweaked to clear out the old aggregated files at the very end of the full cache clear, although that involves knowing which to keep and which to delete.

Does that make sense? Thoughts?

Comments

mikeytown2’s picture

mikeytown2 commented
Category: bug » support

In short when nuking the boost cache, kill whats in the perm dir last, correct? What button are you pressing when doing this operation; 0 or 4? Under "Boost directories and file extensions" -> "Static storage (CSS, JS)" what are the Directory entries? Looking at the code; it appears to do what your requesting already.

Button 0 Code

/**
 * Flushes boost page cache
 */
function boost_clear_cache_submit($form, &$form_state, $first = TRUE) {
  // Temp fix for bug in form api & buttons
  // Check to make sure this button was actually called
  if (!boost_admin_button_router(0, $first, $form, $form_state)) {
    return FALSE;
  }

  $ignore = variable_get('boost_ignore_flush', 0);
  $GLOBALS['conf']['boost_ignore_flush'] = 0;
  if (boost_cache_clear_all()) {
    boost_clear_cache_parallel(BOOST_PERM_FILE_PATH);
    boost_clear_cache_parallel(BOOST_PERM_GZIP_FILE_PATH);
    drupal_set_message(t('Boost: Static page cache cleared.'));
  }
  else {
    drupal_set_message(t('Boost: Set "Ignore cache flushing:" to \'Disabled\' in the <a href="!link">boost advanced settings</a> & try again.', array('!link' => url('admin/settings/performance/boost', array('fragment' => 'edit-boost-ignore-flush-0-wrapper')))), 'warning');
  }
  $GLOBALS['conf']['boost_ignore_flush'] = $ignore;
}

Button 4 Code

/**
 * Resets boost database & cache directory
 */
function boost_reset_database_file_submit($form, &$form_state, $first = TRUE) {
  // Temp fix for bug in form api & buttons
  // Check to make sure this button was actually called
  if (!boost_admin_button_router(4, $first, $form, $form_state)) {
    return FALSE;
  }

  $ignore = variable_get('boost_ignore_flush', 0);
  $GLOBALS['conf']['boost_ignore_flush'] = 0;
  if (boost_cache_clear_all()) {
    db_query("TRUNCATE {boost_cache}");
    db_query("TRUNCATE {boost_cache_settings}");
    db_query("TRUNCATE {boost_cache_relationships}");
    db_query("TRUNCATE {boost_crawler}");
    _boost_rmdir_rf(BOOST_ROOT_CACHE_DIR, TRUE, TRUE, TRUE);
    drupal_set_message(t('Boost: Static page cache & 4 database tables cleared.'));
  }
  else {
    drupal_set_message(t('Boost: Set "Ignore cache flushing:" to \'Disabled\' in the <a href="!link">boost advanced settings</a> & try again.', array('!link' => url('admin/settings/performance/boost', array('fragment' => 'edit-boost-ignore-flush-0-wrapper')))), 'warning');
  }
  $GLOBALS['conf']['boost_ignore_flush'] = $ignore;
}

boost_cache_clear_all() only clears whats in the norm dir (not js/css); boost_clear_cache_parallel() & _boost_rmdir_rf(BOOST_ROOT_CACHE_DIR, TRUE, TRUE, TRUE); then clears out whats in the perm dir (js/css). If you've been using boost for a long time, you have the old way of storing stuff, where I differentiate via gzip instead of by type. If this is the case (my guess is, it is) change the Generated output storage (HTML, XML, AJAX) directories to "normal" and the Static storage (CSS, JS) to "perm". save, grab rules from boost-rules and update your htaccess file. This issue your having should go away then.

Close this issue if this solved the problem.

mcarbone’s picture

mcarbone commented

You are correct in that the "Dir for non compressed files" and "Dir for more permanent files (css/js)" fields are both blank. But before I follow your suggestion there, I want to clarify one thing: I've been clearing the cache using the "Clear cached data" button on /admin/settings/performance, and not any buttons on /admin/settings/performance/boost. If I used either buttons 0 or 4, wouldn't it just clear the cache and not clear out Drupal's actual aggregated CSS/JS files? Would I have to first manually update the css_js_query_string variable?

mikeytown2’s picture

mikeytown2 commented

I was assuming that you had the Ignore cache flushing: set to "Only Ignore Clear Entire Cache Commands (Recommended if caching css/js files)". If this is set to disabled; my guess is it is in your case then pressing the "Clear cached data" button will clear the boost cache & once you change the dirs over, it will not clear the boost js/css cache because they will be in a different dir. Adding in a button to only clear the js/css cache might be the solution your looking for; but it would only work correctly if you have the perm dir set.

mcarbone’s picture

mcarbone commented

OK, I set up the normal and perm dirs, and enabled "Only Ignore Clear Entire Cache Commands", and then pushed the "Clear cached data" button (and I got the warning that Boost didn't clear all of the cache). But I'm seeing quite a bit of 'page not founds' for aggregated CSS files, which tells me that the old aggregated files are being deleted before old cached pages sticking around.

I'm still not sure what the best strategy here is for pushing new CSS files.

mikeytown2’s picture

mikeytown2 commented

Best way to push new CSS is to use the "Clear ALL Boosts Cached Data" button. Since you changed the directories, it would be a good idea to nuke Boost and start with a clean slate; thats what the Reset Button at the bottom is for. The way boost is setup now, it won't flush the entire cache everytime cron runs, but will only flush the cache when those files are expired; puts you back in control of the cache. Let me know if you encounter any other oddities after reseting boost; just be aware its about the same as uninstalling and reinstalling it; except that the settings on the boost configuration page will not be effected.

mcarbone’s picture

mcarbone commented

Thanks, I did the reset.

Best way to push new CSS is to use the "Clear ALL Boosts Cached Data" button.

I still don't see how that alone will work. Unless the 'css_js_query_string' variable is modified, Drupal won't repackage the CSS into a new aggregated file.

mikeytown2’s picture

mikeytown2 commented

So its a 2 step process, sorry that I didn't make this clear. First hit the Clear Cached Data button on the performance page; then on the boost page hit the Clear ALL Boosts Cached Data button. If this answers your question then go ahead and mark this closed; if not keep asking questions :)

mcarbone’s picture

mcarbone commented
Status: Active » Closed (fixed)

Makes perfect sense to me. Closing.

drupalninja99’s picture

drupalninja99 commented

The problem is that you cant always know when that css cache is cleared. for instance many contrib module updates clear the cache. well thats a problem bc you dont know to also clear the boost cache.

My question is if you have cron running every 5 minutes and somehow something flushed the css aggregated caches will boost update the pages in that scenario? If cron always fixes the problem then maybe just running cron often would be the solution.