Support for static page caching (Boost, Varnish, ...)

Would also be interested in this.

An alternative to a javascript could perhaps be to try and parse an accesslog from Apache or something - perhaps make such a feature extendable?

Also - since this might be a reusable feature it might be an idea to create a standalone module for logging access while having an aggressive cache turned on?

Our company, Dynamite Heads has decided to sponsor development of this feature since we need it for one project.

Putting a note here to avoid duplicating code (although nobody responded here for couple months).

First patch expected in a week.

Here is the patch. It works similarly as the statistics integration in Boost module.

1. Apply the patch to a radioactivity_node (it's a module in radioactivity/plugins dir).

2. Clear cache.

3. Go to admin/settings/radioactivity/node_radioactivity there is a new option "Mode". Enable "AJAX".

4. Go to admin/build/block/list and enable "Radioactivity Node: AJAX update" block.

5. Regenerate cached pages.

The block itself doesn't display anything. It'll only expose some javascript code along with invisible image - so it can handle requests even when the javascript is disabled in the user's browser.

Thanks. I'll check this out also. I have a customer case requiring Varnish support. Looking at the patch, this seems to be the solution also for that.

Note: Patch has dos/windows-style line-endings, so you need to use dos2unix (or similar) tool before applying the patch.

Attached is the patch in #3 with the following modifications:

• Broader explanations of modes in admin-pages.
• Check for proper mode in radioactivity_node_exit. This eliminates the double-energy for first views.
• Renamed the configuration variable as radioactivity_node_anon_hook_mode to better reflect the function.
• Added 'external' mode which disables information collection altogether. This is for setups where the energy comes from other sources, such as server analyzers.

The patch is for 6.x-1.3-rc1, but probably also works with 6.x-1.2.

I will test this some more and then make a commit, if no serious issues are found.

Ok, everything seems to work. Tested with Varnish, aggressive page caching, memcached on/off and node click duration on/off. Thus, committed:
http://drupal.org/cvs?commit=382708

If you encounter problems, please reopen this issue or start a new one.

BTW, I just put out 6.x-1.3-rc2 which contains this. Testing will be appreciated!

Amazing, thanks a lot! We will test this very soon.

P.S.: Radioactivity is an example of module done *right*. Thanks for it! It has much broader use than anybody can anticipate

Well, at least almost right this time. There's still a caching-related problem with Pressflow. Pressflow seems to cache also AJAX-responses, so explicit cache disabling is required. Otherwise, AJAX-based energy updates of anonymous users don't get registered, as cached reply is returned, and hooks don't get executed. Attached patch (apply to 6.x-1.3-rc2) should fix this. Will test a bit more before commit.

It seems like you're trying to smash through the cache with a GET-request? GET-requests are naturally cacheable since they shouldn't have any affect on the sites they are made to. The only reason to invalidate a cache of a GET-request is since it has been outdated.

Why not usa a POST-request instead? POST-requests are naturally uncachable since they are used to change stuff on the server and therefor always need to reach through all kinds of caches. Something that's respected by proxies, Drupal etc.

+++ plugins/radioactivity_node.module	21 Jun 2010 09:46:21 -0000
@@ -180,3 +191,104 @@ function radioactivity_node_views_handle
+$.get(Drupal.settings.basePath + "radioactivity_node.php", {nocache: "1", js: "1", nid: Drupal.settings.radioactivity_node.nid}, function(response) {});

Do a jQuery.post() here instead of a $.get()

Powered by Dreditor.

Right. Things magically worked on mainline Drupal without the patch in #12, because page_set_cache() doesn't put anything to cache if the page to cache is empty. Probably this is to prevent caching WSODs or something like that. However, in Pressflow that particular function is rewritten and empty pages may get cached, at least when page compression is enabled.

So, I'm happy with that explanation and here's the commit:
http://drupal.org/cvs?commit=383224

#13: We can't use POST in IMG-based update. So I think it's better to use only one method to cover both these cases. Nevertheless, proxies must honor the cache-directives in headers of responses for GETs. Otherwise, login-sessions would never work.

Anyway, this was a problem with Drupal internal page caching which only manifested in Pressflow, as it has a bit different logic.

#15: Login-sessions are based on a cookie set by a POST-request which differentiate one users requests from another? Do cache-directives really play any big role in that?

Anyhow - using POST for the javascript request doesn't make the img-based fallback function any worse - it just makes sure that the javascript request always smashes through the cache.

Take a look at core's page_get_cache() to see how it especially targets GET-requests - the same is also true for Pressflow.

You're right in that Drupal does considers caching only for GETs and only for anonymous users. For non-anonymous users, Drupal does not cache pages (the !$user->uid part in the if of page_get_cache()). But we prevent the AJAX response ever entering into Drupal page caches by commit in #14 so page_get_cache() never returns anything.

The commit is necessary as Pressflow may cache 0-sized pages but the mainline Drupal never does. This is prevented by if ($data = ob_get_contents()) in page_set_cache().

The cache directives in the HTTP-request instruct browsers and proxies to either enable or disable local caching. Here, you cannot make an assumption that GET requests are cacheable, if the HTTP-directives say so. If you force caching for all GETs, regardless of the directives, you're login-sessions amongst other things will get broken. Try it if you don't believe.

The reason we use GETs instead of POSTs is simple: In javascript we can do a POST just as you pointed above. However, we also want to support browsers with javascript disabled. In that case we use img-tag which fetches an empty image which gets registered by Radioactivity. Browser does GET to fetch the image, and AFAICT, it's impossible to make the browser do POSTs without user interaction (i.e. Submit-button etc) when javascript is disabled. As we can get non-cached GETs working with Drupal, proxy and browser caches, it is better to use GETs and GETs only, instead of doing POSTs in javascript and implicit GETs without javascript. Less corner cases and less code that way.

The multi-layer caching (Drupal, reverse proxies, normal proxies, browsers) can be a bit confusing.

What's the status of static page caching and radioactivity for Drupal 7?