I've been experimenting with this module and filecache on a Acquia Dev Desktop, and am having some problems with new comments being shown in both 7.x-1.2 and 7.x-2.x-dev.

With both versions, new comments do not show up on the node page when you refresh the page after adding a comment, or navigate away and come back. However, the comment count in the view on the front page is updated, and you can see the new comment before you refresh the page. It is very similar to http://drupal.org/node/1881998, but the patch identified there has been implemented.

In 7.x-1.2, a number of warnings in pairs were generated in the log:
Warning: array_map() [function.array-map]: Argument #2 should be an array in _authcache_comment_num_new() (line 155 of C:\Users\gurrma\Sites\acquia-drupal\sites\all\modules\authcache\ajax\authcache.php).
Warning: array_combine() expects parameter 1 to be array, string given in _authcache_comment_num_new() (line 157 of C:\Users\gurrma\Sites\acquia-drupal\sites\all\modules\authcache\ajax\authcache.php).

But nothing on 7.x-2.x-dev. In Authcache debug, the ajax request for the comment form seems to be the same on the page where you can see new comments immediately after adding them, and when refreshing that page.

Can anyone shed any light on this? Thanks...

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

znerol’s picture

I've been experimenting with this module and filecache on a Acquia Dev Desktop

Would you mind switching back to the drupal built-in database cache and test again?

gurrmag’s picture

No prob...

Changed my settings.php to:
$conf['cache_backends'][] = 'sites/all/modules/filecache/filecache.inc';
$conf['cache_backends'][] = 'sites/all/modules/authcache/modules/authcache_ajax/authcache_ajax.inc';
$conf['cache_backends'][] = 'sites/all/modules/authcache/authcache.inc';
$conf['cache_class_cache_page'] = 'DrupalDatabaseCache';

Still the same problem with 7.x-2.x-dev.

I've just noticed two things in the authcache debug:

  • The order of the AJAX requests is reversed from when you submit a comment, and can see that comment, to when you refresh that page, and cannot see that comment.
  • It reports that caching has been cancelled on the page after submitting a comment because of the status message to confirm that you have posted a comment.

Not sure if these have anything to do with it?

znerol’s picture

The order of the AJAX requests is reversed from when you submit a comment, and can see that comment, to when you refresh that page, and cannot see that comment.

Actually only the new-indicator is updated during the AJAX phase, therefore IMHO it is save to ignore that while troubleshooting the case.

It reports that caching has been cancelled on the page after submitting a comment because of the status message to confirm that you have posted a comment.

This is expected behavior. No page should be written to the cache if there is a status message on it.

The problem you face is most probably that a stale copy of the cached page is served up after you submitted the comment. Actually the page cache is cleared (using cache_clear_all) from within comment_form_submit in drupal core.

I suggest investigating into two directions:

  1. Determine whether clearing of the page cache really works. Your best bet is to directly query the database, e.g. using SELECT count(*) FROM cache_page;
  2. Determine whether your browser is keeping a copy of the page in cache. This will be the case when you've set "Expiration of cached pages" in the Performance settings. You can determine whether your browser served up a cached copy using Firebug or the Chrome/Safari inspector.

Question: Do you test with anonymous users or a logged in one? Is the user you test with allowed to post comments without approval?

znerol’s picture

Another thing. Please deactivate the file-cache module entirely. And also only use the following two lines in your settings.php:

$conf['cache_backends'][] = 'sites/all/modules/authcache/modules/authcache_ajax/authcache_ajax.inc';
$conf['cache_backends'][] = 'sites/all/modules/authcache/authcache.inc';

You do not need to specifically set cache_class_cache_page, at least not with the 2.x-dev branch.

gurrmag’s picture

Thanks for the pointers...

I've turned off filecache completely, and have updated the settings as above.

I'm using an authenticated user that can post comments without approval.

The cache is not being cleared. I thought I might be able to use authcacheactions, which was working on 7.x-1.2, to get round this, but this is also not clearing cache pages.

Debugging that as its much smaller than authcache, I think the issue may be with keys. With authcache set for anonymous and authenticated, it tries to clear:
"1a8544http://sandbox.localhost:8082/content/caching-test"
"a544e0http://sandbox.localhost:8082/content/caching-test"

When I set authcache roles to only authenticated, no keys are generated.

However, the cache that needs to be cleared is: e536dba/content/caching-test.

I can't see an option to invalidate session ID's like there was in 7.x-1.2?

I'll see if I can find the relevant functions in authcache and check this out there...

znerol’s picture

Please try to keep the number of enabled third party modules down, so that we can find the root of the cause. Authcacheactions is not adapted to the 2.x branch yet.

Do you have a minimum cache lifetime set in Administration » Configuration » Development? From reading the code of DrupalDatabaseCache::clear it seems to me that this is the only setting capable of preventing cache being cleared by cache_clear_all.

gurrmag’s picture

Sorry about that... just looking for a way through this...

Going through cache_clear_all documentation, and I noticed this:

Otherwise, all cache entries that can expire are deleted.

The expiry timestamp in my cache_page table is set to -1. However, I've got minimum cache lifetime (and expiration of cached pages) in Admin>>Config>>Development>>Performance set to one hour. Not sure if that's the issue?

znerol’s picture

Title: New comments not showing on node pages » Make authcache work with minimum cache lifetime
Category: bug » feature

The expiry timestamp in my cache_page table is set to -1.

Actually the value of CACHE_TEMPORARY is -1. Drupal core uses CACHE_TEMPORARY for the page cache (see drupal_page_set_cache) and so does authcache.

Short answer regarding your cache settings:

  • Set the Minimum cache lifetime to none, it does not work together with authcache (for now).
  • Set Expiration of cached pages to a value you feel comfortable with. You should not set it too high though (couple of minutes probably).

Long answer (and as a note to self, such that I do not have to research it again):

The drupal cache-handling currently suffers from a bunch of short-comings. You find relevant information under all of the following links:

The reason why authcache currently does not work together with the Minimum cache lifetime setting is that in order to prevent serving up stale content to a user who recently triggered an action (like submitting a comment), the cache handler (DrupalDatabaseCache) stores timestamps in the users session. However at the point of time authcache retrieves pages from the cache (see authcache.inc) the session is not yet initialized. Therefore stale content is served to the user as a result.

If you need to set Minimum cache lifetime, you actually need to clear out the relevant cache-entries yourself (that's what authcacheactions is for.

I'm changing this into a feature request.

gurrmag’s picture

Thanks znerol... Really appreciate you taking the time to investigate this.

That's done the trick.

There's a lot to digest in those links... I'll need to re-read them as much of it went right over my head!

Probably an obvious question for someone with more knowledge of how Drupal caching works, but, once Authcache does work with minimum cache lifetime, would this also stop the entire page cache from being flushed when nodes etc are saved so something like authcacheactions could be used to selectively clear it instead?

Happy to help out with testing/documentation for authcache if I can...

znerol’s picture

once Authcache does work with minimum cache lifetime, would this also stop the entire page cache from being flushed when nodes etc are saved so something like authcacheactions could be used to selectively clear it instead?

Yes.

znerol’s picture

Just committed and pushed support-code for the Cache Expiration module. Watch out for the new 2.x-dev release (or pull the code from the git repository).

In Administration » Configuration » Development » Performance (admin/config/development/performance/expire). Disable the option Include base URL in expires. After that set Minimum cache lifetime to one hour in the performance settings. Test and please report any issues.

gurrmag’s picture

Thanks... Will do.

Think I might have finally wrapped my fragile brain around the basics of what those articles were getting at. Pulling all the threads together, for my own sanity if nothing else:

  • Authcache to work with minimum cache lifetime - see above;
  • Mechanism to clear caches selectively - Expire module (see above) and/or rules (authcacheactions with greater use of tokens, or integration with cache actions?) for views;
  • A reset/warming mechanism - something like cache graceful where caches could be pre-emptively rebuilt, or a crawler on a separate cron job. This should have the option of force clearing the cache and rebuilding it for sites with longer minimum cache periods such as a day. If this cron job was set up to run at midnight, it would reset the expiry of all content edited that day so it would remain cached, unless it was changed again, for the following day.
znerol’s picture

Don't blame yourself. There are only two hard things in Computer Science: cache invalidation and naming things -- Phil Karlton

Mechanism to clear caches selectively - Expire module (see above) and/or rules (authcacheactions with greater use of tokens, or integration with cache actions?) for views;

The Expire module actually provides a rules-action. I might be wrong but I think there is no need for authcacheactions anymore. I've chosen to support the Expire-module because it provides an easy but powerful API - and because everyone uses it, so chances are higher that it will remain supported.

gurrmag’s picture

Managed to give the latest dev a bit of a going over - summary attached.

So far as I can see, there's only two issues:

  1. Edit link on comments isn't working - it seems to be pulling in the first part of the URL alias from the page that you are on, i.e. http://localhost:8082/content/comment/3/edit. Not sure if this needs a leading slash somewhere?
  2. When the cache expiry is up for some pages and cron runs, it kills all pages, not just the ones that have passed the expiry time.

Hope this helps...

Any thoughts on keeping the cache warm/resetting the minimum cache period with a crawler to ensure that cached pages are always served - see #12? Is this a feature that you might look to add?

gurrmag’s picture

FileSize
23 KB

This time with attachment!

znerol’s picture

Thank you for your thorough tests, your feedback is very welcome.

I've found and fixed the issue with the comment-edit link. Watch out for a dev-release with todays date.

Regarding cron: This is actually a drupal-core problem. The page cache is emptied entirely after each minimum lifetime period passes. For example if a page was generated only one minute before a cron-run, it still will be wiped from cache when cron kicks in and the last cache-wipe was at least one minimum lifetime period ago. The function responsible for that behavior is system_cron. There are some third-party modules providing finer grained control over the cron-runs, for example Elysia Cron. Using this module you can run system_cron e.g. once per day and still run other cron-hooks more frequently.

gurrmag’s picture

No problem...

Happy to test some of the other sub-modules if needed...

Am I right in thinking that this core problem should be fixed with the patch in http://drupal.org/node/891600#comment-6862838?

If not, the fact that all cached pages are blitzed when cron runs may make having a crawler all the more important to try and undo some of the damage? At least you could then rebuild the cache, particularly for less popular pages, as soon as system_cron has completed.

Is there a roadmap for the development of authcache? In particular, have you considered support for Drupal Commerce as a sub-module?

znerol’s picture

Am I right in thinking that this core problem should be fixed with the patch in http://drupal.org/node/891600#comment-6862838?

Yes. However I have little hope that the patch actually lands. The problem is that it requires a db-schema alteration of all cache-tables. And that's probably too invasive for a minor update in D7. Additionally issues are normally fixed in the development version first (D8) and then backported to D7.

If not, the fact that all cached pages are blitzed when cron runs may make having a crawler all the more important to try and undo some of the damage?

Perhaps. You just have to keep in mind that a crawler also adds to the server load.

Is there a roadmap for the development of authcache?

I'd like to release 2.x soon. The architecture allows much better integration with third party modules. Concerning drupal commerce, I think it should work pretty much out of the box. Product listings are not personalized at all. It does not make sense to cache the shopping cart, so one has to make sure it is excluded. If the cart lives in a block, it probably makes sense to load that block using ajax (i.e. authcache_block). Do you see other potential issues?

gurrmag’s picture

You're right - it does work very well out of the box. There's just two things I'm not sure how best to handle. Both items are on product pages, which has images etc, so I am reluctant to clear the cache for these pages with rules unnecessarily.

First is the price field. This can be affected by a discount that is applied by rules, which would not appear on page load.

Second is the add to cart button, which can be disabled by rules if the item is out of stock. Although an additional stock check happens when you try to add a product to your cart, it may be better if the page accurately reflected the stock status.

These items (and others) are updated by ajax when you change product attributes via the function: commerce_cart_add_to_cart_form_attributes_refresh, so I don't know whether there's an easy way to make this happen when loading a product page in the first instance?

It would also be ideal if price could be updated with discount prices etc wherever it appeared, which could include product listing pages, search results.

Commerce also uses Search API - I haven't had a chance to look at how this behaves yet but I think Expire/Rules should be able to handle selective clearing of these cached pages.

gurrmag’s picture

The new dev version looks good - issue with edit link has been resolved. Is it worth repeating these tests with FileCache?

znerol’s picture

I'm not sure about the commerce-AJAX things. You should probably consult the guys over at the drupal commerce project. However keep in mind that delivering AJAX-calls will stress your server more than just serving cached pages. Depending on how often you need to recalculate prices it may be cheaper to deliver pages from the cache and just invalidate the product pages. It depends very much on the use case.

Is it worth repeating these tests with FileCache?

Sure if you want to use FileCache.

znerol’s picture

Status: Active » Fixed

I consider this fixed, now that we have expire-module support.

mcdruid’s picture

@znerol just for the sake of clarity - would you expect problems using the (currently Supported, Recommended) 7.x-1.2 release of authcache with a minimum cache lifetime > 0?

znerol’s picture

@mcdruid: Yes. In 7.x-1.2 we do not have any mechanism to invalidate certain pages (i.e. no support for Expire Cache module). Therefore problems like the one mentioned in the issue description will occur:

With both versions, new comments do not show up on the node page when you refresh the page after adding a comment, or navigate away and come back.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

RAFA3L’s picture

Hello, I can't make a selective clear cache for authenticated users after update some content, only work for anonymous users. I'm using the Expire module and all possible configurations. This is a normal behavior of the Authcache module? I tried with file and database cache and after update some content only is cleared the anonymous cached pages (front and edited node), not the cached pages with keys. And if I set the minimun life time to none all the cache is cleared.

Thanks in advance

znerol’s picture

In order to use the Expire module together with Authcache Builtin Storage Backend you also need to enable the Authcache Enum module. This definitely needs to be documented somewhere.

Do not hesitate to open a new support request if you still have troubles making it work. This issue is closed and it is easier for me to spot new issues than comments on closed ones. Thanks.

znerol’s picture

@RAFA3L: I just found a bug in Authcache Enum. Would you please try to apply the patch in #2154835: Integration of Cache Expiration and Authcache Builtin Storage Backend broken and report back there if it fixes the problem? Thanks in advance.

RAFA3L’s picture

Thanks! I'll try it tomorrow and comment here the results, for now I think the patch will fix the problem. This is one of the best modules for Drupal.

Sneakyvv’s picture

Issue summary: View changes
Status: Closed (fixed) » Needs review
FileSize
748 bytes

Is this really fixed?

Since authcache_builtin.module still uses CACHE_TEMPORARY...

cache_set($cid, $data, 'cache_page', CACHE_TEMPORARY);

I'm using File Cache and on every cron run (more specific the file cache cron) the whole cache is cleared.

I've attached a patch which sets the expiry date using the cache_lifetime variable.

Status: Needs review » Needs work

The last submitted patch, 30: authcache-use_cache_lifetime-1936108-30.patch, failed testing.

znerol’s picture

Status: Needs work » Closed (fixed)

If file cache does not respect the cache_lifetime, then this is a bug over there. Specifying an expiry time (i.e. maximum time to live) on cache_set is something completely different than the minimum cache lifetime.

So yes, this is fixed and the usage of CACHE_TEMPORARY is correct.

Sneakyvv’s picture

Thanks @znerol for pointing/clearing that out.

I have filed an issue with File Cache (#2530522: Clearing CACHE_TEMPORARY on cron).