We really wanted to use this module on Acquia hosting, but the Varnish module cannot connect to the Varnish terminal there due to security reasons. The recommended way of purging items was to use CURL with some custom headers.

This can easily be integrated in a custom module, but it would be nice to have this working with cache_actions. I'm starting this issue to hopefully get an idea of different varnish setups and how cache_actions support could be generic enough to support most of them.

The following patch is very simplistic, but it does currently work for us on Acquia.

I also looked at the purge and expire modules and there's a lot of code that uses CURL to do the same. There could be optional integration with those or cache_actions adds similar support - duplicated efforts, but less dependency...not sure which way to go.

Looking forward to more input from others.

CommentFileSizeAuthor
#11 varnish_curl_purge-1152244-10.patch1.12 KBnicolash
#6 integrate_purge_module-1152244-6.patch2.45 KBnicolash
#1 varnish_curl_purge-1152244-1.patch2.8 KBnicolash

Comments

nicolash’s picture

nicolash commented
StatusFileSize
new varnish_curl_purge-1152244-1.patch2.8 KB

For this patch to work, a few variables need to be set in settings.php (or somewhere else) and I can only confirm that this works on Acquia:

$conf = array(
    'cache_actions_varnish_use_curl' => TRUE,
    'cache_actions_varnish_url' => 'http://www.example.com',
    'cache_actions_varnish_headers' => array('Accept-Encoding: gzip','X-Acquia-Purge:mysite'),
 );
fabsor’s picture

fabsor commented

Interesting. I have previously discussed adding support for the purge module, that works in a more generic way regarding purges, and currently provides support for nginx, squid and varnish.

I suggest that we try to get the Purge module to support Acquia hosting. Then we could provide a generic purge action that uses Purge to invalidate the cache.

nicolash’s picture

nicolash commented
Assigned: Unassigned » nicolash

Sounds good. I'll file a feature request over there once I got a patch for purge module to add Acquia support (or more specifically the ability to add custom HTTP headers for the CURL request).

The next question would be where this would be configured for cache_actions, since there isn't really a config page where one could choose between the varnish or the purge module. I'm new to rules...don't know where this would be placed best.

Thanks for your quick response, BTW...

fabsor’s picture

fabsor commented

I think that we should have a different action for invalidating the cache using the purge module, since it supports more reverse proxy cache servers than varnish.

To add a new action, the only thing you need to do is add it to the cache_actions_action_info function and then create a form that can handle the options we want to have. The other actions in the module is set up in exactly the same way.

For more generic information see This part of the rules documentation.. Please let me now if you run into any problems. I would love to be able to support this in the module.

nicolash’s picture

nicolash commented
Title: Basic Varnish support without Varnish module (using CURL) » Integration with Purge module (support for nginx, squid and varnish)

Cool, makes perfect sense. I'm ready to test the Purge module modifications tomorrow, then will submit the patch.

Changing the title for now...

nicolash’s picture

nicolash commented
StatusFileSize
new integrate_purge_module-1152244-6.patch2.45 KB

I've started an issue over at Purge: #1154994: Provide ability to add custom headers (for Acquia support etc)

This patch does provide an action to use Purge. However, currently Purge really is an extension to Expire, and they probably would try to do the same thing as Cache Actions most of the time. Therefore it would be good to not have Expire enabled to use CA/Purge - that's part of the feature request above.

I also have left a fallback option in there for CA to use its own CURL request without any dependencies. It's pretty lightweight, but could maybe be a bit expanded upon....or scrapped entirely.

fabsor’s picture

fabsor commented
Status: Active » Needs review

The patch looks good at a first glance. I will test it out and review this in more detail as soon as I can find the time.

Regarding the basic purge action, it is too bad the purge module requires the expire module to operate. I would really like it if we could avoid having this kind of functionality in cache actions, since I don't want it do do anything else than provide actions for invalidating the cache using other modules. That way we can separate issues having to do with the actual invalidation of different caches from this module and deal with them where it's appropriate, may it be in core, the varnish module or in views.

The expire module makes a lot of assumptions for cache invalidation that might hold true for most people but it is not desirable in some cases, for instance if you want to invalidate the cache for one kind of node, but don't mind have other node pages cached for longer periods. I will post an issue in the purge queue and ask if it's possible to remove the dependency in order to use the module purely as an API for other modules. After all, it won't affect the module's current functionality if you have the expire module enabled.

fabsor’s picture

fabsor commented

I looked into the expire module a bit and I realized it actually has an admin page where you can select what you want to purge automaticly. Since you could actually turn of the automatic purging, I think adding it for using purge isn't a big deal. It would still be good to avoid having it enabled though.

nicolash’s picture

nicolash commented

Yes, I also think with the right configuration they can happily co-exist...with the wrong config they could do a lot of things twice and fall over each other. Perhaps more a documentation issue.

The patch I submitted for Purge actually does remove the dependency on Expire (without changing the functionality).

Look forward to your test results and suggestions.

SqyD’s picture

SqyD commented

Hi,

This duplicates functionality already in the expire module and therefore I hope to convince you all not to go this way.

I am the maintainer of the purge module and I help out with the expire module too. I had this discussion with Mike Carper (Expire, Boost) here #1050526: Drush integration and here http://groups.drupal.org/node/125094#comment-406739
and now with Nikolas here #1154994: Provide ability to add custom headers (for Acquia support etc)

Please consider the points made in those posts. I've also pinged Mike about this.

I will not drop the purge dependency on expire anytime soon and I have yet to hear about any problems the expire implementation of the rules integration.

nicolash’s picture

nicolash commented
StatusFileSize
new varnish_curl_purge-1152244-10.patch1.12 KB

Yeah, agree about keeping the actual curl functionality out of this module, so I've removed it and will concentrate on contributing to purge if anything special is needed.

fabsor’s picture

fabsor commented

I didn't know there was rules integration in expire! When did that happen? =P

Since there is rules integration for expire supporting purging of particular URL:s, I feel that having the functionality here as well is pretty unnecessary, since purge module uses expire anyway. I might even consider dropping the varnish support in cache actions altogether, since the varnish module also supports expire.

@NicolasH Is there something missing in the expire module's rules integration? Otherwise I think we should drop the functionality in cache actions and just let expire and purge handle everything.

nicolash’s picture

nicolash commented

To be honest, I didn't really know that either until SqyD's comment. I will be using both for some more complex scenarios and will let either party know if something might be worth moving from one to the other.

However, both modules seem to have different areas they cover for cache clearing. While Expire covers nodes, taxonomy, comments, user pages and voting API related pages, I can't see it having any support for Panels and Views.

So if Expire is the definitive cache clearing tool for external cache systems, shouldn't Panels and Views functionality be moved over there as well? Or maybe have a link between the 2, perhaps via hook_expire_cache()?

Cache Actions could still just clear the panel and views caches, but also flag which actual pages are affected and notify Expire to clear those.

This kinda goes way beyond the original patch and I'm happy for you to drop it....but I'd also really like to help out with getting all these approaches work together. Look forward to your take on this, fabsor and SqyD - this could work out nicely.

SqyD’s picture

SqyD commented

Views and panels support is coming to Expire. Most of the code in expire originates in Boost. Mike Carper is working on adding views and panels support to boost now. After that is done it will be brought over to expire.

Expire is positioned as the definitive API module for cache clearing. Purge implements it's cache clearing hooks. There should be no competition or overlap.

I'll see if we can put up some text explaining this more clearly...

fabsor’s picture

fabsor commented
Status: Needs review » Closed (won't fix)

I'm closing this issue, since going with the purge module is probably the way to go.
Cache actions still supports panels and views caching and it is not interfering with expire in any way right now, so there should be no problem using the modules side by side.

nicolash’s picture

nicolash commented
Assigned: nicolash » Unassigned

Good, thanks for the clarifications.