Problem/Motivation

I am porting a module to D8 (Field as Block) which makes use of a post render cache callback. When I uninstall the module I get a white screen with a not-very-helpful error message:

Error
The website has encountered an error. Please try again later.

To reproduce this error:
- do a standard install
- install the fieldblock module (I assume any module implementing a post render cache callback can trigger this problem)
- create an article node
- set up a fieldblock using the Manage Display and Block Layout screens
- warm the render cache by visiting /node/1
- disable fieldblock (drush pmu fieldblock)
- refresh /node/1

The Drupal log (drush ws --extended) shows the following messages:

 ID  Date          Type      Severity  Message                                                                                    
 35  19/Oct 00:08  php                 Recoverable fatal error: Argument 1 passed to Drupal\Core\Cache\Cache::validateTags() must 
                                       be of the type array, null given, called in                                                
                                       /var/www/drupal8/core/lib/Drupal/Core/Cache/Cache.php on line 45 and    
                                       defined in Drupal\Core\Cache\Cache::validateTags() (line 63 of                             
                                       /var/www/drupal8/core/lib/Drupal/Core/Cache/Cache.php).                 
 34  19/Oct 00:08  php                 Warning: Invalid argument supplied for foreach() in                                        
                                       _drupal_render_process_post_render_cache() (line 3241 of                                   
                                       /var/www/drupal8/core/includes/common.inc).                             
 33  19/Oct 00:08  php                 Warning: Invalid argument supplied for foreach() in                                        
                                       _drupal_render_process_post_render_cache() (line 3241 of                                   
                                       /var/www/drupal8/core/includes/common.inc).                             
 32  19/Oct 00:08  php                 Warning: call_user_func_array() expects parameter 1 to be a valid callback, class          
                                       'Drupal\fieldblock\Plugin\Block\FieldBlock' not found in                                   
                                       _drupal_render_process_post_render_cache() (line 3242 of                                   
                                       /var/www/drupal8/core/includes/common.inc).                             
 31  19/Oct 00:08  php                 Warning: call_user_func_array() expects parameter 1 to be a valid callback, class          
                                       'Drupal\fieldblock\Plugin\Block\FieldBlock' not found in                                   
                                       _drupal_render_process_post_render_cache() (line 3242 of                                   
                                       /var/www/drupal8/core/includes/common.inc).                             
 30  19/Oct 00:07  system              fieldblock module uninstalled. 

It looks like the render cache still assumes that the module implementing the post render cache callback is still present. Rebuilding the cache (drush cr) solves the problem.

I expect Drupal to keep working (without the added functionality of the uninstalled module, obviously) after uninstalling a module. I am labeling this as major because it's pretty common use case, there is no helpful error message, and for people not using drush there is no good way to solve the problem.

Proposed resolution

I guess the best solution would be to automatically invalidate the render cache of objects affected by the uninstalled module. However I'm not sure if we actually have enough information to do that. Alternatively, we might be able to check if the post render cache callback still exists before trying to call it.

Remaining tasks

- Figure out the best approach for a fix.
- Make it work.
- Add tests?

User interface changes

none

API changes

none

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Wim Leers’s picture

When I uninstall the module I get a white screen with a not-very-helpful error message:

Because non-verbose error messages are enabled by default. See https://www.drupal.org/node/2259531 how to disable that while developing.


I guess the best solution would be to automatically invalidate the render cache of objects affected by the uninstalled module. However I'm not sure if we actually have enough information to do that. Alternatively, we might be able to check if the post render cache callback still exists before trying to call it.

Correct! And I thought this was already what was happening, but apparently, that's not true. I thought all caches were being cleared, but apparently that's not the case :(

To invalidate just the render cache, you only need to do this: Cache::invalidateTags(['rendered'].

It'd be great if you could work on this patch :) I'll provide reviews!

Wim Leers’s picture

Title: #post_render_cache callback breaks site when module is disabled » Render cache is not cleared when module is uninstalled
Priority: Major » Critical
Issue tags: +D8 cacheability
marcvangend’s picture

Thanks Wim, that will get me started. I will try post a patch tonight or later this week.

Trying to answer my own question about the best solution: Disabling modules is not something you do on a daily basis, so I think it's OK to invalidate render cache whenever any module is disabled. If we try to be smart about it and only invalidate render cache if the disabled module actually affects render cache, we're probably overcomplicating things.

Wim Leers’s picture

Correct notes, but more generally: ALL caches must be invalidated when uninstalling a module. Lots of things will break if that's not the case. That's why I'm surprised this doesn't happen already. In fact, I'm fairly certain it *does* happen. Hence I'm in fact pretty sure that this is caused by something else.

Berdir’s picture

This changed a few days ago, where we removed the drupal_flush_all_caches() jackhammer from uninstall() and the UI's.

So no, it doesn't happen anymore. And yes, I was expecting we would run into a few cases that we're missing.

I can imagine two things, one would be to go after the render cache explicitly and do a 'rendered' cache tag deletion, or probably better, copy the relevant parts of looping over all cache bins and call deleteAll() into uninstall(). Possibly check if there is something else that we need to clear.

Not sure if we need to flush assets for example? As they, I think, are always built based on a list of js/css that you want to have displayed, if the installed/uninstalled module affects that, it will just results in new asset caches?

Wim Leers’s picture

Ohhh! Then this is definitely critical. That explains a lot! Thanks for chiming in, Berdir!

marcvangend’s picture

Berdir, I tried to find the commit from a few days ago to which you are referring, but I couldn't find it. Also, I don't quite understand what you meant saying "[...] copy the relevant parts of looping over all cache bins and call deleteAll() into uninstall(). Possibly check if there is something else that we need to clear."

I'd like to help, but at the moment I don't know where to start. Any guidance please?

Berdir’s picture

The issue is #2248297: Ensure routes are rebuilt when install modules, but that will not help you much.

The part I was talking about from drupal_flush_all_caches() is this:

// Flush all persistent caches.
// This is executed based on old/previously known information, which is
// sufficient, since new extensions cannot have any primed caches yet.
$module_handler->invokeAll('cache_flush');
foreach (Cache::getBins() as $service_id => $cache_backend) {
$cache_backend->deleteAll();
}

That should fix your problem. After that, we need to check all the other stuff we do in drupal_flush_all_caches() and if there is something else we should call.

catch’s picture

Component: cache system » extension system
mpdonadio’s picture

I didn't see anything in drupal_flush_all_caches() that isn't covered in the

foreach ($module_list as $module) { .. }

loop, other than the code that @Berdir mentions, and what this patch adds back in.

Do we need additional assertions in UninstallTest and InstallUninstallTest? If so, what would be a cache tag that we can guarantee that would exist in those tests to check to see if it is not there? Maybe set one and check it after the uninstall?

mpdonadio’s picture

Status: Active » Needs review
Berdir’s picture

Status: Needs review » Needs work
Issue tags: +Needs tests

Setting a custom cache entry and ensuring it no longer exists after uninstalling a module sounds good to me.

The second sentence in the comment is specific to drupal_flush_all_caches() and should be removed. Instead, explain why we need to clear caches, something like "Caches might implicitly depend on the uninstalled modules, clear them explicitly.".

mpdonadio’s picture

Status: Needs work » Needs review
FileSize
2.4 KB
1.99 KB

I tested this fairly thoroughly by commenting out bits and pieces to make sure the test coverage was actually working. The \Drupal::cache()->set() is done immediately before the confirmation POST to ensure that nothing else in the chain has cleared caches.

Berdir’s picture

Status: Needs review » Active
Issue tags: -Needs tests
  1. +++ b/core/lib/Drupal/Core/Extension/ModuleHandler.php
    @@ -1030,8 +1030,8 @@
    +    // so clear all of thethem explicitly.
    

    Not sure what a "tethem" is :)

  2. +++ b/core/modules/system/src/Tests/Module/UninstallTest.php
    @@ -74,8 +75,19 @@ function testUninstallPage() {
    +    // Set a unique cache entry to be able to test whether all caches are
    +    // cleared during the uninstall
    +    \Drupal::cache()->set('UninstallTest', 'testUninstallPage', Cache::PERMANENT);
    

    Missing "." at the end of the comment.

    Cache keys are usually lowercase, something like 'uninstall_test'.

mpdonadio’s picture

Status: Active » Needs review
FileSize
2.4 KB
2 KB

"thethem" is a slang term in English for someone developing presbyopia who needs a new prescription from the optometrist. Or someone who makes a sloppy copy/paste :)

dawehner’s picture

Status: Needs review » Reviewed & tested by the community
+++ b/core/lib/Drupal/Core/Extension/ModuleHandler.php
@@ -1028,6 +1029,14 @@ public function uninstall(array $module_list, $uninstall_dependents = TRUE) {
 
+    // Flush all persistent caches.
+    // Any cache entry might implicitly depend on the uninstalled modules,
+    // so clear all of them explicitly.
+    $this->invokeAll('cache_flush');

Good! This is why documentation.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 15: render_cache_is_not-2359369-15.patch, failed testing.

Berdir’s picture

Status: Needs work » Reviewed & tested by the community
FileSize
19.73 KB

Re-roll, moved the snippet to the new class, should be still RTBC if tests pass.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 18: render_cache_is_not-2359369-18.patch, failed testing.

Berdir’s picture

Status: Needs work » Reviewed & tested by the community
FileSize
2.84 KB

Much stupid.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 20: render_cache_is_not-2359369-20.patch, failed testing.

bdurbin’s picture

Status: Needs work » Needs review
FileSize
2.85 KB

Attending to issue with patch 20.

Wim Leers’s picture

Status: Needs review » Reviewed & tested by the community
- +    $this->invokeAll('cache_flush');
+ +    $this->moduleHandler->invokeAll('cache_flush');

This is the only change compared to #20, and looks sane.

catch’s picture

Status: Reviewed & tested by the community » Fixed

Yep this is a case where cache tags don't really help much, and uninstalling is sufficiently infrequent that a full cache clear is fine.

Committed/pushed to 8.0.x, thanks!

  • catch committed 20799cf on 8.0.x
    Issue #2359369 by mpdonadio, Berdir, bdurbin: Render cache is not...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.