I've updated my drupal site from drupal 6.19 to drupal 6.22. After the update most of my blocks were disabled, only the navigation appeared together with 'some' primary links. Additional menus were set to the disable section. On top of that none of my views was visible in block form anymore and when I tried to go directly to that view's view page I got a WSOD. I had to enable my blocks again with all their configurations (a pain!) and I increased the memory limit in php.ini from 96MB to 132MB. Now it seems to be working again. I'm thinking to revert back to my back up and wait for a more stable release of the 6 version.
If any of you out there experienced similar behaviour I'd love to hear it.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

YK85’s picture

subscribing

franzkewd’s picture

Subscribe

Juan C’s picture

I upgraded from 6.20 to 6.22. No error reports, admin pages doing fine, but 'blank/white' page shown on frontpage and other menu pages.
Subscribe.

Update: It is fixed now. I forgot to put back some custom codes on /includes.

design.er’s picture

After an upgrade from 6.20 to 6.22 all my blocks were disabled.
I've enabled them and it seems to work fine. But 2 hearbeat blocks are automatically disabled a few minutes after I activate them... this happens again and again. That's spooky!

scottybrookie’s picture

I updated from 6.20 to 6.22. After the update, not only were nearly all my blocks disabled -- top and side navigation among them -- they have also lost their respective configurations. That means that when I re-enable them, they all appear on all pages whether they belong there or not. I am in the process of reconfiguring them now, which I hope I get right. Subscribe.

Araucaria’s picture

I'm experiencing a similar issue. Upgraded from 6.20 to 6.22, and now my Primary Links menu block is being disabled. (For me, it seems to be only that one block which is affected.) I can re-enable it and it then works fine, but it gets disabled again after a few hours.

mcurry’s picture

subscribe

cgrant3d’s picture

subscribe

Araucaria’s picture

The problem seems to recur at the same time every day, so I'm guessing something running in my cron job is disabling the menus

Gábor Hojtsy’s picture

Sounds like this is happening due to #235673: Changes to block caching mode not caught. Looks like there was not enough testing there despite many people saying it worked fine for them in practice :|

Gábor Hojtsy’s picture

Title: blocks and views disappear after core drupal 6.22 update » Blocks disappear after core drupal 6.22 update

The views mentioned in the title are blocks exposed by views. Make title shorter for brevity.

Eric_A’s picture

Priority: Normal » Major

Whenever caches are cleared this may occur. Looks like it will happen when a block providing module is not loaded. One such a case is when updating core.

Loosing block data from the blocks table looks at least major to me.

EDIT: clearing caches is one way to trigger this and this is what is happening now in 6.22. But in fact any call to _block_rehash() can kill blocks under circumstances.

Eric_A’s picture

Looks like it will happen when a block providing module is not loaded.
Actually, it's about what module_list() returns which is not exactly the same.

jbrauer’s picture

@Araucaria re the time of day in #9 is it exactly the same time each day or is it 24-hours + the cron interval? If it's the latter it may be that it's related to the rebuilding of the system table done by the update status module.

LetUsBePrecise’s picture

I did not see any problem. Subscribing to the post.

aveldina’s picture

I am experiencing this issue and most, but not all, of my blocks were disabled. Configuration seemed to be lost but when I checked the block and resaved it, the configuration started working again. Subscribing. Edit: Looks like I don't lose block settings after running cron or clearing the cache.

sun’s picture

Priority: Major » Critical

This sounds pretty critical to me.

carlos8f’s picture

Assigned: Unassigned » carlos8f

I'll try to work out a fix tonight for this.

kristin.e’s picture

subscribing
My Views block has completely disappeared after updating to 6.22. Also, my facebook social plugins gone.

carlos8f’s picture

So far I haven't been able to reproduce. Here are my steps:

- git checkout 6.19
- install drupal using defaults
- add an item to the Primary Links menu
- assign Primary Links block to right sidebar
- git checkout 6.22
- update.php (ran color module update)
- view site, Primary Links are still in the right sidebar
- clear caches on Performance page, Primary Links still there.

Can someone come up with steps to reliably reproduce this? I will also be glad to accept a database dump of an affected site (before the update).

mkindred’s picture

Yep, I'm seeing this. Upgraded from 6.20 -> 6.22. All views blocks and Nice Menu blocks disabled and configs lost.

I'm not sure whether this is related, but I also noticed that the order of the active themes changed on the main block admin page.

Reverting to 6.20 until this gets fixed.

JimCraner’s picture

The issue I'm seeing has got to be related - my views' block displays are cached and unchangeable. What's weirder is that when I go to edit Views, I am (randomly?) seeing OLD values that I had already changed and saved.

(Note: I didn't lose any blocks or block configs)

1) Edit a view with block display
2) Change the footer (or anything else) and save the view
3) Note that the view's block's footer has NOT changed
4) Clear browser cache, clear Drupal caches, clear Views cache
5) Repeat step 3 and swear loudly
6) Manually truncate cache tables in MySQL
7) Repeat step 5 but swear even louder

Throughout this process, if I go back into Views -> Edit and look at the footer field, it randomly displays the footer I had in there before Step 1. Save it, re-edit: it's the footer I changed in Step 2. Save, re-edit, and sometimes it's back to the Step 1 footer. Bizarre.

carlos8f’s picture

Version: 6.22 » 6.x-dev
Status: Active » Needs review
FileSize
1.25 KB

Thanks to a db dump from @arsnova I was able to identify the cause and solution.

The db dump I received had all the contrib modules disabled (views, nice_menus, etc.). After update.php, those entries did get deleted from the {blocks} table. That is expected though, because the cleanup mechanism of _block_rehash() deletes rows that are no longer defined by hooks, which would include blocks from disabled modules.

Although it states in UPGRADE.txt to disable contrib modules before an update, this should not be necessary for minor version revisions like 6.20 -> 6.22. Even so, it makes sense to avoid data loss in that scenario :)

In D6, I think it's always been the case that if you disable a module and then visit the block config page (_block_rehash), configuration for the disabled block is cleaned up (lost). In D7, #560746: Rename hook_block_info_alter() to hook_block_list_alter() and add hook_block_info_alter() DBTNG'd _block_rehash() and also added a subtle change: only blocks from enabled modules are now cleaned up. That explains why the bug doesn't affect D7.

In a simplified manner, the attached patch backports this cleanup change to D6. When I did the update with this patch applied, the views and nice_menus blocks retained their settings.

If anyone has trouble with enabled modules' blocks disappearing, that is another issue. If that can be reproduced, I would love to learn how.

design.er’s picture

yeah, the patch in #23 works like a charm. Thank you so much, you made my day! :)

carlos8f’s picture

Title: Blocks disappear after core drupal 6.22 update » Disabled modules' blocks lose settings during update.php and cache clears

Emphasizing that this only affects disabled modules.

sun’s picture

Hm. So @design.er reported success and the issue might have been limited to blocks of disabled modules only. However, the other comments including the OP of this issue do not really sound like it was limited to blocks of disabled modules only. Though of course this bears the question whether modules have been "temporarily" disabled to perform the core update.

@BassPlaya + others, can you clarify, please?

Eric_A’s picture

Regarding minor D6 updates I said this in the other issue:
_block_rehash() calls module_list() without any parameters which in in the case of a site update will return just system and filter
If this statement is correct than _block_rehash() deletes rows even for blocks provided by enabled modules. The good news is that the patch from #23 will prevent this because as far I can tell this patch will make sure that at the end of updates only missing blocks from system and filter will be deleted. (Meaning no blocks.)

carlos8f’s picture

@Eric_A, I think you're mistaken about module_list() only returning system and filter. How do you explain then, that hook_flush_caches() is invoked for the block module at that point?

To be sure, here's a debug directly before drupal_flush_all_caches() in update.php (using 6.22 tag):


function update_finished($success, $results, $operations) {
  $fp = fopen('./sites/default/files/debug.log', 'a+');
  fwrite($fp, "-------------\n");
  fwrite($fp, "module_list:\n". print_r(module_list(), TRUE));
  // clear the caches in case the data has been updated.
  drupal_flush_all_caches();

Output:

-------------
module_list:
Array
(
    [block] => block
    [color] => color
    [comment] => comment
    [dblog] => dblog
    [filter] => filter
    [help] => help
    [menu] => menu
    [node] => node
    [system] => system
    [taxonomy] => taxonomy
    [update] => update
    [user] => user
)

If it were true that module_list() returned only system and filter at this stage in the code, it would be a huge bug unto itself, and gladly that is not the case here.

However, it's conceivable that a contrib module inside hook_flush_caches() called module_list(TRUE, TRUE), and populated the static cache with only bootstrap modules and then block module picked up the wrong list, but that would be the contrib module's fault for screwing up the module list. module_list() without params is fine and filter, search and user modules use that syntax as well.

igormt’s picture

Version: 6.x-dev » 6.22

Hi, I was led to this thread because since updating to 6.22, all my *active* blocks reset to "none" from whatever region they are assigned to everytime the Drupal cron fires. My sites uses the "Green N Black" theme.

I wonder if this is related to the problem described in this thread. I'm a Drupal user, not a Drupal coder, but my PHP skills are decent. If someone could please point me in the right direction for where to look in the code, it would help me to better figure out what is happened. Thanks.

igormt’s picture

Follow up: when I reverse the patch from #235673, the problem with cron resetting active blocks to "none" goes away.

Eric_A’s picture

@carlos8f, thanks for pointing things out. I looked again and now have a much better understanding of what is happening after the call to drupal_maintenance_theme() and before update_finished().

If anyone has trouble with enabled modules' blocks disappearing, that is another issue. If that can be reproduced, I would love to learn how.
What would happen when system_cron() is clearing caches while throttle module is controlling congestion?

carlos8f’s picture

Title: Blocks lose settings during update.php and cache clears » Disabled modules' blocks lose settings during update.php and cache clears

What would happen when system_cron() is clearing caches while throttle module is controlling congestion?

Throttle module only filters the rendering of blocks via block_list() (which loops through the {blocks} table and invokes the "view" op). That has no affect on _block_rehash() (populates the {blocks} table with the "info" op) since there we use the module API directly: module_list() and module_invoke($module, 'block', 'list').

So there's no problem with the throttle module. edit: throttle module actually mucks up module_list() too, which is a problem.

Eric_A’s picture

Version: 6.22 » 6.x-dev

Sorry for being not specific enough. I was not talking about block_list(). The way I understand it: when throttle status equals 1 (congestion) module_list() will not return modules that are checked for throttle on the modules page. Looks a bit weird and also dangerous to me that you can have all of a module temporarily "disabled" and then "enabled" again, but it appears you can make this happen in Drupal 6. Until a few days ago I did not even know that this existed.

webservant316’s picture

subscribe. same problem here. blocks disabled after upgrade from 6.20 to 6.22. also block configurations settings lost on some disabled blocks, but not others. curious.

first, @carlos8f mentions above 'Although it states in UPGRADE.txt to disable contrib modules before an update, this should not be necessary for minor version revisions like 6.20 -> 6.22. Even so, it makes sense to avoid data loss in that scenario :)' so, to simplify this upgrade is it possible to perform the upgrade without disabling any contributed modules? from reading this post it seems like all would be well. then a patch for this problem could be included in a future release of Drupal 6.

second, I got lost in the details of this post after the patch proposed by @carlos8f. there seem to be concerns with the patch in comments that follow. so does the patch stand?

carlos8f’s picture

Title: Disabled modules' blocks lose settings during update.php and cache clears » Blocks lose settings during update.php and cache clears

@Eric_A, it looks like you're right, throttle module actually alters module_list(). So as silly as that is, it's one possible way enabled modules' blocks can disappear in 6.22.

Patch in #23 should fix that though, by only cleaning up old block deltas from enabled modules. That's what D7 does, and backporting it should fix all the issues mentioned.

webservant316’s picture

Title: Disabled modules' blocks lose settings during update.php and cache clears » Blocks lose settings during update.php and cache clears

still trying to figure out which patch to apply...
do we use #116 from http://drupal.org/node/235673 or use #23 in this posting?

carlos8f’s picture

#23 is still good. Reverse-applying #116 from #235673: Changes to block caching mode not caught is also an option but I have confidence that #23 is the correct approach.

webservant316’s picture

ok, I think I understand now. #116 was already applied in the 6.22 release so leave it as is and also apply #23 manually?

pwolanin’s picture

Patch looks quite reasonable - is there any history as to why this change was made in D7?

carlos8f’s picture

@pwolanin, as far as I can tell, the change was probably an unintended side-effect of chx's _block_rehash() refactor in #560746: Rename hook_block_info_alter() to hook_block_list_alter() and add hook_block_info_alter(). It does make for a more sane clean-up process though.

Reasoning: If an enabled module no longer provides a block with a certain delta, it should be cleaned up (otherwise might result in broken blocks displayed). But a module that's missing from the module_list() (at least in D6) could be due to throttle module, or disabling a module temporarily for update.php (which UPGRADE.txt recommends) or other mysterious reasons so cleanup should be delayed/avoided.

brunorios1’s picture

+1

temelkoff’s picture

sub

Fabianx’s picture

Hi,

I've looked into the issue, why #235673: Changes to block caching mode not caught introduced that problem. The reason is the implementation of hook_flush_caches().

hook_flush_caches can be called under lots of different conditions and such if it gets called from an environment, where not all modules are loaded the wrong behavior is exhibited.

Cases include:

* update.php - clears the cache always at the end
* cron - if a module clears the cache there
* throttled modules status
* hook_boot - If a module clears the cache here, it shouldn't, but oh well ...

This pretty much sums up the experiences people had with #235673: Changes to block caching mode not caught applied.

Note that the problem with throttle was also present before #235673: Changes to block caching mode not caught, when the blocks page was loaded and throttle did disable modules.

While the patch in #23 works really well and fixes the issue, it only fixes it partially, because cache settings of a module could not be adjusted as it was run out of context with not all modules loaded.

I however think this whole approach withh hook_flush_caches of the original patch is WRONG due to the following reason:

* hook_flush_caches API is used for returning cache tables to be cleared. This case basically is a misuse, which many modules do, but which is still wrong.

What block module _should_ do (after applying #23):

* Add a persistent cache_set('block:block_list_builded') at the end of _block_rehash.
* This is cleared on cache
* In hook_init: If cache_get('block:block_list_builded') is missing, run _block_rehash();

This is consistent with many modules rebuilding caches on the first page load afterwards.

-- Throttle problem --

Of course by running it in hook_init the probability of running into the throttle problem are higher. On the other hand system upgrades and cache clear alls really should be done in maintenance mode, so I guess the risk is neglectable.

What _could_ be done is to display a message and add an error to watchdog - if throttle module is active - that not all modules status could be updated, but that seems like a rather low edge case.

If the usage of hook_flush_caches is deemed appropriate ( I still think no build ops should occur there and this is a misuse of the API ), then #23 can be set to "RTBC", but if not this should be set back to "needs work".

I vote for "needs work" and attached a little patch doing both changes: #23 and my proposed changes.

Best Wishes,

Fabian

Fabianx’s picture

Status: Needs review » Needs work
FileSize
2.07 KB

Hi,

UPDATE: This definitely _needs work_ as it rehashes the block list on each cron run.

system.module runs hook_flush_caches to clear expired variables.

By misusing hook_flush_caches the blocks are rehashed on each cron run, so this issue happens whenever a module is not enabled during cron runs. (same happens in strongarm currently, there is also a little more discussion about this here: #944970: Useless implementation of hook_flush_caches())

The attached patch removes the misuse of hook_flush_caches.

Unfortunately this will mean one more DB Hit for each page load (if no memcache, etc. is used).

This could be solved by using a variable_set / variable_get instead. But the variable_set would again need to occur in hook_flush_caches, which would at least clear the variables cache on each cron run.

Tricky issue indeed.

Thoughts?

Best Wishes,

Fabian

brunorios1’s picture

after i upgraded from 6.0 to 6.22, the /admin/build/block/list/ just shows the regions and blocks of the administration theme configured at /admin/settings/admin and ignore the other themes so i can't configure the blocks and regions anymore...

anybody having this problem too?

more details: http://drupal.org/node/281643#comment-4593406

thanks.

dwhutton’s picture

subscribe

Eric_A’s picture

@FabianX, the more principled issue that you are tackling in #44 is not specific to D6 and needs to be applied and tested against D8 and D7 before it makes it to D6. I suggest opening a new issue against D8 (that will have backports for D7 and D6) and have this issue focused on the D6-specific fix from #23. If you agree, then please set this one back to "needs review".

New minor releases for D7 and D6 are not far away, so let's see if we can get something RTBC.

Fabianx’s picture

@Eric_A:

I unfortunately can't agree on that as misuisng hook_flush_caches is bad, but if you check that cron is active and don't do the rebuild, I can live with that.

Getting something out would be nice though.

Best Wishes,

Fabian

Eric_A’s picture

Fabian, I absolutely agree with you that the current block_flush_caches() implementations in D8/D7/D6 are wrong. My point is that the patch in #44 will not be committed to D6 before it has been committed to D8 and D7, while #23 does have a chance. Truly cleaning up implementations of hook_flush_caches() will take more time than we have now to fix D6. This is why I want to work with you on the bigger fix in a separate issue and keep this one for discussing how to quickly rescue D6.

The approach in #23 is the one that could make in into D6 quickly because it more or less just brings D6 in line with code that is already in D7 and D8.

I too want to investigate how to keep hook_flush_caches() as an info hook only and moving everything else to other places. Maybe in D8 this hook should eventually be renamed to something like hook_cache_bins_info() or something like that. But D6 should be fixed ASAP, then the hook implementations can be cleaned up in D8 (and backported) and then we might rename the hook in D8 (which can not be backported.)

Fixing the data loss ASAP in D6 means either rolling back #235673: Changes to block caching mode not caught in D6 or going with something like #23. It's better to keep this discussion separate from a discussion that is on a different level and relevant to Drupal 8 before it becomes relevant to D7 and D6. If we're really, really fast even both issues might be solved and make it into D6 before the next minor point releases. ;-)

Fabianx’s picture

Status: Reviewed & tested by the community » Needs work

@Eric_A: #23 needs work to at least check for cron mode if we want to have any chance at all to get this in.

We won't get something in that is rewriting blocks on each cron run.

[EDIT: This is wrong of course, we already have something in that is rewriting blocks on each cron run. WTF! ]

Best Wishes,

Fabian

Eric_A’s picture

Status: Needs work » Needs review

EDIT: added issue priority levels.

Here is my issue summary. Correct me if I'm wrong.

The patch in #23 deals with D6 specific internals of _block_rehash(). The critical bug it fixes is that D6 _block_rehash() currently may erroneously delete block settings, which is not the case in D8/ D7 _block_rehash(). The bug manifests itself since the introduction in 6.22 of block_flush_caches(). This block_flush_caches() implementation - which calls _block_rehash() - is already in D7 for some time, but in D7 the implementation of _block_rehash() will not erroneously delete block settings. So the patch in #23 brings D6 in line with D7. The other D6 specific fix is to simply roll back the patch that introduced the new code in 6.22: #235673: Changes to block caching mode not caught.

FabianX identified a related non-critical issue and rolled a patch against D6 in #44 that deals with the bigger picture. However, this bigger picture is not D6 specific, but is in the full D8, D7, D6 scope. We cannot deal with the bigger issue in D6 before it is dealt with in D8 en D7. So that leaves us with at least two options:

1) A new issue is opened to work on #44 rolled against D8. The old issue here will then be used for working on the identified D6 specific issue.
2) This issue moves from D6 to D8 and we review #44. We forget about the D6 specific fix by carlos8f and go for the bigger picture. This probably means that #235673: Changes to block caching mode not caught will be reverted in the next D6 release.

On a side note: _block_rehash() is not "rewriting blocks". It does compare blocks in the database with blocks in code and updates the block cache setting or deletes all its settings if it is determined by the comparison that this needs to be done.

I believe that this specific issue should stay focused on the critical bug that is specific to D6. It may be decided to roll back to the situation before 6.22 or that the work of carlos8f in #23 gets in. In other words: #23 needs further review or an RTBC status.

Fabianx’s picture

Status: Needs review » Reviewed & tested by the community

Hi,

While #235673: Changes to block caching mode not caught introdcued another misuse of hook_flush_caches this is indeed the case with D7 and D8 for many many occurences already. (WTF!)

So I am absolutely fine with #23 to just go in.

The patch in #23 solves the problem at hand by checking that the modules are actually loaded of which the blocks should be disabled.

This will allow together with #235673: Changes to block caching mode not caught to still clear the cache and have block cache settings updated, but it won't happen automatically anymore. But as most site-builders do clear caches after upgrading modules that point is moot and may not have been even wanted by the original proposal.

However this is a really minor thing compared to broken installations like we have here => RTBC

Thanks,

Fabian

PS: @Eric_A: You have been right, I am sorry. I mistakenly assumed #23 would have introduced the hook_flush_caches, but it rather fixes the mess. We can indeed mess with breaking our own API in other threads. Lets get this one done, first.

Liliplanet’s picture

Status: Needs work » Reviewed & tested by the community

#23 works perfectly, thank you!

shineshadow’s picture

subscribe

Liliplanet’s picture

oops sorry re #53, spoke too soon, issue still exists ..

When you manually run cron it's fine but when system cron runs it disables certain blocks for the following modules:

http://drupal.org/project/heartbeat
http://drupal.org/project/content_complete

Gábor Hojtsy’s picture

Status: Reviewed & tested by the community » Needs work
mastershake229’s picture

subscribe

carlos8f’s picture

As for content_complete, I think it's a bug in that module rather than the patch:


function content_complete_block($op = 'list', $delta = 0, $edit = array()) {
  if (user_access('access content complete')) {
    switch ($op) {
      case 'list':
        $blocks = array();
        $types = content_complete_get_tagged_types();
        foreach ($types as $type => $name) {
          $blocks[$type]['info'] = t('Content Complete: !name', array('!name' => $name));
          $blocks[$type]['cache'] = BLOCK_NO_CACHE;
        }
        return $blocks;
        break;
      case 'view':
        $type = $delta;
        $types = node_get_types('names');
        if (! isset($types[$type]))
          return;
        $complete_data = content_complete_get_data($type);
        $block['subject'] = t('!name', array(
          '!name' => $types[$type] 
          )
        );
        // only display the block when 100% has not been reached
        if (! empty($complete_data) && $complete_data['percent'] != 100) {
          $block['content'] = theme('content_complete_profile_percent_complete', $complete_data);
        }
        return $block;
        break;
    }
  }
}

Wrapping the entire hook in a user_access() call makes no sense and also makes it impossible for the system to enumerate the blocks reliably. So if the anonymous user doesn't have this permission, and cron is commonly run under anonymous, blocks will drop out since they are assumed to be no longer implemented. Also, even without my patches, if a user has "administer blocks" but not "access content complete", the same effect will happen when visiting the blocks page: they will be unable to see or administer the content_complete block, and in fact will accidentally erase any block settings for that module via _block_rehash().

What this module should do, is only check user_access() when the $block['content'] is formed. An empty $block['content'] causes the block to be hidden when rendered, which is more along the lines of what is intended with the access check.

As for heartbeat, so far it looks like it doesn't implement hook_block(), but exposes views which can be turned into blocks. So I'll have to investigate how views implements hook_block(). see below

izmeez’s picture

subscribe

carlos8f’s picture

Status: Needs work » Needs review

heartbeat has the same issue with user_access() in the wrong place:


function heartbeat_block($op = 'list', $delta = 0, $edit = array()) {

  if (!user_access('view heartbeat messages') || !variable_get('heartbeat_enabled', 1)) {
    return FALSE;
  }

  ...

$op = 'list' is an admin-time op, whereas $op = 'view' is the runtime op. So user_access() needs to target the runtime op and not the admin one. $op = 'list' is already restricted via 'administer blocks'. Again, this will lead to buggy behavior such as blocks disappearing if the block administrator doesn't have 'view heartbeat messages', etc., regardless of my block patches. So I believe the patch still stands.

Fabianx’s picture

@carlos8f: Lets add a check to hook_flush_caches that it is not run by anonmyous. That is easy to do and work arounds those wrong module behavior efficiently.

In the worst case blocks have wrong settings until next cache clear, but that is a frequent sys admin operation when something is wrong, so it should be fine.

EDIT; That does not work as drush is running cache_clear_all() also as anonymous. Oh, well ...

Best Wishes,

Fabian

dimitriseng’s picture

Subscribe.

pearlliang’s picture

Category: bug » support
Status: Needs review » Active
FileSize
13.85 KB

Hi Carol and all,

Thank you for bring up and solve this issue. I run upgrade from drupal 6.19 to 6.22 by following the steps mentioned in upgrade.txt and using Carol's patch. When I was doing step 10 (runing update.php for cores before re-enable custom and contributed modules), this message shown:

The following queires were executed
color module
update #6001
no queires

When I run update.php again after re-enable custom and contributed modules, this message didn't show up.
Are these error messages? Should I do anything to fix it?
My site seems to run OK, but I am not sure whether the message above is serious or not. Please advice if you know the answer. Thanks a lot.

carlos8f’s picture

Category: support » bug
Status: Active » Needs review

@pearlliang, please don't hijack the issue status. Those are not errors, sounds totally normal to me.

pearlliang’s picture

Hi Carlo, Thanks for your response. I don't know what do you mean by hijack the issue status. Is it because I choose "Active" option in the Status filed? Sorry I am new to Drupal issue postings and I don't know the ruie. Should I choose "needs review" in the Status field? Thanks

BrankoC’s picture

@everybody: I am 'lucky' in that only a custom module I wrote seems to suffer from this problem. Would there be a way to write a module so that it is not affected by this bug?

@Pearlliang: the Issue Settings pertain the bug report, not your comments. If you do not wish to change the status of the entire report, don't touch the status field.

BrankoC’s picture

Forget I asked, we have decided to roll back to 6.21.

Flandry’s picture

I also have the problem with a custom block using aggregator moving itself back to None by itself. I browsed through the comments here but am somewhat confused. Can i expect a fix to be included in a future release?

Gábor Hojtsy’s picture

@Flandry: yes, it would be great to get some testing feedback on the patches proposed. So far the results are mixed/contradicting.

pearlliang’s picture

@brankoc: thanks for letting me know.
@everybody: the pathc at #23 works for my site. My blocks and views settings are both normal. I upgrated from 6.19 to 6.22. Thanks Carlo for the effort.

Stalski’s picture

I 'll try to fix it anyway for now. I never saw it that way, but it's correct. The user access check is only needed in the view.

- EDIT - That was for the heartbeat module. Fixed it tonight in git.

James Marks’s picture

I upgraded to 6.22 without temporarily disabling contributed modules (which I don't usually do for minor updates). After the upgrade, all 30 feed blocks created by the aggregator module (which is/was enabled) are routinely having their settings deleted and the blocks are disappearing off the pages. The aggregator blocks appear to be the only ones that are affected. All other blocks (including Views blocks) appear to be unaffected.

UPDATE: I applied the patch provided in #23 and reset the block configurations but by the next cron run all aggregator feed block settings were, again, deleted and the blocks disappeared from their pages. Patch from #23 didn't solve the problem for me.

UPDATE 1: Rolled back to 6.21, appears to have resolved the problem.

jelloleve’s picture

I have the same issue as James Marks.

1. installed drupal 6.22
2. Enabled Aggregator feed module and created several feeds
3. From Site Building->Blocks configure feeds to display in region "user1" ( user1 is a region in my template) , configure nodes to only show on the front page.
* everything is fine up to this point

4. On server set a crontab to run mysite.com/cron.php

problem: every time cron.php updates the feeds, the block region for feeds is reset to "none".

Eric_A’s picture

I've seen issues for Heartbeat (fixed) and Content Complete (nothing yet), but I'm not sure if there is one for Aggregator...

Well, we could temporarily switch to user 1 before invoking hook_block() in _block_rehash()... That would fix the breaking of contrib (and core) blocks that control access like this. I believe we already do user switching somewhere in core?

carlos8f’s picture

@Eric_A, temporarily escalating to user #1 sounds like a Very Bad Idea(tm). Last thing we want to do is create core security bugs for the sake of working around bugs in contrib.

Here's an updated patch. It looks big, but it's only doing two extra things that #23 didn't do:

1. Don't rehash in block_flush_caches() on an anonymous request. That should curtail blocks disappearing on cron due to failing user_access() checks in hook_block(). Unfortunately that means when doing update.php anonymously with $update_free_access = TRUE (or drush), block changes (cache mode) won't take effect like it should, but D6 users have been dealing with that for years :D

2. Fix poll and aggregator modules which wrap entire hook_block() in user_access(), ala #58. That is not necessary as I've stated earlier and only the 'view' $op needs user_access(). While surely there are contrib modules that make this mistake (and therefore fall victim to this issue), 1. above will ensure that we at least have a logged in user rehashing blocks, which has a better chance of passing misplaced user_access() calls in hook_block().

LetUsBePrecise’s picture

carlos8f, do we need to apply patch as per #23 as well before applying patch mentioned in #75.

sun’s picture

Status: Needs review » Needs work

The hook_block() fixes are correct. Contrib modules need to be fixed equally.

I don't think we can do that block_flush_caches() change, as it affects too many scenarios in which it is invoked.

Kinda strange that those hook_block() issues crop up in a stable release after 3+ years.

Deeksha14’s picture

@carlos8f
I have done the updates from 6.19 to 6.22
My issue is as an anonymous user I am not able to see certain blocks and menus and pages which were earlier visible . But after login in I can view them. Please help , if this is a ongoing issue should I revert back to the 6.19 version (I have all the backups in place) but it has security issues.
Let me know if there is some other way so that I can sort this out ASAP, as this is a production site.

carlos8f’s picture

Status: Needs work » Needs review
FileSize
9.66 KB

Removed the block_flush_caches() hunk as per @sun's recommendation. That means contrib modules will lose their blocks on cron if they wrap their hook_block() in a big user_access() check. I'm not sure what else I can do to prevent that other than a rollback of #235673: Changes to block caching mode not caught, which took 3 years to get in.

@Deeksha14, sounds like you might have an unrelated permissions issue. The issue we're dealing with here resets block configurations, so you actually have to re-assign them to regions, etc. to see them again (logging in will not make the blocks re-appear). If reverting to a backup fixes your issue, that might be the safest bet. If you can find a way to reproduce the issue with a clean install, that information would be appreciated so we can fix it generically.

Gabriel R.’s picture

Subscribed.

sjpatrick’s picture

Subscribe.

sarhansg’s picture

Version: 6.x-dev » 6.22

So has a proper fix for this been found? Do we rely on patch in #23 or #79?

nfriend’s picture

With the upgrade to 6.22 I too am having a similar issues as #78 - many of the references to my IMAGES (not menus) do not appear for Anonymous users -once logged in they all appear. These images are typically referenced in View Headers displayed in panels or Custom content in Panels.

Log Entry shows: page not found 07/18/2011 - 09:34 heylivemusic.com/sites/default/files/LiveTVHeader.png Anonymous

I Have to flush all caches once and sometimes twice and then they re-appear.

This happens after Cron runs though not every time - about once a day.

Sorry if this isn't exactly what this topic is about but there could be a tie in since it started with 6.22. Hope this clicks with issues mentioned previously here.

Thanks
-Neil

weekbeforenext’s picture

"Even so, it makes sense to avoid data loss in that scenario :)"

I just upgraded successfully in a development environment without disabling contributed modules. What sort of data loss could this cause? Do you recommend doing this in production or going with the patch?

carlos8f’s picture

I update my sites without disabling anything in production. That will not cause data loss in itself, and in fact I think it's preferred (for minor updates).

If you disable modules before updating to 6.22, you'll lose block configs for the disabled modules. If you really must update with modules disabled, you can apply #79 before running update.php.

This issue however also affects certain modules which have over-zealous access checks in hook_block() -- the chronic block disappearance issue. If you have blocks enabled from aggregator or poll modules, you should either skip the 6.22 update (no security patches in that) or apply patch #79 before running update.php. If you have blocks from heartbeat or content_complete modules, you unfortunately might have to wait for the maintainers of those modules to fix the hook_block() implementations.

allandk’s picture

subscribe

lifer’s picture

Thanks carlos8f (#85!) :)
At first I had the same problem with disappearing and misbehaving blocks on the front page when I updates from Drupal v6.19 to v6.22.
I then reverted to my last backup (huzzah for backups!) and updated to v6.21 with all my modules & themes active.
It worked like a charm!

swentel’s picture

I'm wondering if we could introduce an alther hook here ? Why ? Before 6.22 it was possible to update the cache setting of a block through *insert favorite sql editing tool here* or with a module, in this case blockcache_alter - which I maintain. With 6.22, this module has become useless (and ironically enough, I was the one that also tested out the patch that made this impossible). So, yeah, that patch is a regression in my opinion. An alter hook just before the return $blocks; is the fastest one to make that functionality work again. Thoughts ?

rjbrown99’s picture

IMO I think it will be a tough ride trying to get new alter hooks added to D6. I'm a user of blockcache_alter, stuck at 6.21 for the time being. That module already requires a core patch, why not just extend it to add your own alter in addition to the node grants stuff?

Macronomicus’s picture

subscribe

Tergenev’s picture

I have 6.22 running on three servers and am seeing this issue on a regular basis. Particularly, Aggregator feeds blocks are spontaneously going to the None region, and today I've just discovered that the Recent Polls block is doing the same thing. It seems to happen regularly, and after doing a great deal of looking at permissions, I was starting to suspect that the cron job was somehow disabling my blocks. Then I found this thread. I thought to apply patch 79 and see what happens, but I've never applied a patch, and the servers don't seem to have either patch or git installed. I'm a bit down the rabbit hole here on my drupal knowledge, I don't have a 6.21 backup to revert to, I went from 6.20 to 6.22. But I did want to report about the Recent Polls block also showing the behavior. Otherwise, I'm subscribed to see where this issue goes. . .

Alan D.’s picture

Subs. I just stumbled across this while see what the beat way was to update a block cache value. This issue is a slight worry for mass of D6 sites we have to maintain! :(

dogface’s picture

I have this problem. Tried the patch, aggregator blocks are disabled and settings wiped out after cron runs. Anything yet?
later edit: Tried bumping up memory to 132..no difference.

Wolfflow’s picture

subscribe

dogface’s picture

Solved for me!
Without the patch and adjusting memory settings, I made sure permission was set to allow anonymous user to access feeds. Now they are not wiped out when cron runs. Hope this helps someone.

GaëlG’s picture

Similar problem (some blocks disappearing on update.php) can occur when using Domain Access (at least for nice menus). The cause seems to be again an access problem, as Domain Access relies on the access system.

To be more precise, it can happen when the number of blocks provided by the module depends on this module's settings. Which is the case for Nice Menus (admin/settings/nice_menus). The workaround we found is to ensure that "All domains" is checked when saving these settings.

Gábor Hojtsy’s picture

Dear commenters: if you can actually try the patch and provide feedback on whether it solved your problem, (and you think its a good solution), that could help a great deal in getting this fixed.

Tergenev’s picture

OK, I'm running three sites at v6.22 and have seen the problem with disappearing aggregator and poll blocks on all of them. I previously mentioned that I was a bit uncertain about applying patches, but I finally bit the bullet and figured out how to do it. (Since I don't know anything . . .obviously . . . I decided to write this up for those who are like me and also don't know about patches...)

I'm running the sites on LAMP servers, 2 hosted offsite and 1 hosted here at the company. I have command line access to only the latter. So I wanted a solution that I could apply to all of them. I finally found this simple posting about applying patches using a Windows machine, which is what I work on in the office.

http://www.werockyourweb.com/patching-drupal-modules-in-windows

It got me started. First, I made backup copies of all three sites, Then I installed Cygwin (http://www.cygwin.com/) and made sure that the 'patchutil' package was installed along with the default setup. For those who don't know, Cygwin is an application that essentially creates a virtual Linux machine running on your Windows machine. I downloaded the patch file from #79 into the cygwin home/username directory. I opened the patch file and from the comments noted that three files would be patched. I downloaded those directories and files (modules/aggregator, modules/block, and modules/poll) from one of my live sites to the same location.

And then I called patch -p0 < 0001-Issue-1173012-by-carlos8f-Blocks-disappear-after-cor.patch. It didn't work. It took me a while to figure out that the -p flag is used to strip off a user-defined number of characters from the front of the path to the files to be patched.

In this patch file, the number needed to be 2, so I tried patch -p2 < 0001-Issue-1173012-by-carlos8f-Blocks-disappear-after-cor.patch It worked perfectly. I FTPed the three files that were modified straight into the modules/aggregator, modules/blocks, and modules/polls directories. Then I cleared the caches.

I've reenabled the 'recent poll' block and several aggregator news feed blocks, then called cron manually. The blocks didn't disappear. I've waited past the regularly scheduled cron job on both servers and . . . the patch worked. All of the blocks that I had the problems with before are now showing up as I wish them to. Thank you very much, Carlos8f!

carlos8f’s picture

@Tergenev thanks for the feedback, glad it worked!

pearcec’s picture

I am tracking what sounds like a similar issue here:

#1265630: Blocks disabled (disappear) after site clone.

@Deeksha14 Did you happen to remove any modules during your upgrade?

pearcec’s picture

Another clue, when we built our original platform we didn't specify where we wanted the modules to go. as a result they went into /profiles/xforty/modules. Experience tells us we want them in all/modules/{contrib,custom}. I am wondering if they move makes Drupal think the modules are no longer available during some point in the process when update.php is running?

sachbearbeiter’s picture

sub

ghede’s picture

My aggregator block kept disappearing. A search brought me here. Applying the patch in #79 using #98's directions has fixed it. Thank you guys very much!

puzl’s picture

Subscribing

webservant316’s picture

Any idea when this patch will be applied to core and Drupal 6.23 released? I've held off upgrading from 6.20 because of this problem.

carlos8f’s picture

We just need someone to review #79.

soulfroys’s picture

+1 for #79

kristin.e’s picture

#23 Patch worked for me

marcvangend’s picture

I have applied the patch from #79 - works fine as far as I can see.

izmeez’s picture

I now have a site with D6.22 exhibiting this problem consistently. I will target some time to test the patch and report back.

bardill’s picture

#85 Posted by carlos8f
I update my sites without disabling anything in production. That will not cause data loss in itself, and in fact I think it's preferred (for minor updates).

Hi and many thanks. So is it possible to update drupal core without disabling the modules? What are minor updates?
I always knew that this solution was very dangerous and always to be avoided. Do I remember well?
What problems you may have updating without disabling the modules? Many thanks.

B

carlos8f’s picture

Hi B,

Minor updates are designated by an increment of the second number in the version string. I.e., updating from 6.17 to 6.18. With Drupal core, this corresponds to bug fixes and/or security fixes, and occasionally performance improvements. I can't think of a logical reason why any modules would need to be disabled for these types of updates, and disabling modules has a long history of bugs in itself, see #1199946: Disabled modules are broken beyond repair so the "disable" functionality needs to be removed

Major version updates are for example 6.22 to 7.9. This is a complex scenario since (for example) cck has been superseded by core fields, but there is no official upgrade path, so your D6 modules conflict with "now in core" module names and stuff blows up. Thus the need to disable D6 modules during that process. UPGRADE.txt doesn't even specify whether it is talking about a major or minor update, and frankly the instructions there are ancient and in some parts questionable. This is because Drupal core developers themselves are confused on what the process should be, as evidenced by this issue: http://drupal.org/node/938560#comment-3560302

So in short, I recommend keeping all your modules enabled during minor updates, and disabling and/or removing D6 modules during a D7 upgrade.

bardill’s picture

Many thanks Carlos.
Really very interesting.

B

John Pitcairn’s picture

Just had this problem on an update of a multisite - moving it back to a local dev server to test was persistently disabling blocks supplied by Views and some custom modules after a cron run. I've run the updates directly on the live multisite (gulp), which luckily proceeded without problems.

I'll clone those updated DBs back to local again, see if the issue persists, try the patch in #79 and report back.

John Pitcairn’s picture

I applied the patch in #79 to Drupal 6.22. It doesn't appear to fix my problem - it actually makes things worse, also losing module-provided default views after a DB import followed by a cache flush. Previously I was only losing views-provided block region assignments. I've reverted to unpatched 6.22.

(remainder removed)

John Pitcairn’s picture

I've removed the procedure I posted above. Still not reliable here.

josephcheek’s picture

@JimCraner: we are seeing a similar issue after our 6.20->22 upgrade where certain fields in CCK nodes are missing. We lose an image here, a body there, etc. Based on what I've read in this thread, it doesn't appear to be related to the issues others have seen and @carlos8f has fixed (ie, missing blocks, modules being deactivated, and problems with poll and aggregator).

I never saw a response to your posting, but I thought I would let you know that you are not the only one.

John Pitcairn’s picture

I'm focusing my attention on two very simple blocks provided by a single custom module on one site. Nothing in the module is wrapped in a user_access() check, the only external calls the module makes are drupal_get_path() and drupal_add_js().

I am running unpatched Drupal 6.22 on both production and dev servers. Production is behaving itself, no blocks are disappearing on cache flush (either manually or via cron run).

Here's what happens:

1 - import the DB dump from production to dev.
2 - log in and manually flush all caches (or just the theme registry, or run cron, same result).
3 - The module-provided blocks are now gone from block admin, not just unassigned.
4 - Module admin shows the module as enabled.
5 - Saving the module config there restores the blocks to block admin, but they are now unassigned.

This behaviour does not appear to be theme-related, ie it occurs with the site's custom theme (based on Zen), Zen itself, or Garland. I guess at some point during the cache flush the module is becoming disabled, prior to block discovery running.

That would certainly explain the behaviour I'm seeing with modules on other migrated sites that provide default views, for which I am losing the enabled views and almost all blocks as a result.

I'm out of time right now, but will reread this thread for then perhaps take another look at patching 6.22 when I get a chance. If I can't resolve this soon I'll have to back off to 6.21.

John Pitcairn’s picture

Think I've got it. On unpatched 6.22, this migration procedure prevents a cache flush from disabling modules during block discovery:

1 - Import the production DB dump to development.
2 - Log in to development site.
3 - Immediately go to Site Building >> Modules and save.
4 - Now you can flush the caches.

Hope this helps somebody.

webservant316’s picture

Just checking in again on this issue.

Does anyone know when this problem will be repaired and Drupal 6.23 released? I am using Drupal 6.x and waiting for 6.23. I see that Drupal 7.x is already at 7.9, but 6.x seems at a standstill. Perhaps core development is over for 6.x? Is there an article that explains what can be expected for 6.x maintenance releases?

carlos8f’s picture

@webservant316, D6 is minimally maintained, most core developers have long ignored its issue queue, but whenever a wayward dev might venture to provide a technical peer review of #79, we'll have this fixed.

webservant316’s picture

hmm I see. so is it worth it to move to 6.22 and install patch #79 or just stay at 6.20 ??

Gabriel R.’s picture

@carlos8f: I can assure you there are quite few developers each with many sites that are holding back from upgrading because of this one bug. Having a release with such an outstanding bug, with end-user exposure and data-loss potential, is just a shame. Just saying.

carlos8f’s picture

@Gabriel I totally agree, I produced a patch within a day's notice, but unfortunately the issue depends on peer review which I can't provide for myself.

Gábor Hojtsy’s picture

FileSize
923 bytes

The aggregator_block() issue was independently reviewed and committed in #1283442: Aggregator block keeps disappearing... so we can focus on a more limited patch here. @sun supported the hook_block() changes and I agree they are very straightforward and needed, so I broke out and committed the poll_block() change from here to Drupal 6!

The remaining part of the patch that is under debate then could be this one I guess (attached).

Fabianx’s picture

Status: Needs review » Reviewed & tested by the community

Back to RTBC then for #125:

Tested behavior:

a) Module disable => Block vanishes from block admin page (which is good), but is kept in DB
b) Module re-enable => Block is directly where it was before
c) Block is not removed on cron runs neither enabled or disabled.

Contrib modules with permission problems need to be fixed, but that has nothing to do with this patch, which is not for restoring behavior before #235673: Changes to block caching mode not caught, but for fixing the case of not loosing block settings for disabled modules.

This patch is even an improvement (!) for 6.21, where blocks did loose there settings for disabled modules when visiting the block admin page.

The applied fix in #235673: Changes to block caching mode not caught just made this problem more prominent.

For blockcache_alter (an affected module by ) it can simply save the cache var into its own table, when overridden and implement an form alter on the block admin form (which comes after the _rehash()) and a hook_flush_caches.

Case closed. :-)

Possible follow up issues:

#735900: Deleting module's blocks when module is uninstalled - This one can be applied now as there is now a use-case.
#1194880: _block_rehash() overwrites the caching mode set by Block_Cache_Alter module - blockcache_alter should implement its own table and the hooks outlined above.

and (not created): If we really wanted to restore behavior that _block_rehash was never run as anonymous user, but always by a user with the privilege of 'administer blocks', then we could go ahead and add another patch in a followup issue to check the user has this permissions before running _block_rehash. But if we just say (like sun) that contrib needs to be fixed, then so be it.

And thats it.

Lets get this applied, a new drupal core release released and this issue over and done with.

connell83’s picture

When will this issue be fixed? I am running on 6.22 and this issue has basically put a complete halt to my site over the last several months. I cannot make any block amendments whatsoever and my site is suffering severely.

I think it's very dissapointing that this critical issue cannot be resolved.

All my blocks are showing as disabled even though they are not.

carlos8f’s picture

Assigned: carlos8f » Unassigned

@Fabianx, thanks for the review. Looks like a fair assessment.

The bottom line is, this is a trivial fix for a glaring issue. A quick look at the D6/critical/bug queue only shows 4 issues, some created years ago. The original patch #235673: Changes to block caching mode not caught took years to get in, let's not take years for a simple bugfix for it. It's a great way to drive away core developers and make it seem like writing patches is pointless.

http://drupal.org/project/issues/drupal?text=&status=14&priorities=1&cat...

sun’s picture

I agree this is RTBC.

Gábor Hojtsy’s picture

Status: Reviewed & tested by the community » Fixed

Great, thanks for the reviews, committed.

webservant316’s picture

Great and thanks! Now when is Drupal 6.23 to be released?

oriol_e9g’s picture

@webservant316 Tomorrow! :D

webservant316’s picture

Awesome! You guys are great, thanks!

bardill’s picture

Hi! 6.23, or better 6.24, solve this problem?
Many thanks to all drupal team! Great community!

Gábor Hojtsy’s picture

Drupal 6.23 only contains security fixes on top of Drupal 6.22. Drupal 6.24 includes bug fixes like this one. Your blocks will not get disabled anymore. The blocks that got disabled before (if any) you still need to manually enable again. (There is a nice figure on http://drupal.org/drupal-7.12 to help you understand what is in each release).

bardill’s picture

Sorry Gabor - the best way to upgrade is from 6.22 to 6.24.
Or from 6.22 to 6.23 and to 6.24?

Many thanks.
b

Gábor Hojtsy’s picture

Either should work.

webservant316’s picture

Awesome, Drupal 6.24 released!! Thanks guys!!!

btully’s picture

I'm having a hard time understanding how the patches listed here help preserve the block settings that are getting wiped. The patches listed above seem to wrap a check to see if the module exists before deleting the block. My issue (and how I interpret the OP) is not that blocks are getting DELETED, but that they are getting DISABLED (regions get set to -1) and that other settings like weight, visibility and pages get wiped. Do the patches (or 6.2.4) resolve this or am I confusing this for a separate issue?

carlos8f’s picture

deleting a block entry deletes its settings only. block content is not actually stored in {blocks}, only the region assignment and settings. So my patch prevents those settings from getting reset in various circumstances, namely when the module has been disabled or hook_block() implementation has permission checks. (correction: actually we just removed the permission checks for $op=list. this now reflects how the block api is intended to work)

btully’s picture

Thanks for the explanation, carlos8f. I've applied the patch and have crossed my fingers ;)

regards,
Brian

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.