Block module index fails on MyISAM [#935250]

When upgrading from 7.0-alpha to 7.0-beta1, I'm getting lots of identical error messages when running update.php:

Notice: Undefined index: hash in _registry_update() (line 70 of /var/www/dev/includes/registry.inc).

And finally once:

PDOException: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'hash' in 'field list': UPDATE {registry_file} SET filename=:db_update_placeholder_0, hash=:db_update_placeholder_1 WHERE ( (filename = :db_condition_placeholder_0) ); Array ( [:db_update_placeholder_0] => modules/block/block.module [:db_update_placeholder_1] => 7753997ec424d99a42569242130fd849a75a38cb1da90438fc307e06fa68d8d4 [:db_condition_placeholder_0] => modules/block/block.module ) in _registry_parse_files() (line 150 of /var/www/dev/includes/registry.inc).

The summary reports: "The website encountered an unexpected error. Please try again later."

I can run update.php multiple times, it says every time: "20 pending updates"; those updates which won't run include:

system module
7053 - Upgrade standard blocks and menus. 
7054 - Remove {cache_}.headers columns. 
7055 - Converts fields that store serialized variables from text to blob. 
7057 - Increase the size of session-ids. 
7058 - Remove cron semaphore variable. 
7059 - Create the {file_usage} table. 
7060 - Create fields in preparation for migrating upload.module to file.module. 
7061 - Migrate upload.module data to the newly created file field. 
7062 - Replace 'system_list' index with 'bootstrap' index on {system}. 

block module
7002 - Rename {blocks} table to {block}, {blocks_roles} to {block_role} and {boxes} to {block_custom}. 
7003 - Change the weight column to normal int. 
7004 - Add new blocks to new regions, migrate custom variables to blocks. 
7005 - Update the {block_custom}.format column. 
7006 - Recreates cache_block table. Converts fields that hold serialized variables from text to blob. Removes 'headers' column. 

node module
7010 - Add the {block_node_type} table. 

user module
7010 - Update the {user}.signature_format column. 
7011 - Updates email templates to use new tokens. This function upgrades customized email templates from the old !token format to the new core tokens format. Additionally, in Drupal 7 we no longer e-mail plain text passwords to users, and there is no token for a plain text password in the new token system. Therefore, it also modifies any saved templates using the old '!password' token such that the token is removed, and displays a warning to users that they may need to go and modify the wording of their templates. 
7012 - Add the user's pictures to the {file_managed} table and make them managed files. 
7013 - Add user module file usage entries. 
7014 - Rename the 'post comments without approval' permission. In Drupal 7, this permission has been renamed to 'skip comment approval'.

When ignoring the pending updates and trying to access another area of the site, I'm getting:

rror message
Warning: Cannot modify header information - headers already sent by (output started at /var/www/dev/includes/common.inc:2525) in drupal_send_headers() (line 1048 of /var/www/dev/includes/bootstrap.inc).
PDOException: SQLSTATE[42S02]: Base table or view not found: 1146 Table 'dev_drupal.file_managed' doesn't exist: SELECT fid FROM {file_managed} WHERE status <> :permanent AND timestamp < :timestamp; Array ( [:permanent] => 1 [:timestamp] => 1286467991 ) in system_cron() (line 2891 of /var/www/dev/modules/system/system.module).

So basically updating from alpha to beta doesn't work at all for me. I don't know if this is a relevant error report since there doesn't necessarily need to be an upgrade path between development releases, so feel free to close this issue.

Greetings, -asb

Comments

Comment #1

int CreditAttribution: int commented 7 October 2010 at 22:22

Status:

Active

» Closed (won't fix)

we don't have upgrade path from alpha to beta. So to fix you will to ask help here: http://drupal.org/project/head2head

Comment #2

asb CreditAttribution: asb commented 8 October 2010 at 15:11

Thanks for the clarification, so let's drop the database ;)

Comment #3

wmostrey CreditAttribution: wmostrey commented 18 October 2010 at 23:34

Title:	Update fails - Undefined index: hash in _registry_update()	» Update fails on block module
Status:	Closed (won't fix)	» Active

I'm getting these errors when upgrading from 6.19 to 7.0-beta1.

7003 - Change the weight column to normal int.
7004 - Add new blocks to new regions, migrate custom variables to blocks.
7005 - Update the {block_custom}.format column.
7006 - Recreates cache_block table. Converts fields that hold serialized variables from text to blob. Removes 'headers' column.

Comment #4

wmostrey CreditAttribution: wmostrey commented 2 November 2010 at 10:47

Version:	7.0-beta1	» 7.0-beta2
Component:	database system	» block.module

The update is still failing for me on the block module:

block module
    7003 - Change the weight column to normal int.
    7004 - Add new blocks to new regions, migrate custom variables to blocks.
    7005 - Update the {block_custom}.format column.
    7006 - Recreates cache_block table. Converts fields that hold serialized variables from text to blob. Removes 'headers' column.
    7007 - Change {block_custom}.format into varchar.

More specifically:

Update #7003
    Failed: PDOException: %message in db_change_field() (line 2895 of /Applications/MAMP/mostrey/includes/database/database.inc).

Comment #5

splash112 CreditAttribution: splash112 commented 15 November 2010 at 06:14

Version:

7.0-beta2

» 7.0-beta3

Same problem here with 7053.
Failed: PDOException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'user-menu' for key 'PRIMARY': INSERT INTO {menu_custom} (menu_name, title, description) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2); Array ( [:db_insert_placeholder_0] => user-menu [:db_insert_placeholder_1] => User Menu [:db_insert_placeholder_2] => The User menu contains links related to the user's account, as well as the 'Log out' link. ) in system_update_7053() (line 2409 of /home/dinilu/public_html/drupal7/modules/system/system.install).

Went from 6.x to 7 beta 2 and now 3. Anybody any idea how to solve or work around it? I don't mind this small error, but a whole bunch of other upgrades can't go trough because of it...

Thanks!

Comment #6

wmostrey CreditAttribution: wmostrey commented 1 December 2010 at 21:07

Version:	7.0-beta3	» 7.0-rc1
Issue tags:		+D7 upgrade path

Still getting the same error when upgrading to RC1.

block module
    7003 - Change the weight column to normal int.
    7004 - Add new blocks to new regions, migrate custom variables to blocks.
    7005 - Update the {block_custom}.format column.
    7006 - Recreates cache_block table. Converts fields that hold serialized variables from text to blob. Removes 'headers' column.
    7007 - Change {block_custom}.format into varchar.

The effect of this failed update is that all blocks are no longer in a region (started with Garland and still using Garland):

    The block About was assigned to the invalid region left and has been disabled.
    The block Recent comments was assigned to the invalid region left and has been disabled.
    The block Navigation was assigned to the invalid region left and has been disabled.

Comment #7

wmostrey CreditAttribution: wmostrey commented 11 December 2010 at 23:07

Component:

block.module

» update.module

This is still happening, when upgrading from 6.19 to 7.0-rc2. I made sure to even remove all blocks before the upgrade, but still it fails:

block module
    7003 - Change the weight column to normal int.
    7004 - Add new blocks to new regions, migrate custom variables to blocks.
    7005 - Update the {block_custom}.format column.
    7006 - Recreates cache_block table. Converts fields that hold serialized variables from text to blob. Removes 'headers' column.
    7007 - Change {block_custom}.format into varchar.

Update #7003

    Failed: PDOException: SQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was too long; max key length is 1000 bytes: ALTER TABLE {block} CHANGE `weight` `weight` INT NOT NULL DEFAULT 0 COMMENT 'Block weight within region.', ADD INDEX `list` (`theme`, `status`, `region`, `weight`, `module`); Array ( ) in db_change_field() (line 2854 of /Applications/MAMP/mostrey/includes/database/database.inc).

Even after manually putting the blocks back, update.php returns these errors each time.

Comment #8

wmostrey CreditAttribution: wmostrey commented 11 December 2010 at 22:58

Version:	7.0-rc1	» 7.0-rc2
Priority:	Normal	» Major

Comment #9

catch

he/him

English

CreditAttribution: catch commented 28 December 2010 at 04:03

Title:	Update fails on block module	» Block module index fails on MyISAM
Version:	7.0-rc2	» 7.x-dev
Component:	update.module	» block.module
Priority:	Major	» Critical

The 1000 bytes fails on MyISAM, this explains why this hasn't been caught by the existing upgrade tests most likely. Since 99.9% of Drupal 6 sites are likely running MyISAM, I'm bumping this to critical.

This will probably need to use a partial index on the varchar 64 columns if we can get away with that.

Comment #10

chx CreditAttribution: chx commented 28 December 2010 at 21:00

Status:

Active

» Postponed (maintainer needs more info)

theme is 64, status is an int, region is 64, weight is an int, module is 64. That's way less than 1000 bytes (about 584 if my maths is not terribly off). The index limit is not engine specific and so Drupal would not even install and / or everyone's upgrade inc upgrade test would blow up. We need more information here like a table dump.

Comment #11

crimsondryad CreditAttribution: crimsondryad commented 3 January 2011 at 19:50

While this issue is a pain for the un-savvy (if it is indeed a MyISAM problem), there is a workaround. Migrating a DB from MyISAM to InnoDB is very simple. Do a flat text file dump, change the DB engine to InnoDB from MyISAM, and re-import the DB. That is the manual way to do it, there may be easier ways within other tools.

Then after the db engine is changed, do the normal upgrade.

There may be unintended consequences from switching to InnoDB this way, but we've done it several times on different sites with no issues so far. And realistically, getting people to migrate to InnoDB is probably a best practice...though I do understand about wanting to make using Drupal as painless as possible.

Just not sure if this qualifies as critical.

Comment #12

jbrown CreditAttribution: jbrown commented 3 January 2011 at 20:00

@chx UTF8 chars can be 3 bytes long so the max total index size is 333 chars IIRC

http://bugs.mysql.com/bug.php?id=4541

Comment #13

wmostrey CreditAttribution: wmostrey commented 17 January 2011 at 13:08

Version:

7.x-dev

» 7.0

The exact same error is still happening with Drupal 7.0. I'd be happy to mail the db dump to whoever is interested in looking at this. The dump is only 1.3MB, it's really not a complex site.

Comment #14

MustangGB CreditAttribution: MustangGB commented 15 February 2011 at 14:16

Version:

7.0

» 7.x-dev

Comment #15

chx CreditAttribution: chx commented 15 February 2011 at 14:28

wim, please mail me chx1975 gmail

Comment #16

wmostrey CreditAttribution: wmostrey commented 15 February 2011 at 15:23

The database dump has been sent to chx, thanks.

Comment #17

chx CreditAttribution: chx commented 15 February 2011 at 15:37

Working on it. wim's dump contains a theme(255) which surely triggers this.

Doing some archeology, Drupal 6 and 7 had a theme(64) but see http://drupalcode.org/viewvc/drupal/drupal/modules/system/system.install... for Drupal 5 which had 255. However, http://drupalcode.org/viewvc/drupal/drupal/modules/system/system.install... system_update_6043() changed to 64 and that was in Drupal 6.0. So how did you guys have 255 still?

re @jbrown, i think we two have a history where you are giving correct information probably with good intentions without reading and understanding what I wrote. In this case "theme is 64, status is an int, region is 64, weight is an int, module is 64. That's way less than 1000 bytes (about 584 if my maths is not terribly off)". How did you think I got to 584 if not by 64*3+4+64*3+4+64*3?

Comment #18

chx CreditAttribution: chx commented 15 February 2011 at 15:45

Status:

Postponed (maintainer needs more info)

» Closed (won't fix)

I am closing this issue. I have re-read it and there are at least three bug reports which have absolutely nothing to do with each other and then wim pretty much hijacked it (not that there was anything salvagable in it). Also, wim's issue is probably unique about a botched D5->D6 upgrade easily fixable by hand.

Comment #19

WiredEscape CreditAttribution: WiredEscape commented 11 January 2013 at 03:25

Doing some archeology, Drupal 6 and 7 had a theme(64) but see http://drupalcode.org/viewvc/drupal/drupal/modules/system/system.install... for Drupal 5 which had 255. However, http://drupalcode.org/viewvc/drupal/drupal/modules/system/system.install... system_update_6043() changed to 64 and that was in Drupal 6.0. So how did you guys have 255 still?

I'll confirm it is theme(255) in a D6 site causing update error. I just ran into a site which was updated from D5 long ago. Changing to theme(64) fixed.

Comment #20

prifre CreditAttribution: prifre commented 28 May 2013 at 15:34

Hi,

I tried to upGrade from 6 to 7 and ran into exactly this issue... (#6 Posted by wmostrey on December 1, 2010 at 10:07pm)
The problem is that when you guys write "Migrating a DB from MyISAM to InnoDB is very simple." it is definitely asubjective view.
I have upgraded from 5 to 6 fairly easily. But from 6 to 7 just failed on this and thereafter all my menuas are gone, and after having spent a long time debugging and installing/uninstalling, I have switched back an non-working 7 to an old Drupal 6 installation.

One really sad thing is that the Backup/Migrate module does not have an option "Backup data only, no structure" to be able to migrate data from an Drupal 6 to a fresh Drupal 7 installation.

As I understand that is somehow the solution the crimsondryad on January 3, 2011 at 8:50pm suggests - using some smart tools/solutions (only the savvy know :-( ).

With a sad smile
prifre

Comment #21

crimsondryad CreditAttribution: crimsondryad commented 20 June 2013 at 20:47

@prifre Did you export the db to a flat text file and switch it there then reimport the db? Also make sure all the cache tables ( and menu_router) are clear first. Menu_router is basically a cache table, it will get rebuilt if you delete it. Not sure if it would help in this scenario but if you aren't using Drush, I would definitely recommend it. Depending on the module sometimes Drush will let you get further along than using the UI.

We didn't have to do this but it may also help if you disable as many contrib modules as possible before the upgrade. Do core first and once that's working go through and turn the rest of your modules back on one at a time. That way if one particular module is barfing you can get further along with the troubleshooting.

May be a good idea to work out the migration with a local development copy before trying it in production so if it blows up there isn't the stress of having a site down for the interim.

I'm sorry you're having a hard time with this. I hope this will help. :)