Upgrade from 6.20 to 7.2 - Update #7003 still fails

Did you tried disabling all (3rd party) modules ?

texis: Try the attached patch. This should resolve some (but maybe not all) of your issues.

Setting to CNW just because the testbot is testing over and over and we need to figure out why. OK to retest later.

The patch looks good to me.

I don't believe this needs to go into 8.x, retagging.

Summary: Completed patch review. The patch looks good. This is my first review, so please let me know if it needs any modifications or additions.

Pass 1: The "Coder module" pass

- Coding standards are being followed.

Pass 2: The "Bottom up" pass

- Code makes sense.

Pass 3: The "Comments-only" pass

- Comments passed spell check and adequately explain code additions.

Pass 4: The "Actually read the issue" pass

- Patch addresses PDOException present in error log provided by reporter.

Pass 5: The "Does it actually work?" pass

- Yes patch works:

patching file modules/node/node.install
Hunk #1 succeeded at 430 (offset -1 lines).
patching file modules/system/system.install
Hunk #1 succeeded at 1712 (offset -1 lines).

- No errors reported after applying patch and then completing upgrade from 6.20 to 7.x

The following updates returned messages
user module
Update #7000

    * User passwords rehashed to improve security

Update #7002

    * Migrated user time zones

Update #7014

    * Renamed the 'post comments without approval' permission to 'skip comment approval'.

system module
Update #7007

    * Inserted into {role_permission} the permissions for role ID 1, Inserted into {role_permission} the permissions for role ID 2

Pass 6: The "Put it all together" pass

- The patch looks good.

@rfay -- If we want to support a straight upgrade from 6.x to 8.x then this code does need to go into 8.x. Otherwise, I agree with you.

@pillarsdotnet, we always require major-to-major upgrades, so a 6 to 8 would require 6 to 7 and then 7 to 8.

This should probably be critical as its related to the upgrade path.

Hmmm....

system_update_7067() should not be calling user_roles(), instead there should be a _update_7000_user_roles() or similar function that does a direct query. It might still be necessary to add the dependency, but that should be fixed here as well.

#3: 1072450.patch queued for re-testing.

This still needs work per #16.

+  $dependencies['system'][7067] = array(
+    'user' => 7007,
+  );

We already have in 7.0:

  // Run all the critical user upgrades before everything.
  $dependencies['system'][7000] = array(
    'user' => 7008,
  );

What happened? Dependencies were not enforced or they were but system_update_7067() was allowed to run before system 7000()?

Tagging.

I was going to add _update_7000_user_roles(), but what about #20?

Re-titling so it's more specific, not sure about that dependencies issue.

It would be good to have tests here that reproduce the bug.

This could well be a broken upgrade path due to conflicting update dependency declarations.

Looking at taxonomy_update_dependencies() and user_update_dependencies() is enough to see such a conflict.

The first is trying to accomplish this ordering:

system_update_7049() ---> taxonomy_update_7002() ---> user_update_7006().

And this is from the second hook implementation:

  // Run all the critical user upgrades before everything.
  $dependencies['system'][7000] = array(
    'user' => 7008,
  );

To achieve all this one of the next two must happen:
system_update_7000() runs after system_update_7049()
user_update_7008() runs before user_update_7006()

The last type of scenario seems like one that an upgrade could survive sometimes. The first I don't think so.
What your dependency graph ends up like depends on which modules are installed I guess. (with D7 update dependencies.) Adding more dependencies would not fix the real problem but just hide it for certain cases.

What? A user update depending on a taxonomy update..?

  $dependencies['user'][7006] = array(
    'taxonomy' => 7002,
  );

@texis: But what about the patch in #3?

Even while it is not correct due to #16, but does it solve the issue? Can you upgrade with #3 applied?

Best Wishes,

Fabian

I just did my first D6 upgrade and patch at #3 worked for me.

subscribing

@texis - could you make a sanitized copy of your database or similar available?

Or even just a dump of the Drupal 6 system table may help.

@texis:

thanks for the db dump, that helps a bit.

Just restoring the filter dump failed for me before I even tried to upgrade, in the latest Drupal 6 there is a unique key 'fmd' on the filter table, this is not shown in your database dump.

See http://api.drupal.org/api/drupal/modules--filter--filter.install/functio...

So doing a clean install of D6 and then running the insert statement given above produced this error:

ERROR 1062 (23000): Duplicate entry '1-filter-1' for key 'fmd'

Which version of Drupal 6 are you trying to upgrade from? Should be in CHANGELOG.txt if nowhere else.

Also please post the results of this query:

MariaDB [d6]> SELECT schema_version FROM system WHERE name = 'system';
+----------------+
| schema_version |
+----------------+
|           6055 |
+----------------+
1 row in set (0.00 sec)

It looks like the fmd index was added in system_update_6043() - so to me it looks like you are running an older version of Drupal 6.

Alternatively you already had duplicates in your {filter} table before that update was run, in which case it may have failed to create the unique key.

A workaround for this would be ALTER IGNORE TABLE {filter} ADD UNIQUE KEY fmd (etc. etc.) on the D6 databse - however unless you're absolutely desperate to upgrade asap, please hold of doing this until we can figure out what's going on (or keep a full database backup of the database that can't be upgraded).

@texis sorry reading through your posts you were indeed upgrading from 6.22, so it's not that this update never ran.

Do you remember when you first installed this site (and/or if it started on Drupal 5)?

I think the overall issue at least with this failure, is that we added some unique indexes in Drupal 6, on tables that previously had no unique index, and that update didn't attempt to fix duplicates when adding the index.

By default MySQL will fail to create that index and leave the duplicates in, unless you create it with ALTER IGNORE TABLE which is MySQL-specific syntax.

I'm not sure what we could do about this in Drupal 7- except perhaps try to create the index again, having manually removed duplicates first (or add IGNORE support to the MySQL driver for schema changes).

It looks like there's two distinct issues. I've opened #1211796: Drupal 6 -> 7 upgrade can fail due to unique indexes added in Drupal 6 for the duplicate key error, that is going to be a separate issue to #3.

--

Also looking through the taxonomy dependency that is a bit weird as Eric_A points out.

The idea of those reverse dependencies is that taxonomy module can see "I really want my update to run before user_update_7006() - because that was added after my update was written" - so not a dependency as such, but trying to enforce the order that the updates were added to the system. However all this stuff was added way, way after the actual updates were written, and before we had upgrade tests, so the potential for circular dependencies is definitely there.

@texis - let's move this to #1211796: Drupal 6 -> 7 upgrade can fail due to unique indexes added in Drupal 6 - that is a proper core bug, but we should try to get the patch from #3 in here separately.

Applying #3 gives us the image below which still did not fix the reported item from #717834: The dependencies declared in core's hook_update_dependencies() implementations aren't actually correct and #211182: Updates run in unpredictable order ... see the second image.

Image generated with http://drupal.org/project/graphapi and quick patch (to be committed)

drush php-eval "print_r(graphapi_to_dot(_graphapi_devel_system_module_install_layout()));" | dot -Tpng -o../core-7-1072450-patched.png

XRef: #1217648: drupal_depth_first_search (graph.inc) allows for circular graphs

Marking this as duplicate of #717834: The dependencies declared in core's hook_update_dependencies() implementations aren't actually correct.

I'm trying to update from 6.28 and getting the same error:

Failed: PDOException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '0-php_code' for key 1: INSERT INTO {filter} (format, module, name, weight, settings, status) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5); Array ( [:db_insert_placeholder_0] => 0 [:db_insert_placeholder_1] => php [:db_insert_placeholder_2] => php_code [:db_insert_placeholder_3] => 2 [:db_insert_placeholder_4] => a:0:{} [:db_insert_placeholder_5] => 1 ) in filter_update_7003() (line 334 of /Volumes/Data/Users/david/Sites/fpo/modules/filter/filter.install).

I've tried the patch and I get this as output:

patching file modules/node/node.install
Hunk #1 FAILED at 431.
1 out of 1 hunk FAILED -- saving rejects to file modules/node/node.install.rej
patching file modules/system/system.install
Hunk #1 succeeded at 1706 with fuzz 2 (offset -7 lines).

Any help greatly appreciated.

Thanks,
David

@tknospdr as this issue was marked as a duplicate you should not reopen it.x

Best think you can do is searching for _open_ issues and if not found try on IRC or file a new issue with cross reference to both this and #1211796: Drupal 6 -> 7 upgrade can fail due to unique indexes added in Drupal 6

I'm having this bugs upgrading from D6 to D7.

Finally i found that the problem is some server unable to change table structure in {users} for "password" from 60 character to 128 characters.

So, I changed it manually in phpmyadmin and solved this problem.