Just commited the varialbes migration files to Google Analytics, but I'm asking me how I can change settings that older upgrade hooks have run. It's not that simple I guess and we cannot tell users they must install latest D6 or latest D7 version first before they are able to migrate.

How are we getting the last hook ID that ran on D6 and how are we able to run all missing D6 upgrade hooks + all missing D7 upgrade hooks to upgrade the settings data? These are really a lot of changes and I do not like to re-develop all upgrade hooks from past years! I guess this process asks for failures and data loss. There are so many functions inside the old upgrade hooks that are no longer available in D8 and this makes it extremly hard to migrate as we need to investigate all logics and functions again what has been developed years ago. Thats sooo crazy and a waste of time doing all this stuff again... I do not really like to make all my hard work again.

Any example code available?

Comments

hass’s picture

hass commented
Issue summary: View changes
cameron tod’s picture

cameron tod commented

Does this help at all? #2125745: [meta] Variables to config migration [D6]

hass’s picture

hass commented

Not at all. It does not explain how to run missing update hooks.

longwave’s picture

longwave
he/him
Primary language English
Location UK
commented

we cannot tell users they must install latest D6 or latest D7 version first before they are able to migrate

I would think this is a reasonable thing to ask? https://drupal.org/node/550152 mentions it, presumably this is talking about upgrading from 6 to 7, but 6/7 to 8 should follow the same guidelines:

It is important for your current site to be updated to the latest stable release of the current release prior to upgrade because point releases sometime introduce minor database schema changes. These changes will be expected to be in place by the target version of Drupal when you upgrade.

hass’s picture

hass commented
Category: Support request » Task
Priority: Normal » Major

We always had all D6 upgrade hooks inside D7 install files. This make sure users can upgrade. Otherwise it will become a horrible job to make structural changes in a previous version. Maybe we release a new D7 branch version post 8.x release and than the upgrade patch breaks as you expect d7 1.x but users have 2.x and you changed something.

This migration cannot be just a copy of variables. I could also made very complex db changes. I may changed variables in D8 to arrays where it was single values in D7. How should I upgrade? I do not like to write all code again. I cannot remember why I wrote some upgrade code in D6 4 years ago and more fun if there is a maintainer change. No no no... That will all not work.

How can I get the update hook number of the current D6/D7 I'm migrating from?

hass’s picture

hass commented

https://drupal.org/upgrade/migrate does not help here, too.

chx’s picture

chx commented
Category: Task » Support request
Priority: Major » Normal
chx’s picture

chx commented

> How are we getting the last hook ID that ran on D6

DrupalSqlBase::getModuleSchemaVersion -- if you are writing your own source you likely will extend DrupalSqlBase. If you aren't, well, that's not written yet.

> and how are we able to run all missing D6 upgrade hooks + all missing D7 upgrade hooks to upgrade the settings data

The same as you always ran previous versions on any Drupal: you don't. Your description is confusing:

We always had all D6 upgrade hooks inside D7 install files.

Do you mean upgrade from D6 to D7 or D5 to D6? The latter you most certainly didn't and the equivalent of D6 to D7 is now D7 to D8 and for that you have migrations. And as longwave points out, asking for latest version is customary. Also your source might want to extend checkRequirements.

> This migration cannot be just a copy of variables.

For sure, we have many that are not.

hass’s picture

hass commented

We cannot require the latest version of a D6 or D7 module is installed as we release D6 and D7 versions after D8 RTM release and this means the latest version is permanently changing.

People may be at level 6014 when they migrate, but a later release adds 6201. The D7 branch will also have a copy of 6014 and 6201 before 7000 runs. In D8 we must know if the user migrates from 6014 or 6201 or 7003 or 7005. All missing update hooks must run. We cannot say for sure that all migrations start at 6201 as example. This means we must clone all D6 and D7 upgrade hooks to the migration scripts of D8.

This means we must redo all heavy work we done in past and re-write and re-test something that already wasted hundred of working hours again. Do you really think that every module developer will do this??? I think we will all stand in rain as nobody like doing all the work again from scratch. Upgrading a D6 upgrade hook to run under D7 was a lot less complex and has not required a full rewrite with 100% previous core API removed.

I do not understand how above code may looks like.

chx’s picture

chx commented

> People may be at level 6014 when they migrate, but a later release adds 6201.

Well, then you need to release a 8.x version which migrates from that version. That's it. There's absolutely no reason to "clone all D6 and D7 upgrade hooks to the migration scripts of D8". Also, looking at http://drupalcode.org/project/google_analytics.git/blob/refs/heads/7.x-2... I am not quite sure where "hundreds of hours" of work come from but w/e.

benjy’s picture

benjy commented
Status: Active » Closed (fixed)

As previously mentioned, it's on the users of your module to make sure they're up-to-date before running the migration. Seems pretty reasonable to me.

benjy’s picture

benjy commented
Status: Closed (fixed) » Fixed
hass’s picture

hass commented
Status: Fixed » Active

It sounds like you do not understand me. It's technical not possible! What you are telling me require that I will never ever release a new D6 or D7 version after I released a D8 version. In this case I'm not allowed to make any changes on my d6/d7 modules. Forget it! How unworldly is this???

Looks like I identifies a heavy design flaw.

chx’s picture

chx commented
Status: Active » Closed (works as designed)

When you release 7.13 with a change that requires an update hook from 7.12 then you will need to release a new 8.x release as well say 8.4. Irregardless of the architecture if we are migrating data then you need some way to read and make a meaning out of your source and so if your source semantics change then obviously the source reader / processor needs to change too. If you can make 8.4 migrate from 7.12 that's great if not, then add a requirements check for the schema version set in 7.13 and that's it.

chx’s picture

chx commented

And yes that might mean that some of the logic in the update hook added to 7.13 gets repeated in the migration in 8.4. That can't be avoided within the update-minor, migrate-major framework we are currently operating in. Migrating for minor releases would require migrating in place which might not be possible. A code upgrade might cause Drupal to not run in a normal fashion; upgrade avoids this but migration is operating within a fully bootstrapped Drupal. So yes, the situation might not be ideal but please only reopen if you have concrete suggestions on how to improve. Mind you, these problems are completely independent of the implementation.

hass’s picture

hass commented
Status: Closed (works as designed) » Active

Can you share an example code snippet, please? I'd like to get an understanding how I can run all previous update hooks in same order in D8 and do the changes. Cannot find any examples.

chx’s picture

chx commented
Status: Active » Closed (works as designed)

Sigh. You can not run D7 update hooks in D8. That's not how it works. You need to write a migration. Again: I am aware this is not ideal but we can't avoid it.

hass’s picture

hass commented
Status: Closed (works as designed) » Active
Issue tags: +DX (Developer Experience)

It sound like we are not on the same paper.

I'd like to write a migration and the migration must work like the good old update hooks. This means the migration code will check the last updater hook version from D6 or D7 and than run all missing update code to get up to the latest D8 level. If the last update hook number is 6004 I need to run all update hook code from 6004 over 7xxx until 8000 to get the data migrated.

This I guess I will end with migration function names in D8 that are clones of D6 and D7 code, but nevermind. I need code examples how I can check the version of the previous core version and I have no idea how this code may look like. Please share an example how such a migration may looks like.

I strongly believe this question will pop up tons of times in very near future. If you like it or not. This is suxxx DX right now and I do not understand at all what you are trying to tell me with "this is not ideal" and finally I have no clue how the code around DrupalSqlBase::getModuleSchemaVersion looks like.

chx’s picture

chx commented
Status: Active » Closed (works as designed)

> I'd like to write a migration and the migration must work like the good old update hooks.

Won't happen.

> Then run all missing update code to get up to the latest D8 level.

That's not this works.

chx’s picture

chx commented

> This I guess I will end with migration function names in D8 that are clones of D6 and D7 code, but nevermind.

Understand this: the migration system is entirely, fundamentally, absolutely different from the upgrade system and has absolutely nothing to do with it. None. Migration reads from a database that is not the current database, processes the data and then calls the current Drupal to save the new data.

chx’s picture

chx commented 
function checkRequirements() {
  return $this->getModuleSchemaVersion('mymodule') >= 6010;
}

if that doesn't succeed then the migration doesn't run and your users need to upgrade the source system first to that version. Alternatively you can write code that interprets settings from 6001, 6002 etc if you want. You can not run upgrade code, that makes no sense.

chx’s picture

chx commented

Finally, I asked webmasters to lock this issue so you can't reopen it.

hass’s picture

hass commented

I open a new case of you lock valid questions. Unsubscribe yourself from the case if you like.

Understand this: the migration system is entirely, fundamentally, absolutely different from the upgrade system and has absolutely nothing to do with it. None. Migration reads from a database that is not the current database, processes the data and then calls the current Drupal to save the new data.

It looks like you do not understand the issues we are running into or you just ignore them. I do not care if you name the child migration or upgrade hooks. I need the last update hook that ran on D6 and D7 and than the conversion plugin will run all code from ALL previous update hooks. I have no idea what is wrong with this.

I have zero plans to release a D6 module version that has the same structure of variables like latest D7 version. This means the structure will dramatically change from D6 to D8, but not from D7 to D8. That's why I need to run upgrade code for D6, but not for D7. Compared to what you plan, I need to say - there is no upgrade possible from D6.