migration from D6 fails for Content Taxonomy fields

subscribe - same issue here.

This is definitely strange, after much testing and trials I still can't get the data to populate the new Term Reference fields it creates. Also strange when I migrate a CT field that allows unlimited tax terms, it generates a "Taxonomy upgrade extras" Term Ref field, but when I migrate a CT that allows single terms, it does not generate the "Taxonomy upgrade extras" Term Ref field.

I just figured out a workaround bypassing CT Migrate altogether:

1. in the settings for each CT field in D6 there is a check box for "Save values additionally to the core taxonomy system (into the 'term_node' table)." - so I checked that for all my CT fields.

2. To retroactively update all my nodes I created a View listing them all and added in the Views Bulk Operations with the action of "Save post (node_save_action)". Then just use VBO to re-save all the nodes.

Now when I update to D7, I don't even need the CT migration because the associated terms are already converted into term reference fields. Since I don't plan on using CT in D7, I'm all set.

Found the issue. The CCK in D6 stored the term tid with a prefix of _value, but the module uses _tid

Patch attached. Hope it works for you.

Also, this migration forgets to fill field_data_{field_name} and field_revision_{field_name} tables.
I just added node_save() to it.

Please check my patch.

new patch, sorry about node_save() :)

New version again :)

wow !
this made my day, I was struggling with CT migration but with this patch it works perfectly.
I used it with a fixed size / multiple-values / multilanguage taxonomy, every terms got attached to the LANGUAGE_NONE but then this is about i18n_taxonomy to handle its upgrade path.

thanks a lot !

This patch worked for me as well

Here is a simpler approach that accomplishes the same thing. Any hope of this issue being committed soon?

please note that the first patch in #13 cause the following notice:
Undefined index: field_XXX_value content_taxonomy_migrate.module:48
will try #16

#16 didn't work for me
About #13, at the very least, the initialization of $tid and $field_name should happen after the if construct because the function is run for every fields, not only old content_taxonomy fields.

Moreover there are cases where the field_NAME_value does not exists while I can't say why.
Sample of such a failing $record :

array(6) {
  'entity_id' =>  string(3) "561"
  'revision_id' =>  string(3) "568"
  'delta' =>  string(1) "0"
  'entity_type' =>  string(4) "node"
  'language' =>  string(3) "und"
  'bundle' =>  string(12) "article"
}

while $field["field_name"] == "field_taxo";

The way such a record is initially loaded must be wrong, but I don't know where that could be.

How the hell would that work:
Here is a sample query as run by cck's content_migrate.admin.inc :
SELECT old_table.nid AS entity_id, old_table.vid AS revision_id, old_table.delta AS delta, 'node' AS entity_type, 'und' AS language, 'article' AS bundle FROM content_field_taxo old_table;
The field_taxo_value columns is NOT fetched !

The reason $record does not contains tid (so field_data_FIELDNAME is not populated):

For fields to be added to the request, content_migrate.admin.inc should go through foreach ($context['sandbox']['old_cols'] as $column_name => $db_column_name)
For $context['sandbox']['old_cols'] to be defined $old_cols = content_migrate_old_columns($field_value, $instance_value); should succeed (look at key/value)

Need n°1:
$field_value['columns'] = array('value' => 'tid'); inside content_taxonomy_migrate_content_migrate_field_alter()

But this is not enough !
addField() is also ignored if(!isset($context['sandbox']['new_cols'][$column_name])).
So content_migrate_new_columns() must succeed too and returns $context['sandbox']['new_cols']['value']) (as "value" is the old column name)

But unlike content_migrate_old_columns(), content_migrate_new_columns() does not allow manual tweak if empty($field['storage']['details']) what is the case.

All that stuff is a big undocumented crappy mess to make money from nothing (probably because adding uneeded complex API instead of documentation and samples is the only way to obfuscate GPL'ed code).
Do some git blame to look at the couple of people who old the key of taxonomy/cck upgrades of the thousands of D6 installations around the world.

Most hostages will end-up using insert into field_data_field_taxo (entity_id, field_action_tid) select nid as entity_id, field_taxo_value as field_action_tid from content_field_taxo; derivatives.
... good old days of home-made CMS.

All that stuff is a big undocumented crappy mess to make money from nothing

Wow, really? I get your frustration, but let's keep in mind that ALL of this code is totally free to you, and if you see something wrong with it, you're capable of fixing it for everyone. You've obviously traced the root of this problem, yet I don't see a patch. Doesn't that kind of make you one of the "couple of people who old the key of taxonomy/cck upgrades of the thousands of D6 installations around the world?" Would you care to do anything to fix that?

unlike content_migrate_old_columns(), content_migrate_new_columns() does not allow manual tweak if empty($field['storage']['details'])

I'm not willing to go deeper, I'm loosing my time. only cck maintainers (maybe) know what to do with this.

As always in the Drupal world, here's the kind of upgrade-path used by non-profit NGO:
$FIELD being the name of the field:
INSERT INTO field_data_field_$FIELD (entity_type, entity_id, revision_id, bundle, delta, language, field_action_tid) SELECT "node", c.nid, c.vid, n.type, delta, n.language, field_$FIELD_value FROM content_field_$FIELD c LEFT JOIN node n ON c.nid = n.nid;
(won't care about other entities, but you can't know if this module care about it either, that's the main difference between overly complex solutions and obvious ones).

INSERT INTO taxonomy_index SELECT n.nid, f.field_taxo_tid, n.created, n.sticky FROM field_data_field_taxo f LEFT JOIN node n ON f.entity_id = n.nid ;

That obviously does not deal with widget or any other stuff.
drush stuff attached

Patch in #16 worked for me.

Similar description in http://drupal.org/node/1371368#comment-5447982

Agree with @hernani: patch #16 get's me far enough to actually uninstall and remove the module without getting an error.

Yeah nevermind... Still getting the error...

Hi, Is the patch to get correct field committed please?
I do not think so but http://drupal.org/files/content_taxonomy-migration-fails-1208164-16.patch worked for me!
Thanks a lot in advance.
greetings, Martijn

#16 worked for me

#16 works for me too

#16 works great - it should be committed.

Thanks rbayliss!

#16 works very well - thanks so much for that
I'm wondering why this hasn't been committed. It renders many peoples upgrades fairly useless without it - can the maintainer please add this patch?

Also a note for others who are new to migrating as i was until quite recently, here's how you fix your installation with this patch, assuming you've already migrated and are facing the problem described here within your new drupal 7 installation.

- Apply the patch: in Terminal use 'patch < [location of patch]' (assuming you're in the folder with the patch and module file to be patched.
- Roll back your content taxonomy fields that have failed to pick up the older drupal 6 info. (here: admin/structure/content_migrate). They will all be taxonomy_term_reference fields now.
- Then re-migrate those fields
- Clear your cache

I got a little confused about this so for anyone looking for this it seems to be as follows:

the migration (without content taxonomy) does include a field into the contenttype containing the terms that were stored in term tables in drupal 6. this field is of type "entity reference" and does include an option to redundantly store the values into term tables too. so if your terms in content_taxonomy used this feature too you are fine if there is no other reason for using taxonomy term.

if your terms in content_taxonomy were ONLY stored in the field, you need to use content_taxonomy_migrate, need to patch as of #16. as a result you'll get a "term reference" field. I don't know why but couldn't find the option "Save values additionally to the core taxonomy system" on the D7 version any more which makes it useless for my usecase.

I had the same problem and the patch from #16 seems to solve it. Thanks! It would be nice if a maintainer committed it.

Back on this track....can the patch in #16 be committed so at least the content taxonomy migrate can work?
Greetings, Martijn

#12 works for me!

I think we should stop confuse people and leave only #13 patch, because it has been market as RTBC.

If you suggest another version of the patch - please change a status of the issue as well.

This tidies up patch #16 slightly - no logic changes, just formatting.

Wrapping in some other minor improvements to the migrate submodule.

Committed.