We want to move our drupal 5 site to our drupal 7 build. When I migrate 100 nodes with their taxonomy (4 vocabs) , 20 fail giving me this error:

PDOException: SQLSTATE[HY000]: General error: 1366 Incorrect integer value: '' for column 'tid' at row 1: INSERT INTO {taxonomy_index} (nid, tid, sticky, created) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3); Array ( [:db_insert_placeholder_0] => 8809 [:db_insert_placeholder_1] => [:db_insert_placeholder_2] => 0 [:db_insert_placeholder_3] => 1163099397 ) in taxonomy_field_insert() (line 1728 of /var/www/eci-d7/modules/taxonomy/taxonomy.module).

I use this query while migrating the nodes:

    $query = db_select($source_db_name .'.node', 'n')
      ->fields('n', array('nid', 'vid', 'type', 'title', 'uid', 'status', 'created', 'changed', 'comment', 'promote', 'moderate', 'sticky'))
      ->condition('n.type', 'article', '=');
      
    $query->join($source_db_name .'.node_revisions', 'nr', 'n.vid = nr.vid');
    $query->addField('nr', 'body');
    $query->addField('nr', 'teaser');
    
    $query->join($source_db_name .'.users', 'u', 'n.uid = u.uid');
    $query->addField('u', 'name');
    $query->orderBy('n.changed');
    
    $query->leftJoin($source_db_name .'.term_node', 'tn', 'n.nid=tn.nid');
    $query->leftJoin($source_db_name .'.term_data', 'td', 'tn.tid=td.tid');
    $query->addExpression('GROUP_CONCAT(DISTINCT tn.tid)', 'tag_list');

    $query->groupBy('n.nid');

I'm mapping the fields like this:

$this->addFieldMapping('field_tags', 'tag_list')
     ->separator(',')
     ->sourceMigration('EciLeTaggingTerms')
     ->arguments(array('source_type' => 'tid'));

    $this->addFieldMapping('field_tags_country', 'tag_list')
     ->separator(',')
     ->sourceMigration('EciLeCountryTerms')
     ->arguments(array('source_type' => 'tid'));
    
    $this->addFieldMapping('field_tags_blog', 'tag_list')
     ->separator(',')
     ->sourceMigration('EciLeBlogTerms')
     ->arguments(array('source_type' => 'tid'));
     
    $this->addFieldMapping('field_tags_encyclopaediae', 'tag_list')
     ->separator(',')
     ->sourceMigration('EciLeEncyclopaediaeTerms')
     ->arguments(array('source_type' => 'tid'));

And do the following in the prepareRow function:

     $tags = explode(',',$current_row->tag_list);
  	
     $query_source = db_select($source_db_name .'.term_data', 'td')
      ->fields('td', array('vid','tid'))
      ->condition('td.tid', $tags,'IN')
      ->condition('td.vid', '12', '=');
      
    $query_source->addExpression('GROUP_CONCAT(DISTINCT td.tid)', 'tag_list');
    $query_source = $query_source->execute(); 
    $newVoc = $query_source->fetchObject();
    $current_row->field_tags = $newVoc->tag_list;
 
    $query_source = db_select($source_db_name .'.term_data', 'td')
      ->fields('td', array('vid','tid'))
      ->condition('td.tid', $tags,'IN')
      ->condition('td.vid', '4', '=');
      
    $query_source->addExpression('GROUP_CONCAT(DISTINCT td.tid)', 'tag_list_country');
    $query_source = $query_source->execute(); 
    $newVoc = $query_source->fetchObject();
    $current_row->field_tags_country = $newVoc->tag_list_country;
    
    $query_source = db_select($source_db_name .'.term_data', 'td')
      ->fields('td', array('vid','tid'))
      ->condition('td.tid', $tags,'IN')
      ->condition('td.vid', '6', '=');
      
    $query_source->addExpression('GROUP_CONCAT(DISTINCT td.tid)', 'tag_list_blog');
    $query_source = $query_source->execute(); 
    $newVoc = $query_source->fetchObject();
    $current_row->field_tags_blog = $newVoc->tag_list_blog;
    
    $query_source = db_select($source_db_name .'.term_data', 'td')
      ->fields('td', array('vid','tid'))
      ->condition('td.tid', $tags,'IN')
      ->condition('td.vid', '10', '=');
      
    $query_source->addExpression('GROUP_CONCAT(DISTINCT td.tid)', 'tag_list_encyclopaediae');
    $query_source = $query_source->execute(); 
    $newVoc = $query_source->fetchObject();
    $current_row->field_tags_encyclopaediae = $newVoc->tag_list_encyclopaediae;

When i don't map the terms all the nodes get created.
I have no idea why with some terms I still get the error, and others not.

Can someone help me? I'm out of ideas.

Comments

drewish’s picture

It sounds like you're having problems when there's no terms. Try modifying your prepareRow stuff to only set the field value if there are terms.

Also you might want to think about wrapping all that code up into a for loop. Define your term ids and field names in an array and iterate over them:

$things = array(
  'field_tags' => 12,
  'field_tags_country' => 4,
  'field_tags_blog' => 6,
  'field_tags_encyclopaediae' => 10,
);
foreach ($things as $field_name => $vid) {
  $query_source = db_select($source_db_name .'.term_data', 'td')
    ->fields('td', array('vid','tid'))
    ->condition('td.tid', $tags,'IN')
    ->condition('td.vid', $vid, '=');

  $query_source->addExpression('GROUP_CONCAT(DISTINCT td.tid)', 'tag_list');
  $query_source = $query_source->execute(); 
  $newVoc = $query_source->fetchObject();
  $current_row->$field_name = $newVoc->tag_list;
}

Then you can put the fix for this in one place and be good to go.

Birgit’s picture

I changed the code, thanks for the tip.
The problem does not occur when I've got no terms, it occurs on certain terms.
For example, when I do a dump of the $current_row i get:

["field_tags_encyclopaediae"]=>string(3) "215"

But than I get:

WD node: PDOException: SQLSTATE[HY000]: General error: 1366 Incorrect[error]
integer value: '' for column 'tid' at row 1: INSERT INTO
{taxonomy_index} (nid, tid, sticky, created) VALUES
(:db_insert_placeholder_0, :db_insert_placeholder_1,
:db_insert_placeholder_2, :db_insert_placeholder_3); Array
(
    [:db_insert_placeholder_0] => 9281
    [:db_insert_placeholder_1] => 
    [:db_insert_placeholder_2] => 0
    [:db_insert_placeholder_3] => 1172584533
)
 in taxonomy_field_insert() (line 1728 of
/var/www/eci-d7/modules/taxonomy/taxonomy.module).

I've checked and the taxonomy term exists in the database and is linked to the right voc, it has no parent.
Like that term, there are several other terms which cause the same problem.

Those certain terms are not rejected all the time, there are nodes that link those terms without any problem.

drewish’s picture

I'd try getting to it in the debugger or putting a debug_print_backtrace() in there and see what's actually going on. My gut feeling is that the row you're printing out isn't the problem.

g.i.joe’s picture

Category: bug » support

It seems to be a bug !!

I wrote a patch for /migrate/plugins/destinations/fields.inc
at MigrateTaxonomyTermReferenceFieldHandler->prepare().
because $values[0]=false it will generate a error by inserting a tid in the {taxonomy_index} table.

Index: ../../../modules/contrib/migrate/plugins/destinations/fields.inc
===================================================================
--- ../../../modules/contrib/migrate/plugins/destinations/fields.inc
+++ ../../../modules/contrib/migrate/plugins/destinations/fields.inc
@@ -190,6 +190,9 @@
     else {
       $arguments = array();
     }
+    if (empty($values[0])) {
+      $values = array();
+    }
     if (isset($arguments['source_type']) && $arguments['source_type'] == 'tid') {
       // Nothing to do. We have tids already.
       $tids = $values;

The function prepare() expects:

Array
(
    [arguments] => Array
        (
            [source_type] => tid
        )
)

instead of:

Array
(
    [0] => 
    [arguments] => Array
        (
            [source_type] => tid
        )
)

If you look at Migration->handleSourceMigration() in /migrate/includes/migration.inc.
You see that if $source_keys = 1 this function will return an boolean: false in stead of an empty array().

<?php
  ...
  protected function handleSourceMigration($source_migrations, $source_keys, $default = NULL, $migration = NULL) {

     ...

    // Return a single result if we had a single key
    if (count($source_keys) > 1) {
      return $results;
    }
    else {
      return reset($results);
    }
  }
  ...
?>

Because Migration->handleSourceMigration() (in migrate.inc) is used by other FieldMappings. I patched MigrateTaxonomyTermReferenceFieldHandler->prepare() (in field.inc)

g.i.joe’s picture

Category: support » bug

It seems to be a bug !!

mikeryan’s picture

Category: support » bug
Status: Active » Needs review

This is a dupe of #1284592: Taxonomy term hierarchy not working. Keeping this one open because it has more info.

@G.I.Joe - please submit patches as attachments, and set the status to "needs review". Thanks!

mikeryan’s picture

Status: Needs review » Fixed

Committed, thanks!

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.