This is my migration (creating a bunch of "papers" which live in "volumes"):

class VEPaperMigration extends Migration {
  public function __construct() {
    parent::__construct();
    $this->description = t('Migrate papers.');
    $this->map = new MigrateSQLMap($this->machineName,
        array(
          'id' => array('type' => 'integer',
                        'length' => 11,
                        'not null' => FALSE,
                        'description' => 'Paper ID',
           )
        ),
        MigrateDestinationNode::getKeySchema()
      );
    $query = db_select('virtualexplorer.papers', 'papers')
             ->fields('papers', array('id', 'title'))
             ->condition('is_deleted', 0)
             ->orderBy('volume_id', 'ASC')
             ->orderBy('weight', 'ASC');
    $this->source = new MigrateSourceSQL($query);
    $this->destination = new MigrateDestinationNode('paper');
    $this->addFieldMapping('title', 'title');
  }
}

class VEVolumeMigration extends Migration { 
  public function __construct() {
    parent::__construct();
    $this->description = t('Migrate volumes.');
    $this->dependencies = array('VEPaper', 'VEEditor');
    $this->map = new MigrateSQLMap($this->machineName,
      array(
        'id' => array('type' => 'int',
                      'length' => 11,
                      'not null' => FALSE,
                      'description' => 'Volume ID',
        ),
      ),
      MigrateDestinationNode::getKeySchema()
    );
    $query = db_select('virtualexplorer.volumes', 'volumes')
             ->fields('volumes', array('id', 'created', 'modified', 'published', 'is_published', 'is_doisubmitted', 'type', 'title', 'titleabbrev', 'number', 'year', 'description'));
    $this->source = new MigrateSourceSQL($query);
    $this->destination = new MigrateDestinationNode('volume');
    $this->addFieldMapping('field_papers', 'id')->sourceMigration('VEPaper');
  }
}

The migration works but field_papers, a multi-valued field, only gets referenced to a single paper instead of the multiple papers expected.

Am I doing the migration wrong or is this a feature limitation?

Comments

aidanlis’s picture

Issue summary: View changes

Clean up example code.

drewish’s picture

Status: Active » Postponed (maintainer needs more info)

How would a row in VEVolumeMigration have more than one value for 'id'? Seems like you've got something setup wrong. Are papers grouped into volumes? It looks like you're trying to look up the volume id in the papers import map... Provide a little more detail on what you're actually trying to accomplish.

aidanlis’s picture

Status: Postponed (maintainer needs more info) » Active

My papers table looks like: id, volume_id, title

When my volumes get imported I want the node:volume->field_papers field to have a node reference to each of the imported papers.

I see why this is a little tricky, the relationship has gone from a one-to-many on the paper to a many-to-one on the volume. Is there a neat way to handle this?

aidanlis’s picture

Status: Active » Fixed

I've come up with this which uses the existing mapping tables to create the relationships. It works as expected and seems like a reasonable way to proceed:

  public function prepareRow($row) {
    $row->paper_nids = $this->getPaperNids($row->id);
    $row->editor_nids = $this->getEditorNids($row->id);
  }

  protected function getPaperNids($volume_id) {
    $query = db_select('virtualexplorer.papers', 'p')
             ->fields('m', array('destid1'))
             ->condition('volume_id', $volume_id);
    $query->join('migrate_map_vepaper', 'm', 'p.id = m.sourceid1');
    $res = $query->execute();
    $nids = array();
    foreach($res as $row) {
      $nids[] = $row->destid1;
    }
    return $nids;
  }

  /**
   * This function links our editors and volumes together manualy.
   */
  public function getEditorNids($volume_id) {
    $query = db_select('virtualexplorer.users_volumes', 'uv')
             ->fields('map', array('destid1'))
             ->condition('volume_id', $volume_id);
    $query->join('virtualexplorer.users', 'u', 'uv.user_id = u.id');
    $query->join('migrate_map_veeditor', 'map', 'map.sourceid1 = u.editor_id');
    $res = $query->execute();
    $nids = array();
    foreach($res as $row) {
      $nids[] = $row->destid1;
    }
    return $nids;
  }

drewish’s picture

Yeah sucking them in in a prepare row is the right way to gather up that info.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

grasmash’s picture

Status: Postponed (maintainer needs more info) » Active

I need to accomplish the same thing that you did. I currently can pass a single nid into a node reference field (field_neighborhoods), but multiple values are not working.

I've tried your method, and I've also attempted to use GROUP_CONCAT to generate a comma-separated string.

Both of these approaches work to pull data from the source as expected, but passing either an array or comma-separated string to the field mapper results in an empty field.

Any ideas or suggestions would be greatly appreciated!

I've pasted some of my code below. It's been trimmed down to show the important parts, but there may be a few extra lines that seem extraneous.

Here's my GROUP_CONCAT attempt:


class yslNodeServiceProviderMigration extends Migration {
  public function __construct() {
    parent::__construct();

    $this->description = t('Migrate ysl service provider nodes.');
    $this->dependencies = array('yslUser', 'yslTermServices');

    $this->map = new MigrateSQLMap($this->machineName,
      array(
        'nid' => array(
          'type' => 'int',
          'unsigned' => TRUE,
          'not null' => TRUE,
          'description' => 'D6 Unique Node ID',
          'alias' => 'n',
        )
      ),
      MigrateDestinationNode::getKeySchema()
    );

    $query = db_select(YSL_MIGRATION_DATABASE_NAME . '.node', 'n')
      ->fields('n', array('nid', 'vid', 'type', 'language', 'title', 'uid', 'status', 'created', 'changed', 'comment', 'promote', 'moderate', 'sticky', 'tnid', 'translate'))
      ->condition('n.type', 'service_provider', '=');
    
    // Attempting to get values from multivalue node reference field.
    $query->leftJoin(YSL_MIGRATION_DATABASE_NAME . '.content_field_neighboorhoods', 'cfn', 'n.nid = cfn.nid AND n.vid = cfn.vid');
    $query->addExpression('GROUP_CONCAT(DISTINCT cfn.field_neighboorhoods_nid)', 'neighborhoods');
    $query->groupBy('n.nid');

    $source_fields = array(
      'nid' => t('The node ID of the page'),
      'services' => t('The service terms for the node'),
    );

    $this->source = new MigrateSourceSQL($query, $source_fields);
    $this->destination = new MigrateDestinationNode('service_provider');

    // Node reference w/ multiple values
    $this->addFieldMapping('field_neighborhoods', 'neighborhoods')->separator(',');
  }

  public function prepareRow($current_row) {
    
    dpm($current_row->neighborhoods); // SHOWS EXPECTED COMMA-SEPARATED VALUES
    return TRUE;
  }
}

Here's my prepareRow() attempt:


class yslNodeServiceProviderMigration extends Migration {
  public function __construct() {
    parent::__construct();

    $this->description = t('Migrate ysl service provider nodes.');
    $this->dependencies = array('yslUser', 'yslTermServices');

    $this->map = new MigrateSQLMap($this->machineName,
      array(
        'nid' => array(
          'type' => 'int',
          'unsigned' => TRUE,
          'not null' => TRUE,
          'description' => 'D6 Unique Node ID',
          'alias' => 'n',
        )
      ),
      MigrateDestinationNode::getKeySchema()
    );

    $query = db_select(YSL_MIGRATION_DATABASE_NAME . '.node', 'n')
      ->fields('n', array('nid', 'vid', 'type', 'language', 'title', 'uid', 'status', 'created', 'changed', 'comment', 'promote', 'moderate', 'sticky', 'tnid', 'translate'))
      ->condition('n.type', 'service_provider', '=');

    $source_fields = array(
      'nid' => t('The node ID of the page'),
      'services' => t('The service terms for the node'),
    );

    $this->source = new MigrateSourceSQL($query, $source_fields);
    $this->destination = new MigrateDestinationNode('service_provider');

    // Node reference w/ multiple values
    $this->addFieldMapping('field_neighborhoods', 'neighborhoods');

  }

  public function prepareRow($current_row) {

    // Process multivalue node reference field.
    $current_row->neighborhoods = ysl_migration_get_neighborhoods($current_row->nid);
    dpm($current_row->neighborhoods); // SHOWS EXPECTED ARRAY OF VALUES

    return TRUE;
  }
}

/**
 * Retrieve the neighborhood node references from the migration database.
 */

function ysl_migration_get_neighborhoods($nid) {
  $query = db_select(YSL_MIGRATION_DATABASE_NAME . '.content_field_neighboorhoods', 'cfn');
  $query->addField('cfn', 'field_neighboorhoods_nid');
  $query->condition('cfn.nid', $nid, '=');
  $query->orderBy('cfn.delta', 'ASC');
  $result = $query->execute()->fetchCol();

  return is_array($result) && !empty($result) ? $result : NULL;
}

Thanks!

mikeryan’s picture

Status: Closed (fixed) » Postponed (maintainer needs more info)

I don't have time to review the whole thing there, but I would suggest checking your spelling (neighboorhoods vs. neighborhoods).

grasmash’s picture

There was a misspelling in the old db.

Actually, I'm retrieving the data from the source without issue. The problem is that once I have the source data, I can't seem to provide it to Migrate in a format that allows multiple values to be passed.

I've edited my previous comment to cut out the slack.

:(

mikeryan’s picture

Status: Active » Postponed (maintainer needs more info)

Are the neighborhoods migrated in a previous migration process? The neighborhood nids you've set in $current_row->neighborhoods are nids from the source database, which need to be translated to their corresponding nids in the destination Drupal database:

  $this->addFieldMapping('field_neighborhoods', 'neighborhoods');
       ->sourceMigration('Neighborhood');
jeff veit’s picture

MikeRyan, I just want to say thanks: it's your comment at #9 that made me realise what was missing from my import, and why it wasn't working in linking referenced nodes.

Probably something for the documentation pages - since all the examples I found assumed a nid or tid mapping without sourceMigration.

grasmash’s picture

Status: Postponed (maintainer needs more info) » Closed (works as designed)

Had forgotten about this issue! I reworked the migration with your advice and things are now working. :)

grasmash’s picture

Issue summary: View changes

Ask a question