I need some help with my first migration using this module. I read bear.inc but I didn't understand some things. Below (in bold) I have some questions to clarify some things. I would like to document this migration to help new users to Migrate module. I would really appreciate the help of an advanced user.

Here are my tables:

  1. test_category (taxonomy)
  2. test_subcategory (taxonomy)
  3. test_users (user)
  4. test_questions (node)

I attached the structure of each table in png files (also sql files in the zip archive).

1. Migration of test_category

class TestParentTermMigration extends TestMigration {
    public function __construct() {
        parent::__construct();
        $this->description = t('Migrate parent categories from the source database to taxonomy terms');

        $this->map = new MigrateSQLMap($this->machineName,
            array(
                'id' => array('type' => 'int',
                    'length' => 10,
                    'not null' => TRUE,
                    'description' => 'Category ID',
                )
            ),
            MigrateDestinationTerm::getKeySchema()
        );

        $query = db_select('test_category', 'c')->fields('c', array('id', 'title'));
        $this->source = new MigrateSourceSQL($query);

        // Set up our destination - terms in the category vocabulary
        $this->destination = new MigrateDestinationTerm('category');

        // Assign mappings TO destination fields FROM source fields. To discover
        $this->addFieldMapping('name', 'title');

        // Unmapped fields
        $this->addUnmigratedDestinations(array('path', 'description', 'format', 'weight', 'parent', 'parent_name'));
    }
}

No problems so far, just a simple migration of two fields.

2. Migration of test_subcategory

class TestChildTermMigration extends TestMigration {
    public function __construct() {
        parent::__construct();
        $this->description = t('Migrate child categories from the source database to taxonomy terms');
        $this->dependencies = array('TestParentTerm');

        $this->map = new MigrateSQLMap($this->machineName,
            array(
                'id' => array('type' => 'int',
                    'length' => 10,
                    'not null' => TRUE,
                    'description' => 'Subcategory ID',
                )
            ),
            MigrateDestinationTerm::getKeySchema()
        );

        $query = db_select('test_subcategory', 's')->fields('s', array('id', 'title', 'category_id', 'description'));
        $this->source = new MigrateSourceSQL($query);

        // Set up our destination - terms in the category vocabulary
        $this->destination = new MigrateDestinationTerm('category');

        // Assign mappings TO destination fields FROM source fields. To discover
        $this->addFieldMapping('name', 'title');
        $this->addFieldMapping('description', 'description');

        // Translate the old ID to Drupal identifier
        $this->addFieldMapping('parent', 'category_id')
            ->sourceMigration('TestParentTerm')
            ->defaultValue(0);

        // Unmapped fields
        $this->addUnmigratedDestinations(array('path', 'format', 'weight', 'parent_name'));
    }
}

Question: When importing the child terms, I have to map them to the ID of the imported taxonomy term. Is this the correct way, using sourceMigration('TestParentTerm')?

3. Migration of test_users

class TestUserMigration extends TestMigration {
    public function __construct() {
        // The basic setup is similar to BeerTermMigraiton
        parent::__construct();
        $this->description = t('Users from test migration');
        $this->map = new MigrateSQLMap($this->machineName,
            array('id' => array(
                'type' => 'int',
                'not null' => TRUE,
                'description' => 'User ID'
            )
            ),
            MigrateDestinationUser::getKeySchema()
        );
        $query = db_select('test_users', 'u')
            ->fields('u', array('id', 'username', 'password', 'email', 'created_at'));
        $this->source = new MigrateSourceSQL($query);
        $this->destination = new MigrateDestinationUser();

        $this->addFieldMapping('mail', 'email');
        $this->addFieldMapping('init', 'email');
        // Dedupe assures that value is unique. Use it when source data is non-unique.
        // Pass the Drupal table and column for determining uniqueness.
        $this->addFieldMapping('name', 'username')
            ->dedupe('users', 'name');

        // The migrate module automatically converts date/time strings to UNIX timestamps.
        $this->addFieldMapping('created', 'created_at');
        $this->addFieldMapping('pass', 'password');
        $this->addFieldMapping('roles')
            ->defaultValue(2);
        $this->addFieldMapping('status')
            ->defaultValue(1);

        // Unmapped destination fields
        $this->addUnmigratedDestinations(array('theme', 'signature', 'signature_format', 'access', 'login',
            'timezone', 'language', 'picture', 'is_new', 'path'));
    }
}

First problem: Some users do not have email address. I know Drupal needs it, so I would like to generate on-the-fly something like current_username@example.com. How can I prepare each row using Migrate?

4. Migration of test_questions

class TestNodeMigration extends TestMigration {
    public function __construct() {
        parent::__construct();
        $this->description = t('User questions');

        // You may optionally declare dependencies for your migration - other migrations
        // which should run first. In this case, terms assigned to our nodes and
        // the authors of the nodes should be migrated before the nodes themselves.
        $this->dependencies = array('TestUser', 'TestChildTerm');

        $this->map = new MigrateSQLMap($this->machineName,
            array(
                'id' => array(
                    'type' => 'int',
                    'not null' => TRUE,
                    'description' => 'Question ID'
                )
            ),
            MigrateDestinationNode::getKeySchema()
        );

        $query = db_select('test_questions', 'q')
            ->fields('q', array('id', 'content', 'user_id', 'subcategory_id', 'status', 'created_at'));
        $this->source = new MigrateSourceSQL($query);

        // Set up our destination - nodes of type question
        $this->destination = new MigrateDestinationNode('question');

        // Mapped fields
        $this->addFieldMapping('field_article_state', 'status');
        $this->addFieldMapping('status')->defaultValue(1);
        $this->addFieldMapping('comment')->defaultValue(2);
        $this->addFieldMapping('created', 'created_at');
        // Data too long for column 'title'
        /*$this->addFieldMapping('title', 'content')
            ->description(t('Mapping question content in source to node title'));*/
        $this->addFieldMapping('title', NULL)->defaultValue('Question title');
        $this->addFieldMapping('sticky')->defaultValue(0);
        $this->addFieldMapping('is_new')->defaultValue(TRUE);
        $this->addFieldMapping('body', 'content');

        // Translate old user_id to new Drupal UID
        $this->addFieldMapping('uid', 'user_id')
            ->sourceMigration('TestUser');

        // Map to subcategory
        $this->addFieldMapping('field_category', 'subcategory_id')
            ->sourceMigration('TestChildTerm');

        // Unmapped destination fields
        $this->addUnmigratedDestinations(array('changed','promote', 'revision', 'language', 'path', 'revision_uid', 'log', 'tnid'));
    }
}

Second problem: I don't have a title field for the question to import into the title field of the node. I know the title is varchar(255) in node table, so I can't dump the content of the question into the title field (Migrate throws errors like: Data is too long). I have to prepare that title with substr() function, so how can I create a function for that?
The real problem: Although the relation to each user is correctly maintain, I can't assign the correct child term. I'm using the same sourceMigration which should take the old ID of the subcategory, look in the map and get the new taxonomy term to attach it to the node: $this->addFieldMapping('field_category','subcategory_id')->sourceMigration('TestChildTerm');

The structure of the tables is quite different from beer.inc migrate_example, but this shouldn't arise any problems for some experienced Migrate user. I'm new so I really need some answers to continue.

Any help is appreciated. Thanks

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

sergiu.popa’s picture

FileSize
26.63 KB

Long story short: These are the tables I want to import:

Tables structure

I have problems when migrating the last table, test_questions. Although the users are assigned correctly to the created nodes, the taxonomy terms are not.

tenken’s picture

You can define in each Migration class you have a prepareRow($row) function that preps the $row->title field to whatever you need, a substr() or whatever you want. Same thing with the emails for Users -- make a prepareRow($row) function for that migration and see if $row->email exists in the given row, if not you make your foo@example email address ...

hope that helps. Beyond that it all looks good so far.

EDIT: you might find this post (and search his blog for others) helpful -- http://btmash.com/article/2011-03-25/migrating-content-part-2-nodes

Ashok Negi’s picture

Hi,
As posted by Tenken,You can use prepareRow($row) function to assign email id's to those who dont
have one.Do something like this:
public function prepareRow($row) {
if($row->email == NULL) {
$row->filelist = $row->username;
}
else {
$row->filelist = $row->email;
}
}
Now in the mapping where you are mapping from source to destination email field,use
$this->addFieldMapping('mail', 'filelist');

Actually prepareRow($row) function is used to alter your source(whether a Database or a CSV file or anything else) before actual mapping takes place.
Hope this works for you!
Will work on the rest of the issues!

Ashok Negi’s picture

Regarding Your second problem,it can be solved using same prepareRow($row) function.Try this in the file for Migration of test_questions :

public function prepareRow($row) {
$row->filelist = substr($row->content,1,10);
}

Again change the mapping
$this->addFieldMapping('title', NULL)->defaultValue('Question title');
INTO
$this->addFieldMapping('title','filelist');

As simple as that.

This is working for me.Hope works for you too.

brahmjeet789’s picture

Hey!!
Need to know one thing regarding the real problem .Is the child term not getting picked up or is it picking up wrong child term?

sergiu.popa’s picture

It's not getting picked. I guess something more should be done in prepareRow().

I don't have the time to investigate, I decided to import the content using my own script (bootstrapping and the adding manually with node_save(), etc. )

This should be a very simple migration, I guess I didn't get right the migrate module.

mikeryan’s picture

Status: Active » Postponed (maintainer needs more info)

Sorry for the slow response. While usually the problem with responding to issues in the queue is too little information provided, the problem here was too much - I didn't have enough time in one sitting to look through everything you posted. Now that I have some time:

When importing the child terms, I have to map them to the ID of the imported taxonomy term. Is this the correct way, using sourceMigration('TestParentTerm')?

This looks right to me - did you have trouble with the term hierachy itself getting imported?

Some users do not have email address. I know Drupal needs it, so I would like to generate on-the-fly something like current_username@example.com. How can I prepare each row using Migrate?

As previously suggested, you can use prepareRow() here. You can also use a callback:

  $this->addFieldMapping('mail', 'email')
       ->callbacks(array($this, 'validateEmail'));
...
  protected function validateEmail($value) {
    if (!$value) {
      $value = // Generate @example.com email here
    }
    return $value;
  }
I don't have a title field for the question to import into the title field of the node. I know the title is varchar(255) in node table, so I can't dump the content of the question into the title field (Migrate throws errors like: Data is too long). I have to prepare that title with substr() function, so how can I create a function for that?

Again, the answer is a callback:

  $this->addFieldMapping('title', 'content')
        ->callbacks(array($this, 'truncateTitle'));
...
  protected function truncateTitle($value) {
    $value = substr($value, 0, 255);
    return $value;
  }
Although the relation to each user is correctly maintain, I can't assign the correct child term.

Your mapping is

  $this->addFieldMapping('field_category', 'subcategory_id')
       ->sourceMigration('TestChildTerm');

The problem is, by default the value for term references is assumed to be the term name, not the term ID. To get it to interpret the ID, do

  $this->addFieldMapping('field_category', 'subcategory_id')
       ->arguments(array('source_type' => 'tid'))
       ->sourceMigration('TestChildTerm');

Is this helpful?

sergiu.popa’s picture

Status: Postponed (maintainer needs more info) » Fixed

Thank you very much Mike, everything it's solved know.

The solution was this little line of code:

->arguments(array('source_type' => 'tid'))

I think it should be added into the documentation, with the specification that by default the value for term references is the term name.

I successfully imported the comments, now I'll import the real deal, a counseling system to Drupal 7.

Thanks again

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

grasmash’s picture

@sergiu.popa Thanks for putting that last part in bold! I definitely would have missed it.