I need some help with my first migration using this module. I read bear.inc but I didn't understand some things. Below (in bold) I have some questions to clarify some things. I would like to document this migration to help new users to Migrate module. I would really appreciate the help of an advanced user.
Here are my tables:
- test_category (taxonomy)
- test_subcategory (taxonomy)
- test_users (user)
- test_questions (node)
I attached the structure of each table in png files (also sql files in the zip archive).
1. Migration of test_category
class TestParentTermMigration extends TestMigration {
public function __construct() {
parent::__construct();
$this->description = t('Migrate parent categories from the source database to taxonomy terms');
$this->map = new MigrateSQLMap($this->machineName,
array(
'id' => array('type' => 'int',
'length' => 10,
'not null' => TRUE,
'description' => 'Category ID',
)
),
MigrateDestinationTerm::getKeySchema()
);
$query = db_select('test_category', 'c')->fields('c', array('id', 'title'));
$this->source = new MigrateSourceSQL($query);
// Set up our destination - terms in the category vocabulary
$this->destination = new MigrateDestinationTerm('category');
// Assign mappings TO destination fields FROM source fields. To discover
$this->addFieldMapping('name', 'title');
// Unmapped fields
$this->addUnmigratedDestinations(array('path', 'description', 'format', 'weight', 'parent', 'parent_name'));
}
}
No problems so far, just a simple migration of two fields.
2. Migration of test_subcategory
class TestChildTermMigration extends TestMigration {
public function __construct() {
parent::__construct();
$this->description = t('Migrate child categories from the source database to taxonomy terms');
$this->dependencies = array('TestParentTerm');
$this->map = new MigrateSQLMap($this->machineName,
array(
'id' => array('type' => 'int',
'length' => 10,
'not null' => TRUE,
'description' => 'Subcategory ID',
)
),
MigrateDestinationTerm::getKeySchema()
);
$query = db_select('test_subcategory', 's')->fields('s', array('id', 'title', 'category_id', 'description'));
$this->source = new MigrateSourceSQL($query);
// Set up our destination - terms in the category vocabulary
$this->destination = new MigrateDestinationTerm('category');
// Assign mappings TO destination fields FROM source fields. To discover
$this->addFieldMapping('name', 'title');
$this->addFieldMapping('description', 'description');
// Translate the old ID to Drupal identifier
$this->addFieldMapping('parent', 'category_id')
->sourceMigration('TestParentTerm')
->defaultValue(0);
// Unmapped fields
$this->addUnmigratedDestinations(array('path', 'format', 'weight', 'parent_name'));
}
}
Question: When importing the child terms, I have to map them to the ID of the imported taxonomy term. Is this the correct way, using sourceMigration('TestParentTerm')?
3. Migration of test_users
class TestUserMigration extends TestMigration {
public function __construct() {
// The basic setup is similar to BeerTermMigraiton
parent::__construct();
$this->description = t('Users from test migration');
$this->map = new MigrateSQLMap($this->machineName,
array('id' => array(
'type' => 'int',
'not null' => TRUE,
'description' => 'User ID'
)
),
MigrateDestinationUser::getKeySchema()
);
$query = db_select('test_users', 'u')
->fields('u', array('id', 'username', 'password', 'email', 'created_at'));
$this->source = new MigrateSourceSQL($query);
$this->destination = new MigrateDestinationUser();
$this->addFieldMapping('mail', 'email');
$this->addFieldMapping('init', 'email');
// Dedupe assures that value is unique. Use it when source data is non-unique.
// Pass the Drupal table and column for determining uniqueness.
$this->addFieldMapping('name', 'username')
->dedupe('users', 'name');
// The migrate module automatically converts date/time strings to UNIX timestamps.
$this->addFieldMapping('created', 'created_at');
$this->addFieldMapping('pass', 'password');
$this->addFieldMapping('roles')
->defaultValue(2);
$this->addFieldMapping('status')
->defaultValue(1);
// Unmapped destination fields
$this->addUnmigratedDestinations(array('theme', 'signature', 'signature_format', 'access', 'login',
'timezone', 'language', 'picture', 'is_new', 'path'));
}
}
First problem: Some users do not have email address. I know Drupal needs it, so I would like to generate on-the-fly something like current_username@example.com. How can I prepare each row using Migrate?
4. Migration of test_questions
class TestNodeMigration extends TestMigration {
public function __construct() {
parent::__construct();
$this->description = t('User questions');
// You may optionally declare dependencies for your migration - other migrations
// which should run first. In this case, terms assigned to our nodes and
// the authors of the nodes should be migrated before the nodes themselves.
$this->dependencies = array('TestUser', 'TestChildTerm');
$this->map = new MigrateSQLMap($this->machineName,
array(
'id' => array(
'type' => 'int',
'not null' => TRUE,
'description' => 'Question ID'
)
),
MigrateDestinationNode::getKeySchema()
);
$query = db_select('test_questions', 'q')
->fields('q', array('id', 'content', 'user_id', 'subcategory_id', 'status', 'created_at'));
$this->source = new MigrateSourceSQL($query);
// Set up our destination - nodes of type question
$this->destination = new MigrateDestinationNode('question');
// Mapped fields
$this->addFieldMapping('field_article_state', 'status');
$this->addFieldMapping('status')->defaultValue(1);
$this->addFieldMapping('comment')->defaultValue(2);
$this->addFieldMapping('created', 'created_at');
// Data too long for column 'title'
/*$this->addFieldMapping('title', 'content')
->description(t('Mapping question content in source to node title'));*/
$this->addFieldMapping('title', NULL)->defaultValue('Question title');
$this->addFieldMapping('sticky')->defaultValue(0);
$this->addFieldMapping('is_new')->defaultValue(TRUE);
$this->addFieldMapping('body', 'content');
// Translate old user_id to new Drupal UID
$this->addFieldMapping('uid', 'user_id')
->sourceMigration('TestUser');
// Map to subcategory
$this->addFieldMapping('field_category', 'subcategory_id')
->sourceMigration('TestChildTerm');
// Unmapped destination fields
$this->addUnmigratedDestinations(array('changed','promote', 'revision', 'language', 'path', 'revision_uid', 'log', 'tnid'));
}
}
Second problem: I don't have a title field for the question to import into the title field of the node. I know the title is varchar(255) in node table, so I can't dump the content of the question into the title field (Migrate throws errors like: Data is too long). I have to prepare that title with substr() function, so how can I create a function for that?
The real problem: Although the relation to each user is correctly maintain, I can't assign the correct child term. I'm using the same sourceMigration which should take the old ID of the subcategory, look in the map and get the new taxonomy term to attach it to the node: $this->addFieldMapping('field_category','subcategory_id')->sourceMigration('TestChildTerm');
The structure of the tables is quite different from beer.inc migrate_example, but this shouldn't arise any problems for some experienced Migrate user. I'm new so I really need some answers to continue.
Any help is appreciated. Thanks
Comment | File | Size | Author |
---|---|---|---|
#1 | tables.png | 26.63 KB | sergiu.popa |
sql-files.zip | 4.76 KB | sergiu.popa | |
test_questions.png | 31.56 KB | sergiu.popa | |
test_users.png | 15.51 KB | sergiu.popa | |
test_subcategory.png | 41.17 KB | sergiu.popa |
Comments
Comment #1
sergiu.popa CreditAttribution: sergiu.popa commentedLong story short: These are the tables I want to import:
I have problems when migrating the last table, test_questions. Although the users are assigned correctly to the created nodes, the taxonomy terms are not.
Comment #2
tenken CreditAttribution: tenken commentedYou can define in each Migration class you have a prepareRow($row) function that preps the $row->title field to whatever you need, a substr() or whatever you want. Same thing with the emails for Users -- make a prepareRow($row) function for that migration and see if $row->email exists in the given row, if not you make your foo@example email address ...
hope that helps. Beyond that it all looks good so far.
EDIT: you might find this post (and search his blog for others) helpful -- http://btmash.com/article/2011-03-25/migrating-content-part-2-nodes
Comment #3
Ashok Negi CreditAttribution: Ashok Negi commentedHi,
As posted by Tenken,You can use prepareRow($row) function to assign email id's to those who dont
have one.Do something like this:
public function prepareRow($row) {
if($row->email == NULL) {
$row->filelist = $row->username;
}
else {
$row->filelist = $row->email;
}
}
Now in the mapping where you are mapping from source to destination email field,use
$this->addFieldMapping('mail', 'filelist');
Actually prepareRow($row) function is used to alter your source(whether a Database or a CSV file or anything else) before actual mapping takes place.
Hope this works for you!
Will work on the rest of the issues!
Comment #4
Ashok Negi CreditAttribution: Ashok Negi commentedRegarding Your second problem,it can be solved using same prepareRow($row) function.Try this in the file for Migration of test_questions :
public function prepareRow($row) {
$row->filelist = substr($row->content,1,10);
}
Again change the mapping
$this->addFieldMapping('title', NULL)->defaultValue('Question title');
INTO
$this->addFieldMapping('title','filelist');
As simple as that.
This is working for me.Hope works for you too.
Comment #5
brahmjeet789 CreditAttribution: brahmjeet789 commentedHey!!
Need to know one thing regarding the real problem .Is the child term not getting picked up or is it picking up wrong child term?
Comment #6
sergiu.popa CreditAttribution: sergiu.popa commentedIt's not getting picked. I guess something more should be done in prepareRow().
I don't have the time to investigate, I decided to import the content using my own script (bootstrapping and the adding manually with node_save(), etc. )
This should be a very simple migration, I guess I didn't get right the migrate module.
Comment #7
mikeryanSorry for the slow response. While usually the problem with responding to issues in the queue is too little information provided, the problem here was too much - I didn't have enough time in one sitting to look through everything you posted. Now that I have some time:
This looks right to me - did you have trouble with the term hierachy itself getting imported?
As previously suggested, you can use prepareRow() here. You can also use a callback:
Again, the answer is a callback:
Your mapping is
The problem is, by default the value for term references is assumed to be the term name, not the term ID. To get it to interpret the ID, do
Is this helpful?
Comment #8
sergiu.popa CreditAttribution: sergiu.popa commentedThank you very much Mike, everything it's solved know.
The solution was this little line of code:
I think it should be added into the documentation, with the specification that by default the value for term references is the term name.
I successfully imported the comments, now I'll import the real deal, a counseling system to Drupal 7.
Thanks again
Comment #10
grasmash CreditAttribution: grasmash commented@sergiu.popa Thanks for putting that last part in bold! I definitely would have missed it.