Last updated March 18, 2013. Created by EclipseGc on April 3, 2012.
Edited by mikeryan, cam8001. Log in to edit this page.
Migrating book module nodes is one of the more difficult tasks to accomplish with Migrate module. This is due largely to certain workflow issues that can come up during migrating book nodes, and the fact that book node hierarchy is stored within menu links. The following code is an example of how to deal with this.
Overview
- Migrate nodes as normal;
- Get hierarchy from the source database 'book' table, and weights from the 'menu_links' table;
- Use
complete()in your Migration class to call _book_update_outline() to migrate the hierarchy.
Explanation
Books are nodes, so they can inherit most of their migration from a common class. You can use something like the Drupal-to-Drupal data migration classes, or roll your own, like the one provided below.
Node migration
Let's kick this off with a good NodeMigration abstract class that should get us most of the way there. This class should be useful for other node types as well.
<?php
abstract class NodeMigration extends Migration {
public function __construct(array $arguments) {
parent::__construct();
$type = isset($arguments['type']) ? $arguments['type'] : NULL;
$new_type = isset($arguments['new type']) ? $arguments['new type'] : $type;
$this->description = t('Migrate nodes');
$this->map = new MigrateSQLMap($this->machineName,
array(
'nid' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
'description' => 'd6 Unique Node ID',
'alias' => 'n',
)
),
MigrateDestinationNode::getKeySchema()
);
$query = $this->node_query($type);
$this->set_highwater_field();
$this->set_source($query);
$this->destination = new MigrateDestinationNode($new_type);
$this->node_field_mapping();
}
public function set_highwater_field() {
$this->highwaterField = array(
'name' => 'changed',
'alias' => 'n',
);
}
public function node_query($type) {
$query = Database::getConnection('d6')
->select('node', 'n')
->fields('n', array('nid', 'vid', 'title', 'uid', 'status', 'created', 'changed', 'comment', 'promote', 'moderate', 'sticky'))
->condition('n.type', $type);
$query->join('node_revisions', 'nr', 'nr.vid = n.vid');
$query->fields('nr', array('body', 'teaser', 'format'));
return $query;
}
public function set_source($query) {
$this->source = new MigrateSourceSQL($query);
$this->source->setMapJoinable(FALSE);
}
public function node_field_mapping() {
$body_arguments = MigrateTextFieldHandler::arguments(array('source_field' => 'teaser'), array('source_field' => 'format'), NULL);
// Make the mappings
$this->addSimpleMappings(
array(
// Assumes you have done a migration of Users which maintains uids.
'uid',
'title',
'status',
'created',
'changed',
'comment',
'promote',
'moderate',
'sticky',
)
);
$this->addFieldMapping('body', 'body')->arguments($body_arguments);
$this->addFieldMapping('language')->defaultValue('en');
$this->addFieldMapping('is_new')->defaultValue(TRUE);
}
public function prepareRow($current_row) {
$formats = array(
'1' => 'filtered_html',
'2' => 'full_html',
'3' => 'plain_text',
'4' => 'markdown',
);
$current_row->format = isset($formats[$current_row->format]) ? $formats[$current_row->format] : 'plain_text';
}
}
?>There are some components of this you should change for your site, especially the $formats array in prepareRow(). In general this breaks a typical node migration into its various components of:
- What are we migrating? (mostly $this->map)
- How do we get it? ($this->node_query())
- How do we update it later? ($this->set_highwater_field())
- Set the source
- Set the Destination
- Do field mappings ($this->node_field_mapping())
This gives us enough granularity to write fairly minor migration classes from this point forward. As an example, migrating the default page nodes with this in hand looks something like:
<?php
class NodePageMigration extends NodeMigration {
public function __construct() {
parent::__construct(array('type' => 'page'));
}
}
?>And migrating Story nodes to Article nodes could look like:
<?php
class NodeArticleMigration extends NodeMigration {
public function __construct() {
parent::__construct(array('type' => 'story', 'new type' => 'article'));
}
}
?>Adding book hierarchies
So, with that foundation, let's discuss migrating book nodes.
The only difference between book nodes and other types of nodes is their table of contents hierarchy. This data is stored in its own table, handily called 'book'. There's also some information about relative weights for book items stored in menus.
There's three things we need to do to bring in the TOC data:
- Get the source data from the source database;
- Map source nids to destination nids;
- Update each migrated book with its position in its TOC.
<?php
class NodeBookMigration extends NodeMigration {
public function __construct() {
parent::__construct(array('type' => 'book'));
}
/**
* Overrides parent::nodeQuery to add more data to the source, in our case,
* book hierarchy stuff.
*/
public function nodeQuery() {
$query = parent::nodeQuery();
// Add in book parent child relationships.
$query->join('book', 'b', 'n.nid = b.nid');
$query->addField('b', 'bid', 'book_id');
$query->join('menu_links', 'ml', 'b.mlid = ml.mlid');
$query->addField('ml', 'weight', 'book_weight');
return $query;
}
/**
* Acts right after a book node has been saved. Map the book hierarchy.
*
* @param object $node
* A node object for the newly migrated book.
* @param stdClass $row
* An object representing data from the source row.
*/
public function complete($node, stdClass $row) {
// If this is a top-level book, don't set a parent.
$book_id = $row->nid == $row->book_id ? $node->nid : $this->lookupMigratedBook($row->book_id);
// Book id - effectively, parent nid.
$node->book['bid'] = $book_id;
$node->book['nid'] = $node->nid;
$node->book['weight'] = $row->book_weight;
_book_update_outline($node);
node_save($node);
}
/**
* Returns a mapping for a migrated book.
*
* @param int $source_book_nid
* Nid of book in source database to lookup.
*/
protected function lookupMigratedBook($source_book_nid) {
$dest_book_nid = parent::handleSourceMigration('NodeBook', $source_book_nid);
return $dest_book_nid;
}
}
?>So far, so good. If you check your migration at admin/content/migrate, you should be able to see the new fields in your source data.
Migrate module helps us out in two ways here.
- We use a complete() method which runs straight after an individual node has been migrated. This way, we can just call a book module function to add book-specific data.
- Migrate API keeps mappings of source nids to migrated nids. We can then manually call parent::handleSourceMigration() to lookup the mapping. Previously, I tried setting a sourceMigration on the book_parent_nid field mapping, but this creates unwanted stubs for top-level books. Doing a manual lookup ensures that nodes are only looked up (or created as stubs) when required.
I have used this approach to migrate several hundred book nodes with great success. I highly recommend using Drupal-to-Drupal data migration rather than rolling your own node migration classes, but every Migration is different, so do what you have to.
Comments
Great, but does not map hierarchy
Great code example, but I got all nodes to be a child of the 'root', while my original book had serveral parents-child relations. Therefor I came up with the following approch:
Extend the query to also get the old parent's nid. (You could do a lookup with the 'book' table, but it is in the link_path as well (in the form "node/[nid]").)
<?php// Original query
$query->join('book', 'b', 'n.nid = b.nid');
$query->addField('b', 'bid', 'book_id');
$query->join('menu_links', 'ml', 'b.mlid = ml.mlid');
$query->addField('ml', 'weight', 'book_weight');
//Query extention to get nid of parent
$query->leftJoin('menu_links', 'mm', 'ml.plid = mm.mlid');
$query->addField('mm', 'link_path', 'parent_id');
// Sort by depth so there is always a parent
$query->orderBy('ml.depth');
?>
Then in the complete function we add another query:
<?php
public function complete($node, stdClass $row) {
// If this is a top-level book, don't set a parent.
$book_id = $row->nid == $row->book_id ? $node->nid : parent::handleSourceMigration('NodeBook', $row->book_id);
// Book id - effectively, parent nid.
$node->book['bid'] = $book_id;
$node->book['nid'] = $node->nid;
$node->book['weight'] = $row->book_weight;
// We strip the "node/"-part from the link_path
$row->parent_id = str_replace("node/", "", $row->parent_id);
// If there is a parent we lookup the migrated node
if(isset($row->parent_id) && $row->parent_id > 0) {
$parent_node = parent::handleSourceMigration('NodeBook', $row->parent_id);
// We get the mlid of the migrated parent
$query = db_select("menu_links", "ml")
->fields("ml")
->condition("ml.link_path", "node/" . $parent_node, "=")
->execute()
->fetchAssoc();
$plid = $query['mlid'];
// We add the parent mlid (plid)
$node->book['plid'] = $plid;
}
_book_update_outline($node);
node_save($node);
}
?>
It probably isn't the best way to do this, but it works.
Using migrate_d2d
Thanks for the great article! For others wishing to use migrate_d2d, here is how we accomplished it, using the information in the article as our starting point.
Basically if you are using migrate_d2d, you can skip the entire section on "Node migration" since this is handled already by the DrupalNodeMigration class.
Rather than nodeQuery, however, DrupalNodeMigration uses a function called query(). However it does exactly the same thing as in your example. (See http://drupal.org/node/1819738 for more information).
So here was our final migration class:
class BookPageMigration extends DrupalNode6Migration {
public function __construct(array $arguments) {
parent::__construct($arguments);
}
/**
* Overrides parent::query to add more data to the source, in our case,
* book hierarchy stuff.
*/
public function query() {
$query = parent::query();
// Add in book parent child relationships.
$query->join('book', 'b', 'n.nid = b.nid');
$query->addField('b', 'bid', 'book_id');
$query->join('menu_links', 'ml', 'b.mlid = ml.mlid');
$query->addField('ml', 'weight', 'book_weight');
return $query;
}
/**
* Acts right after a book node has been saved. Map the book hierarchy.
*
* @param object $node
* A node object for the newly migrated book.
* @param stdClass $row
* An object representing data from the source row.
*/
public function complete($node, stdClass $row) {
// If this is a top-level book, don't set a parent.
$book_id = $row->nid == $row->book_id ? $node->nid : $this->lookupMigratedBook($row->book_id);
// Book id - effectively, parent nid.
$node->book['bid'] = $book_id;
$node->book['nid'] = $node->nid;
$node->book['weight'] = $row->book_weight;
_book_update_outline($node);
node_save($node);
}
/**
* Returns a mapping for a migrated book.
*
* @param int $source_book_nid
* Nid of book in source database to lookup.
*/
protected function lookupMigratedBook($source_book_nid) {
$dest_book_nid = parent::handleSourceMigration('ExBookPage', $source_book_nid);
return $dest_book_nid;
}
}
Note that "ExBookPage" was the "machine_name" we gave to this migration in our custom migration module. Worked like a charm! Hope this helps someone else.
I really appreciate all the
I really appreciate all the expanding on this topic that others are doing. When I wrote the original it was out of frustration of not being able to find any practical examples of how this could be done, so seeing other use it and go further is very encouraging. Keep it up!
Eclipse
Eclipse
Drupal Evangelist