Migrating Book module nodes

Note: for migrate menu destination support and a Drupal-to-Drupal menu migration class that uses it, see #763880: Import Hierarchy as Book and #2146961: Preserve Book hierarchies respectively.

Migrating book module nodes is one of the more difficult tasks to accomplish with Migrate module. This is due largely to certain workflow issues that can come up during migrating book nodes, and the fact that book node hierarchy is stored within menu links (Drupal 6 and later). The following code is an example of how to deal with this.

For migrating from Drupal 5 see below. The concept used there may also be used for migrating from Drupal 6 with some rewrites (to join the menu links). Its advantage is, that only one migration is used instead of two.

Migrating from Drupal 6

Overview

• Migrate nodes as normal;
• Get hierarchy from the source database 'book' table, and weights from the 'menu_links' table;
• Use complete() in your Migration class to call _book_update_outline() to migrate the hierarchy.

Explanation

Books are nodes, so they can inherit most of their migration from a common class. You can use something like the Drupal-to-Drupal data migration classes, or roll your own, like the one provided below.

Node migration

Let's kick this off with a good NodeMigration abstract class that should get us most of the way there. This class should be useful for other node types as well.

abstract class NodeMigration extends Migration {
  public function __construct(array $arguments) {
    parent::__construct($arguments);
    $type = isset($arguments['type']) ? $arguments['type'] : NULL;
    $new_type = isset($arguments['new type']) ? $arguments['new type'] : $type;
    $this->description = t('Migrate nodes');
    $this->map = new MigrateSQLMap($this->machineName,
      array(
        'nid' => array(
          'type' => 'int',
          'unsigned' => TRUE,
          'not null' => TRUE,
          'description' => 'd6 Unique Node ID',
          'alias' => 'n',
        )
      ),
      MigrateDestinationNode::getKeySchema()
    );

    $query = $this->node_query($type);
    $this->set_highwater_field();
    $this->set_source($query);
    $this->destination = new MigrateDestinationNode($new_type);
    $this->node_field_mapping();
  }

  public function set_highwater_field() {
    $this->highwaterField = array(
      'name' => 'changed',
      'alias' => 'n',
    );
  }

  public function node_query($type) {
    $query = Database::getConnection('d6')
      ->select('node', 'n')
      ->fields('n', array('nid', 'vid', 'title', 'uid', 'status', 'created', 'changed', 'comment', 'promote', 'moderate', 'sticky'))
      ->condition('n.type', $type);
    $query->join('node_revisions', 'nr', 'nr.vid = n.vid');
    $query->fields('nr', array('body', 'teaser', 'format'));
    return $query;
  }

  public function set_source($query) {
    $this->source = new MigrateSourceSQL($query);
    $this->source->setMapJoinable(FALSE);
  }

  public function node_field_mapping() {
    $body_arguments = MigrateTextFieldHandler::arguments(array('source_field' => 'teaser'), array('source_field' => 'format'), NULL);
    // Make the mappings
    $this->addSimpleMappings(
      array(
        // Assumes you have done a migration of Users which maintains uids.
        'uid',
        'title',
        'status',
        'created',
        'changed',
        'comment',
        'promote',
        'moderate',
        'sticky',
      )
    );
    $this->addFieldMapping('body', 'body')->arguments($body_arguments);
    $this->addFieldMapping('language')->defaultValue('en');
    $this->addFieldMapping('is_new')->defaultValue(TRUE);
  }

  public function prepareRow($current_row) {
    $formats = array(
      '1' => 'filtered_html',
      '2' => 'full_html',
      '3' => 'plain_text',
      '4' => 'markdown',
    );
    $current_row->format = isset($formats[$current_row->format]) ? $formats[$current_row->format] : 'plain_text';
  }
}

There are some components of this you should change for your site, especially the $formats array in prepareRow(). In general this breaks a typical node migration into its various components of:

1. What are we migrating? (mostly $this->map)
2. How do we get it? ($this->node_query())
3. How do we update it later? ($this->set_highwater_field())
4. Set the source
5. Set the Destination
6. Do field mappings ($this->node_field_mapping())

This gives us enough granularity to write fairly minor migration classes from this point forward. As an example, migrating the default page nodes with this in hand looks something like:

class NodePageMigration extends NodeMigration {
  public function __construct() {
    parent::__construct(array('type' => 'page'));
  }
}

And migrating Story nodes to Article nodes could look like:

class NodeArticleMigration extends NodeMigration {
  public function __construct() {
    parent::__construct(array('type' => 'story', 'new type' => 'article'));
  }
}

Adding book hierarchies

So, with that foundation, let's discuss migrating book nodes.

The only difference between book nodes and other types of nodes is their table of contents hierarchy. This data is stored in its own table, handily called 'book'. There's also some information about relative weights for book items stored in menus.

There's three things we need to do to bring in the TOC data:

• Get the source data from the source database;
• Map source nids to destination nids;
• Update each migrated book with its position in its TOC.

class NodeBookMigration extends NodeMigration {
  public function __construct() {
    parent::__construct(array('type' => 'book'));
  }

  /**
   * Overrides parent::nodeQuery to add more data to the source, in our case,
   * book hierarchy stuff.
   */
  public function nodeQuery() {
    $query = parent::nodeQuery();
    // Add in book parent child relationships.
    $query->leftJoin('book', 'b', 'n.nid = b.nid');
    $query->addField('b', 'bid', 'book_id');
    $query->leftJoin('menu_links', 'ml', 'b.mlid = ml.mlid');
    $query->addField('ml', 'weight', 'book_weight');

    return $query;
  }
  
  /**
   * Acts right after a book node has been saved. Map the book hierarchy.
   *
   * @param object $node
   *   A node object for the newly migrated book.
   * @param stdClass $row
   *   An object representing data from the source row.
   */
  public function complete($node, stdClass $row) {
    // If this is a top-level book, don't set a parent.
    $book_id = $row->nid == $row->book_id ? $node->nid : $this->lookupMigratedBook($row->book_id);
    // If we're updating a migration, this node may already have a mlid.
    $mlid = db_query("SELECT mlid FROM {book} WHERE nid = :nid", array(
      ':nid' => $node->nid,
    ))->fetchAssoc();
    $mlid = (empty($mlid)) ? NULL : $mlid['mlid'];
    // Book id - effectively, parent nid.
    $node->book['bid'] = $book_id;
    $node->book['nid'] = $node->nid;
    $node->book['mlid'] = $mlid;
    $node->book['weight'] = $row->book_weight;
    _book_update_outline($node);
    node_save($node);
  }

  /**
   * Returns a mapping for a migrated book.
   *
   * @param int $source_book_nid
   *   Nid of book in source database to lookup.
   */
  protected function lookupMigratedBook($source_book_nid) {
    $dest_book_nid = parent::handleSourceMigration('NodeBook', $source_book_nid);
    return $dest_book_nid;
  }
}

So far, so good. If you check your migration at admin/content/migrate, you should be able to see the new fields in your source data.

Migrate module helps us out in two ways here.

• We use a complete() method which runs straight after an individual node has been migrated. This way, we can just call a book module function to add book-specific data.
• Migrate API keeps mappings of source nids to migrated nids. We can then manually call parent::handleSourceMigration() to lookup the mapping. Previously, I tried setting a sourceMigration on the book_parent_nid field mapping, but this creates unwanted stubs for top-level books. Doing a manual lookup ensures that nodes are only looked up (or created as stubs) when required.

I have used this approach to migrate several hundred book nodes with great success. I highly recommend using Drupal-to-Drupal data migration rather than rolling your own node migration classes, but every Migration is different, so do what you have to.

Migrating from Drupal 5

Drupal 5 does not make use of the menu system for books as Drupal 6. It's all in the book table.

To import books from Drupal 5 I use migrate_d2d, a custom destination handler and an extended DrupalNode5Migration class as below (all together in my node_book.inc file).

You have to check how deep is your deepest book hierarchy (in my case it is 5). Not sure if this could be done dynamically or in one central place, but for me setting $max_depth manually is just fine.

The Migration could be done in one single migration by joining all necessary information in one query. This includes multiple joins of the book table on the parent nid with the book items nid, to get the parent of a parent and its parent and so on.

The orderBy clause makes sure that all parents are imported before the children. As one book page has only one parent this should be safe and there should be no need to create stubs.

Attention: Maybe the mapping from legacy nids to new nids is missing as I use the value mapped to nid as its ID, rather than generated a new sequential ID (see MigrateDestinationNode > Fields > is_new) – so stuff in prepare() should better be in complete().

/**
 * @file
 * Implementation of MyNodeBookMigration for Drupal 5 sources.
 */

/**
 * Handling book nodes with book type article.
 */
class MyNodeBookMigration extends DrupalNode5Migration {

  /**
   * @param array $arguments
   */
  public function __construct(array $arguments) {
    parent::__construct($arguments);

    $this->addFieldMapping('book', NULL)
         ->description('Is book/bookpage')
         ->defaultValue(TRUE);

    $this->addFieldMapping('book:pid', 'b_parent')
         ->description('Book Parent')
         ->defaultValue(0);

    $this->addFieldMapping('book:weight', 'b_weight')
         ->description('Book Weight')
         ->defaultValue(0);

    $max_depth = 5;
    for ($depth = 1; $depth <= $max_depth; $depth++) {
      // $this->query().
      $this->addFieldMapping('book:nid' . $depth, 'b' . $depth . '_nid')
           ->description('Book Parent ID from join nr. ' .$depth);
    }
  }
  /**
   * Query for basic node fields from Drupal 5.
   *
   * @return QueryConditionInterface
   */
  protected function query() {
    $query = parent::query();

    $query->innerJoin('book', 'b', 'n.vid=b.vid');
    $query->addField('b', 'weight', 'b_weight');
    $query->addField('b', 'parent', 'b_parent');
    // $query->isNotNull('b.vid');

    // Join book table on parent node ids $max_depth times to sort nodes, so
    // that all book-nodes (parents = 0 come first), then all subpages and then
    // all sub-subpages so on till the pages with maximum depth.
    // In our case is 5 for maximum depth enough.
    $max_depth = 5;
    for ($depth = 1; $depth <= $max_depth; $depth++) {
      // Alias for the first time book table
      // was joined is b, not b0.
      $query->addField('b' . $depth, 'nid', 'b' . $depth . '_nid');
      $condition = sprintf('b%s.parent=b%s.nid', $depth - 1 == 0 ? '' : $depth - 1, $depth);
      $query->leftJoin('book', 'b' . $depth, $condition);
      // Sort ascending so NULL comes first, starting with the deepest item
      // ensures that all parent items where created before adding an item to
      // to the outline.
      if ($max_depth - $depth) {
        $query->orderBy('b' . ($max_depth - $depth) . '.parent', 'ASC');
      }
    }
    $query->orderBy('n.nid');

    return $query;
  }
}

class MigrateBookDestinationHandler extends MigrateDestinationHandler {
  public function __construct() {
    $this->registerTypes(array('node'));
  }

  /**
   * Implementation of MigrateDestinationHandler::fields().
   */
  public function fields($entity_type, $bundle, $migration = NULL) {
    $fields = array();

    if (module_exists('book')) {
      $fields['book'] = t('Is book/bookpage.');
      $fields['book:bid'] = t('Subfield: ID of book to add this bookpage.');
      $fields['book:pid'] = t('Subfield: Parent Node ID.');
      $fields['book:weight'] = t('Subfield: Book weight.');

      $max_depth = 5;
      for ($depth = 1; $depth <= $max_depth; $depth++) {
        $fields['book:nid' . $depth] = t('Subfield: !depth. parents node ID.', array('!deepth' => $depth));
      }
    }

    return $fields;
  }

  public function prepare($entity, stdClass $row) {
    if (module_exists('book') && !empty($entity->book[0])) {

      // Unset the migrate variable $entity->book to set book object later;
      unset($entity->book[0]);

      $arguments = array();
      if (isset($entity->book['arguments'])) {
        $arguments = $entity->book['arguments'];
        unset($entity->book['arguments']);
      }

      // bid = nid in case it book-page is book itself.
      $entity->book['bid'] = $entity->nid;

      // In case the book has parents, we need to go to the top parent to get
      // the book id. Parents should be in b{$depth}_nid, see join.
      $max_depth = 5;
      for ($depth = 1; $depth <= $max_depth; $depth++) {
        if (!empty($arguments['nid' . $depth])) {
          $entity->book['bid'] = $arguments['nid' . $depth];
        }
        else {
          // If $depth parent is empty (0 or NULL) we already have the book id.
          break;
        }
      }

      // Get menu link id of parent.
      if (empty($entity->book['plid']) && $arguments['pid'] != $entity->book['bid']) {
        $entity->book['plid'] = db_select('book', 'b')
          ->fields('b', array('mlid'))
          ->condition('nid', $arguments['pid'])
          ->execute()
          ->fetchField();
      }

      if ($arguments['weight']) {
        $entity->book['weight'] = $arguments['weight'];
      }

      if ($entity->nid) {
        // For Updates we need to set book['mlid'], otherwise we get PDOException
        // because of inerting a duplicate entry (see _book_update_outline()).
        $query = db_select('book', 'b')
                  ->fields('b', array('mlid'))
                  ->condition('nid', $entity->nid);
        $query->leftJoin('menu_links', 'ml', 'b.mlid=ml.mlid');
        $query->fields('ml', array('has_children'));
        $book = $query->execute()->fetchObject();

        if ($book->mlid) {
          $entity->book['mlid'] = $book->mlid;
        }
        if (!isset($entity->book['has_children']) && $book->has_children) {
          $entity->book['has_children'] = $book->has_children;
        }
      }
    }
  }
}

Revisions

The book migration will always crate a new revision, if it is not executed by a user with the 'administer nodes' permissions, for example when using drush without the global option--user. For more details see issue #2504813: Updating book nodes allways crates new revision