Support for importing forums

Note that in addition to prepare/complete (which are executed for each input row), there are preImport/postImport (executed before and after the full migration import is run) - updating/resetting the variable would best be done there.

public function __construct() {
...
  $this->containers = variable_get('forum_containers', array());
...
}

public function complete($entity, stdClass $row) {
  if ((!$entity->parent || !$entity->parent[0]) && !in_array($entity->tid, $this->containers) {
    $this->containers[] = $entity->tid;
  }
}

public function postImport() {
  variable_set('forum_containers', $this->containers);
}

public function postRollback() {
  // This assumes the only forums are those being imported - to preserve any forums created before
  // migration, they would need to be either hard-coded here or saved to persistent storage for 
  // restoration here
  variable_set('forum_containers', array());
}

I'll add a page to the docs when I have a chance, thanks!

Untested:

public function completeRollback($entity_id) {
  $this->removedTermIds[] = $entity_id;
}

There is also a corresponding prepareRollback($entity_id), called before taxonomy_term_delete/node_delete/whatever_delete.

Hello,

I only have one forum category/board, so I want everything to be in the forum with tid = 1.
You say forum importation works fine for you, but I cannot update forum and forum_index table values; then, even though my nodes and comments are created (with the right taxonomy_forum value), they do not appear on my website.

What is the trick ?

A tiny mistake

It should be:
if (!in_array($container, $containers)) {
not
if (!in_array($entity->tid, $containers)) {

in the postImport function. Now it just keeps adding to the array because $entity does not exist.

Just to add another example, here's how I managed containers.

function postImport() {
  parent::postImport();
  
  $mapTable = $this->map->getMapTable();
  
  // Get current containers
  $current_containers = variable_get('forum_containers', array());
  
  // Get IDs of of all categories that were imported
  $result = db_query('select destid1 from ' . $mapTable);
  $new_containers = $result->fetchCol();
  
  $containers = array_merge($current_containers, $new_containers);
  variable_set('forum_containers', $containers);
}

function preRollback() {
  parent::preRollback();

  $mapTable = $this->map->getMapTable();

  // Get IDs of all categories that were imported
  $result = db_query('select destid1 from ' . $mapTable);
  $imported_containers = $result->fetchCol();

  // Save these for postRollback()
  $this->imported_containers = $imported_containers;
}

function postRollback() {
  parent::postRollback();

  // Get current containers
  $current_containers = variable_get('forum_containers', array());
  
  // Remove any containers from this list that were imported
  $containers = array_diff($current_containers, $this->imported_containers);

  variable_set('forum_containers', $containers);
}

Just in case anyone is looking for more info, I've added half a dozen pages on migrating a vBulletin forum to Drupal 7 to the documentation

Looking back at this, we could easily define a MigrateDestinationForum class that extends MigrateDestinationTerm, adding a "container" boolean to the list of destination fields and managing the forum_containers variable automatically...

Hi Mike,

If it hasn't been implemented already, would it be possible to make this a sort of general MigrateDestinationStructure? It could then be used for forums, as well as galleries, directories etc, which also use containers that aren't quite the same as taxonomies?

Stuart

Hi, i have problem with #5 approach. I migrating forums from D6 to D7. All forum terms are migrated correctly and all correctly marked as containers. But rollback not working as expected. All rows from map table is removed but all terms still remains in D7 database. If i debug my code i find out my function completeRollback is never called.

/**
 * Forum taxonomy migration.
 */
class ForumTaxonomyMigration extends NextdocMigration {
  public $oldContainers = array();
  public $newContainers = array();
  public $deletedContainers = array();

  public function __construct() {
    parent::__construct();
    $this->description = t('Turnusartz taxonomy forum to Nextdoc');
    $this->map = new MigrateSQLMap($this->machineName,
      array(
        'tid' => array(
          'type' => 'int',
          'unsigned' => TRUE,
          'not null' => TRUE,
          'description' => 'D6 Unique term ID',
          'alias' => 't',
        )
      ),
      MigrateDestinationTerm::getKeySchema()
    );

    $query = Database::getConnection('default', 'old')->select('term_data', 'td');
    $query->fields('td', array('tid', 'name', 'description', 'weight'))
          ->fields('th', array('parent'));
    $query->join('term_hierarchy', 'th', 'th.tid = td.tid');
    $query->condition('td.vid', 1);
    $query->orderBy('th.parent');
    $query->orderBy('td.weight');

    //dpq($query);

    // set source and destination
    $this->source = new MigrateSourceSQL($query, array(), NULL, array('map_joinable' => FALSE));
    $this->destination = new MigrateDestinationTerm('forums');

    $this->addFieldMapping('name', 'name');
    $this->addFieldMapping('description', 'description');
    $this->addFieldMapping('parent', 'parent')
         ->sourceMigration($this->getMachineName())
         ->defaultValue(NULL);
    $this->addFieldMapping('format')
         ->defaultValue(1)
         ->callbacks(array($this, 'mapOldFormatToFormat'));
    $this->addFieldMapping('weight', 'weight');
    $this->addFieldMapping('group_audience')->defaultValue(35);
  }

  /** Import related functions ***********************************************/

  /**
   * Add forum containers for variable.
   * @return void
   */
  public function complete($entity, stdClass $row) {
    drush_print('complete');

    if (in_array($row->tid, $this->oldContainers)) {
      $this->newContainers[] = $entity->tid;
    }
  }


  /**
   * Get old forum containers.
   * @return void
   */
  public function preImport() {
    parent::preImport();
    drush_print('preImport');

    $query = Database::getConnection('default', 'old')->select('variable', 'v');
    $query->fields('v', array('value'));
    $query->condition('v.name', 'forum_containers');
    $this->oldContainers = unserialize($query->execute()->fetchField());
  }

  /**
   * Add forum containers to variable.
   * @return void
   */
  public function postImport() {
    parent::postImport();
    drush_print('postImport');

    $forum_containers = variable_get('forum_containers', array());
    foreach($this->newContainers as $container){
      // check if term already exists - if not add them
      if (!in_array($container, $forum_containers)) {
        $forum_containers[] = $container;
      }
    }
    variable_set('forum_containers', $forum_containers);
  }

  /** Rollback related functions *********************************************/

  /**
   * Add forum containers for delete.
   * @param $entity_id
   * @return void
   */
  public function completeRollback($entity_id) {
    drush_print('completeRollback' . $entity_id);

    if (in_array($entity_id, $this->newContainers)) {
      $this->deletedContainers[] = $entity_id;
    }
  }

  /**
   * Get new forum containers before rollback begin.
   * @return void
   */
  public function preRollback() {
    parent::preRollback();
    drush_print('preRollback');

    $this->newContainers = variable_get('forum_containers', array());
  }

  /**
   * Remove forum containers from variable.
   * @return void
   */
  public function postRollback() {
    parent::postRollback();
    drush_print('postRollback');

    $containers = variable_get('forum_containers', array());
    $remaining = array();
    foreach ($containers as $container) {
      if (!in_array($container, $this->deletedContainers) && !in_array($container, $remaining)) {
        $remaining[] = $container;
      }
    }
    variable_set('forum_containers', $remaining);
  }
}

Thanks for any advice.

I found out problem was in module og_forum. This module prevent migration load taxonomy_terms and then this terms remains in taxonomy_term_data table. If i disable this module rollback run correct and remove all taxonomy terms and containers.

Also if you imported forum topic you need set destination variable forum_id. I use for now:

  public function prepare($entity, $row) {
    $entity->forum_tid = $entity->taxonomy_forums[LANGUAGE_NONE][0]['tid'];
  }

Then node correctly stored and show on its forum. But this still create notice in function forum_node_presave() :-(.

--

Edit:

I find solution (thanks to Devel issue #839770) - for forum nodes we need set node nid to NULL:

  public function prepare($entity, $row) {
    $entity->forum_tid = $entity->taxonomy_forums[LANGUAGE_NONE][0]['tid'];
    $entity->nid = NULL;
  }

Then notice go on.

Discussion seems complete on this old issue. Closing.