Download & Extend

Support for importing forums

Project:Migrate
Version:7.x-2.x-dev
Component:Code
Category:feature request
Priority:normal
Assigned:Unassigned
Status:active

Issue Summary

I'm just sharing what I figured out about import forum board structures.

  • Ordering the query by parent id is a good idea (taken from migrate_example)
  • The variable forum_containers is used to decide what are containers and what are forums:
    <?php
      
    public function complete($entity, stdClass $row) {
          if (!
    $entity->parent || !$entity->parent[0]) {
           
    $containers = variable_get('forum_containers', array());
            if (!
    in_array($entity->tid, $containers)) {
             
    $containers[] = $entity->tid;
             
    variable_set('forum_containers', $containers);
            }
          }
        }
    ?>
    Is there a better way to update that variable? I found it just fine.
AttachmentSize
boardmigration.php_.txt1.94 KB

Comments

#1

Is there a way to hook in during rollback and undo the variable_set?

#2

Note that in addition to prepare/complete (which are executed for each input row), there are preImport/postImport (executed before and after the full migration import is run) - updating/resetting the variable would best be done there.

<?php
public function __construct() {
...
 
$this->containers = variable_get('forum_containers', array());
...
}

public function
complete($entity, stdClass $row) {
  if ((!
$entity->parent || !$entity->parent[0]) && !in_array($entity->tid, $this->containers) {
   
$this->containers[] = $entity->tid;
  }
}

public function
postImport() {
 
variable_set('forum_containers', $this->containers);
}

public function
postRollback() {
 
// This assumes the only forums are those being imported - to preserve any forums created before
  // migration, they would need to be either hard-coded here or saved to persistent storage for
  // restoration here
 
variable_set('forum_containers', array());
}
?>

I'll add a page to the docs when I have a chance, thanks!

#3

Very good.
I suppose there is no way to get the rolled back term ids in postRollback() or to gather them while the single rows are rolled back? postRollback would be much more useful, if you knew what was rolled back.

Update: Of course, the same is true for postImport().

#4

Untested:

<?php
public function completeRollback($entity_id) {
 
$this->removedTermIds[] = $entity_id;
}
?>

There is also a corresponding prepareRollback($entity_id), called before taxonomy_term_delete/node_delete/whatever_delete.

#5

Yes, that works well.

<?php
 
public function complete($entity, stdClass $row) {
    if (!
$entity->parent || !$entity->parent[0]) {
     
$this->addedContainers[] = $entity->tid;
    }
  }

  public function
postImport() {
   
$containers = variable_get('forum_containers', array());
    foreach (
$this->addedContainers as $container) {
      if (!
in_array($entity->tid, $containers)) {
       
$containers[] = $container;
      }
    }
   
variable_set('forum_containers', $containers);
  }

  public function
completeRollback($entity_id) {
   
$this->deletedContainers[] = $entity_id;
  }

  public function
postRollback() {
   
$containers = variable_get('forum_containers', array());
   
$remaining = array();
    foreach (
$containers as $container) {
      if (!
in_array($container, $this->deletedContainers) && !in_array($container, $remaining)) {
       
$remaining[] = $container;
      }
    }
   
variable_set('forum_containers', $remaining);
  }
?>
  • I use variable_get / variable_set only in the post(Import|Rollback) functions, to make unlikely race conditions even unlikelier.
  • In completeRollback I don't care if it was actually a parent container.

--

The Node (= Thread) and Comment (= Post) imports for this are just like any other.
So: "fixed for me".

#6

Hello,

I only have one forum category/board, so I want everything to be in the forum with tid = 1.
You say forum importation works fine for you, but I cannot update forum and forum_index table values; then, even though my nodes and comments are created (with the right taxonomy_forum value), they do not appear on my website.

What is the trick ?

#7

I barely have time for explaining, so I'll just attach my full import script for the chance that it might help. Terms are boards, nodes are threads, comments are posts.

#8

I barely have time for explaining, so I'll just attach my full import script for the chance that it might help. Terms are boards, nodes are threads, comments are posts.

AttachmentSize
forum.inc_.txt 7.19 KB

#9

A tiny mistake

It should be:
if (!in_array($container, $containers)) {
not
if (!in_array($entity->tid, $containers)) {

in the postImport function. Now it just keeps adding to the array because $entity does not exist.

#10

Thank you, that's right.

#11

Just to add another example, here's how I managed containers.

function postImport() {
  parent::postImport();
 
  $mapTable = $this->map->getMapTable();
 
  // Get current containers
  $current_containers = variable_get('forum_containers', array());
 
  // Get IDs of of all categories that were imported
  $result = db_query('select destid1 from ' . $mapTable);
  $new_containers = $result->fetchCol();
 
  $containers = array_merge($current_containers, $new_containers);
  variable_set('forum_containers', $containers);
}

function preRollback() {
  parent::preRollback();

  $mapTable = $this->map->getMapTable();

  // Get IDs of all categories that were imported
  $result = db_query('select destid1 from ' . $mapTable);
  $imported_containers = $result->fetchCol();

  // Save these for postRollback()
  $this->imported_containers = $imported_containers;
}

function postRollback() {
  parent::postRollback();

  // Get current containers
  $current_containers = variable_get('forum_containers', array());
 
  // Remove any containers from this list that were imported
  $containers = array_diff($current_containers, $this->imported_containers);

  variable_set('forum_containers', $containers);
}

#12

Just in case anyone is looking for more info, I've added half a dozen pages on migrating a vBulletin forum to Drupal 7 to the documentation

#13

Title:Documentation for importing forums» Support for importing forums
Component:Documentation» Code
Category:task» feature request

Looking back at this, we could easily define a MigrateDestinationForum class that extends MigrateDestinationTerm, adding a "container" boolean to the list of destination fields and managing the forum_containers variable automatically...

#14

Hi Mike,

If it hasn't been implemented already, would it be possible to make this a sort of general MigrateDestinationStructure? It could then be used for forums, as well as galleries, directories etc, which also use containers that aren't quite the same as taxonomies?

Stuart