Migrations and migration groups

For the migration classes you implement to be available for reviewing status, running imports, etc., instances of them must be registered. There are two methods of getting your migrations and migration groups registered:

hook_migrate_api()

This is the recommended means of registering your migrations and migration groups if they're static - that is, if you don't need to pick-and-choose at runtime what migrations to register or what arguments to set for them.

It's recommended that you explicitly assign your migrations to a migration group (if you don't, they will be in the 'default' group). Groups can be registered in the API array by including a 'groups' key, whose value is an array keyed by the group machine name and the value an array of arguments for the group. The 'title' argument is special - this is the name of the group as it will be presented in the UI (if you omit the title, the machine name will be displayed instead). Any other arguments are saved as part of the group, and made available to any migrations in the group when they are instantiated.

The 'migrations' key should be an array, keyed by the migration machine name, with the value an array of arguments. 'class_name' is the one required argument (although 'group_name' is recommended) - any others are class-specific arguments, and are passed in the $arguments array to the class constructor.

Note that you can pass a list of hooks to disable during migration through the arguments - see Disable hooks during migration for details.

function example_migrate_api() {
  $api = array(
    'api' => 2,
    'groups' => array(
      'example' => array(
        'title' => t('Example'),
        'default_format' => 'filtered_html',
      ),
    ),
    'migrations' => array(
      'ExampleUser' => array(
        'class_name' => 'ExampleUserMigration',
        'group_name' => 'example',
      ),
      'ExampleArticle' => array(
        'class_name' => 'ExampleArticleMigration',
        'group_name' => 'example',
      ),
    ),
  );
  return $api;
}

After adding new migrations, or making changes to the arguments of previously-registered migrations, if you want them to be recognized by Migrate, you need to:

  1. Clear the Drupal cache, so any new classes are added to the Drupal cache registry.
  2. Perform the registration, either with drush migrate-register or by visiting admin/content/migrate/configure and clicking the "Register statically-defined classes" button.

MigrationBase::registerMigration()

This would typically be used when the arguments, or the list of migrations to register, is determined dynamically (as when you have a UI letting users choose what to migrate). The first argument is the class name to be instantiated, the second the machine name by which the migration will be known, and the third argument is an array of arguments to be passed to the migration constructor:

MigrationBase::registerMigration('ExampleUserMigration', 'ExampleUser', array('group_name' => 'example', 'default_uid' => 1));

Destination and field handler classes

Destination and field handler classes defined by your module must be explicitly registered in hook_migrate_api as follows:

function example_migrate_api() {
  $api = array(
    'api' => 2,
    'destination handlers' => array(
      'ExampleNodeHandler',
    ),
    'field handlers' => array(
      'ExampleCustomFieldHandler',
    )
  );
  return $api;
}

Comments

ifrik’s picture

This page duplicates a lot of the content from the previous page. It would be clearer for visitors if we add anything that's missing about using the api into that page, and keep this for explaining how to use MigrationBase::registerMigration()

Building websites for a better world

nakedscientist’s picture

It would be really helpful to know where this code is supposed to go! There is no instruction regarding which of the three files we're instructed to create (.module / .migrate.inc / .info) on the previous page should contain this hook code.

Can someone please clarify?

ehj-52n’s picture

The examples are placing the api hook in the ...migrate.inc files.

But in my case I doesn't result in listed migrations. :-(

This issue was caused by missing dependencies. Now, everything is listed correctly.

vabii12’s picture

The _migrate_api code is in .migrate.inc and the 'MigrationBase::registerMigration' code is in .module file. I kept the Migration class in a separate file but I have read that one can put that in .module file also. Hope this helps!

ehj-52n’s picture

.

bburg’s picture

Is there to define the order of migration groups? e.g. when I run drush migrate --all, what dictates the order between groups? The dependency system seems to handle ordering within groups fine, but fails a bit between groups.

Metatag should be in core.

robneidel’s picture

have you find a solution to order migration groups manually? i have got the same problem.

bburg’s picture

@robneidel I just wrote a shell script that ran my migrations in the order I wanted. Here are some snippets:

Migrate script

#!/bin/bash
#
# Requirements: Drush, Bash
#
# Usage:
# Update the values for DIR and OUTPUT in this script
# DIR: This should be the root of the destination Drupal site
# OUTPUT: This is a reference to a file that will direct the output of the migration to that file.
#
# Rollbacks can be run but using the accompanying rollback.sh script. The same instructions apply to using that script
# as well.
#
# If any of the migration fail, or the script ends prematurely, simply continue the migration process by copying the
# commands below (in order) and executing them against the D7 site (without the ">>$OUTPUT 2>&1 || true", or replace
# with the path to the logging file).
#

# Print commands to the console.
set -x

# Don't error out when one of the commands here errors.
set +e

set -v
# Run migrations in dependency order.

printf "Initializing"
start=`date +%s`

# F1 dev environment
DIR="/var/www/vhosts/mysite.dev/public"
OUTPUT="/var/www/vhosts/mysite.dev/migrate-output.txt"

# Vagrant environment
DIR="/vagrant/public"
OUTPUT="/vagrant/migrate-output.txt"

DATE=`date +%Y-%m-%d:%H:%M:%S`

printf "\nMigration on $DATE\n" >>$OUTPUT 2>&1

cd $DIR

# Stop and reset any currently running migrations
drush mst --all >>$OUTPUT 2>&1 || true
drush mrs --all >>$OUTPUT 2>&1 || true

# re-register migrations.
drush migrate-register >>$OUTPUT 2>&1 || true


printf "\nRunning migrations"
# The || true suppresses
drush mi MyUser6Migration >>$OUTPUT 2>&1 || true

drush mi --group=vocabularies >>$OUTPUT 2>&1 || true

...

end=`date +%s`

runtime=$((end-start))

printf "\nMigration took $runtime seconds to run\n" >>$OUTPUT 2>&1

Rollback script

#!/bin/bash
# This rolls back a migration by performing rollbacks in the reverse order of the migration.

#DIR="/var/www/vhosts/mysite.dev/public"
#OUTPUT="/var/www/vhosts/mysite.dev/migrate-output.txt"

DIR="/vagrant/public"
OUTPUT="/vagrant/migrate-output.txt"


# Print commands to the console.
set -x

# Don't error out when one of the commands here errors.
set +e

set -v

DATE=`date +%Y-%m-%d:%H:%M:%S`

printf "Rollback on \n$DATE\n" >>$OUTPUT 2>&1

start=`date +%s`

cd $DIR

# Stop and reset any currently running migrations
drush mst --all || true
drush mrs --all || true

drush migrate-register || true

drush cc drush || true

# Comments
drush mr --group=comment >>$OUTPUT 2>&1 || true

# Other types
drush mr ResourceMigration >>$OUTPUT 2>&1 || true

end=`date +%s`

runtime=$((end-start))

printf "\nRollback took $runtime seconds to run\n" >>$OUTPUT 2>&1

I think the " || true" bits are totally unnecessary, they force the script to continue if the command causes an error. I was having problems with the script error-ing out before I learned that shell scripts are read one line at a time, and editing a script while it is running WILL cause errors! Which I was likely to do, since my migration took a couple of hours to complete.

Metatag should be in core.