The Migrate module provides an abstract class named Migration - in most cases, you'll do nearly all your work in extending this class. The Migration class handles the details of performing the migration for you - iterating over source records, creating destination objects, and keeping track of the relationships between them.

Steps to take:

  1. Define where the source data is coming from.
  2. Define the destination object type and "bundle" (where appropriate).
  3. Define the mapping between a single row of source data and the resulting Drupal data.
  4. Define mappings from source fields to destination fields.

In doing so, you will make use of other classes provided by Migrate - some you can use as-is, some you may need to override.

Here is an example of a bare-bones (but fully-functional) migration class (source data not included). Scroll down to get the detailed explanation of what every piece of code does:

class ExampleMigration extends Migration {
  public function __construct($arguments) {
    parent::__construct($arguments);
    $query = Database::getConnection('default', 'legacy')
             ->select('example_pages')
             ->fields('example_pages', array('pgid', 'page_title', 'page_body'));
    $this->source = new MigrateSourceSQL($query);
    $this->destination = new MigrateDestinationNode('page');

    $this->map = new MigrateSQLMap($this->machineName,
        array(
          'pgid' => array('type' => 'int',
                          'unsigned' => TRUE,
                          'not null' => TRUE,
                          'description' => t('Source ID'),
                         )
        ),
        MigrateDestinationNode::getKeySchema()
      );

    $this->addFieldMapping('title', 'page_title');
    $this->addFieldMapping('body', 'page_body');
  }
}

Though you can place this code your .module file, it's a good practice is to create separate .inc files for each migration or set of closely-related migrations. Be sure to explicitly list each .inc file in your .info, otherwise they will not be discoverable by Drupal.

Once you have defined this migration class (and registered it), you should be able to see your migration class with the command:

drush migrate-status

Then, assuming you registered your migration with the machine name "Example",

drush migrate-import Example

will create a Drupal page node for each row in the example_pages table. Then, running

drush migrate-rollback Example

will undo the import, deleting the pages that were initially created.

Let's take it apart piece by piece:

Class declaration

First, the basic overhead of extending a class and overriding its constructor:

class ExampleMigration extends Migration {
  public function __construct($arguments) {
    parent::__construct($arguments);

This says that the class we're declaring, ExampleMigration, will behave exactly like the Migration class except for any behavior we override. Then, we override the class constructor - this is the first method to be called when an instance of the class is instantiated. In almost all cases, you begin a method by calling the parent's implementation of that method - usually, you are adding behavior to the base class, so you want to make sure that the base class does its thing.

Note the parameter to the constructor. An array of arguments, specified at the time the migration was registered, is passed in, and then passed on down to the parent constructor. Usually this will include a 'group_name" member - if at registration time we had included 'group_name' => 'example' in the arguments array, this assigns our migration to the 'example' group, and if we have multiple migrations assigned to this group we can run all them with:

drush migrate-import --group=example

This may be useful if you have many migrations which fall into logical groups - say, a group for importing from a MySQL database, and another group operating off XML feeds.

The source object

Next, we identify where our source data is coming from:

    $query = Database::getConnection('default', 'legacy')
             ->select('example_pages')
             ->fields('example_pages', array('pgid', 'page_title', 'page_body'));
    $this->source = new MigrateSourceSQL($query);

Our data is in an external database, and we have set up a database connection named 'legacy' to point to it. So, to reference that database instead of the default Drupal database, we replace the familiar db_select() call with a Database::getConnection() to retrieve the connection, and a select() on that connection (db_select() is really just a shortcut for Database::getConnection('default', 'default')->select()).

It is important that this query return one row containing all the data for what will become a single node - dealing with cases where the data might span multiple rows (e.g., if we also joined to a category table with multiple rows per page) is a little more advanced and is discussed elsewhere.

Once we've constructed the query, we use it to create a MigrateSourceSQL object and save it in our ExampleMigration object.

The destination object

Next, we identify what we want to create from the source data:

    $this->destination = new MigrateDestinationNode('page');

That was even easier than the source - we just told the migration class we want to create nodes of the type (or, in Drupal 7 parlance, bundle) page.

The map object

Next, we tell the migrate module how source and destination records are related.

    $this->map = new MigrateSQLMap($this->machineName,
        array(
          'pgid' => array('type' => 'int',
                          'unsigned' => TRUE,
                          'not null' => TRUE,
                          'description' => t('Source ID'),
                         )
        ),
        MigrateDestinationNode::getKeySchema()
      );

To track migration status, and allow rollback, the migrate module needs to remember which source record resulted in the creation of which destination object. It does this using a MigrateSQLMap object - this class creates database tables which map source and destination keys. To do this, it needs to know the data types of the respective keys. Thus, we pass in schema arrays declaring the respective source and destination keys. For the source key, since the Migrate module knows nothing about the source table, you need to explicitly define the schema array (making sure that the key of the array, pgid in this case, matches the field name used in the query passed to MigrateSourceSQL above). On the other hand, since we're using the canned destination class MigrateDestinationNode, it knows what the correct schema is already and can provide it through the static method getKeySchema().

Oh, and the machineName passed as the first argument? MigrateSQLMap will create a map table and message table for each migration, and will name them by prepending migrate_map_ and migrate_message_ to this parameter. It is a handy convention to pass your migration's machine name here, so that the names are unique and it's easy to identify what migration they're associated with, but you could pass a different suffix here if you prefer (as when your migration machine names are very long and risk creating table names that are too long).

Field mapping objects

Finally, we define data mappings:

    $this->addFieldMapping('title', 'page_title');
    $this->addFieldMapping('body', 'page_body');
  }
}

This tells the migration that, for each source row processed, to take the page node title from the source row's page_title field, and the node body from the page_body field. When looking at an addFieldMapping() call, remember that the destination field is the first parameter and the source field is the second parameter. Think of it like an assignment statement:

    $title = $page_title;

Comments

rvandyke’s picture

Any one know where I can find sample code to import from a CSV file?

gbernier’s picture

If you are still looking for an example CSV migrate profile I'm almost done working the kinks out of one myself I will be sharing. Process not as migrate newbie friendly as I'd hoped so will be sharing my experience and code.

Gene Bernier
gene@cheekymonkeymedia.ca
COO, Top Monkey Wrencher
Cheeky Monkey Media
http://cheekymonkeymedia.ca

paralax’s picture

I am very interested on your CSV Class, would post it please so I may get an idea where to start?
thanks
patrick

Anonymous’s picture

These docs are excellent, but now you should follow their advice and go read their example, beer.inc.
It's heavily commented, and includes tips like converting a date/time string into a UNIX timestamp that Drupal understands. Print it out and take notes. :)

Here's the git version of 6.x-2.0:
http://drupalcode.org/project/migrate.git/blob/836109b2c3098109d6ed8ff77...

geografa’s picture

So true ^^ Really helpful doc. Thanks!

paralax’s picture

Here is an example import from a CSV file:


class ImportExampleCSVMigration extends Migration {
  	public function __construct() {
    	parent::__construct();
		
		//The defintion of the collumns. Keys are integers. values are array(field name, description).
	    $columns[0] = array('id_csv', 'Id');
	    $columns[1] = array('title_csv', 'Title');
	    $columns[2] = array('body_csv', 'Body');
	    $columns[3] = array('another_field_csv', 'Just another field');
	
		//The Description of the import. This desription is shown on the Migrate GUI
    	$this->description = t('A CSV sample import ');

		//The Source of the import
		$this->source = new MigrateSourceCSV(drupal_get_path('module', 'module_name').'/import_file_name.csv', $columns);

		//The destination CCK (boundle)
		$this->destination = new MigrateDestinationNode('cck_maschine_name');

		//Source and destination relation for rollbacks
		$this->map = new MigrateSQLMap(
			$this->machineName, 
			array(
				'id_csv' => array(
					'type' => 'int',
					'unsigned' => TRUE,
					'not null' => TRUE,
					'alias' => 'import'
				)
			),
			MigrateDestinationNode::getKeySchema()
		);

		//Field ampping
		$this->addFieldMapping('title', 'title_csv');
		$this->addFieldMapping('body', 'body_csv');
		$this->addFieldMapping('another_field', 'another_field_csv');
	}
}

Niklas Fiekas’s picture

It took me a while to find out that: #982694: Uppercase letters in column names not supported.
This caused strange error when used in the MigrateSQLMap source key schema definition.

adub’s picture

When defining your destination node type, hyphens should be replaced with underscores e.g.

$this->destination = new MigrateDestinationNode('my_custom_content_type');
dshumaker’s picture

I'm looking for reasons why and descriptions of when to use the sourceMigration function. In the above example it is not called and yet in the beer and wine examples it is.

Why? What does it signify and why is it not necessary sometimes?