Source fields are mapped to destination fields by making calls to $this->addFieldMapping() in your Migration class constructor. Field mappings are represented as instances of the MigrateFieldMapping class, which implements a fluent interface akin to that of the Drupal 7 database API. This enables optional behavior to be attached to mappings, without using a long list of optional arguments on addFieldMapping().

    // Each method returns a FieldMapping object
    $this->addFieldMapping('sticky')
         ->description(t('Should we default this to 0 or 1?'))
         ->issueGroup(t('Client questions'))
         ->issueNumber(765736)
         ->issuePriority(MigrateFieldMapping::ISSUE_PRIORITY_LOW);

The following methods can be chained:

  • ->description($description): provide a description for the mapping. This will be displayed in the Migrate UI.
  • ->defaultValue($value): provide a default value for this field.
  • ->separator($delimiter): the source string will be exploded to an array using the specified delimiter.
  • ->callbacks($callback1, $callback2, ...): adds functions to be executed in order to prepare the data for that specific field.
  • ->sourceMigration(array($migrationMachineName, ...)): If specified, the source value in the mapping will be looked up in the map tables for any listed migrations and if found, translated to the destination ID from the map table.
  • ->dedupe($table, $column): Search for duplicate value in $table and $column. When a duplicate value is found a counter will be appended.
  • ->issueGroup($group): Each issue group will generate a tab on the detail page in the UI, for grouping the mappings.
  • ->issueNumber($group): If present, used to link to an issue in an external ticketing system.
  • ->issuePriority($group): Indicates the priority of an issue (with highlighting for non-OK values).

Simple mapping

In its simplest form, a field mapping defines a straight copy of source data to the destination Drupal object:

$this->addFieldMapping('title', 'source_subject');

In a node migration, this indicates that when each source row is processed, the contents of its source_subject field will be copied to the node title.

It's not uncommon for fields to have the same name on both the source and destination sides. For example, when migrating nodes from a system with title and body fields, you might expect to write:

$this->addFieldMapping('title', 'title');
$this->addFieldMapping('body', 'body');

But, you can take a shortcut:

$this->addSimpleMappings(array('title', 'body'));

Similarly, when you have several destination or source fields which are not being migrated, rather than writing out an addFieldMapping() call (with issueGroup() attached) for each one, you can do it in one call:

$this->addUnmigratedDestinations(array('status', 'uid'));
$this->addUnmigratedSources(array('old_field', 'older_field', 'oldest_field'), t('Do Not Migrate'));

The issue group for these mappings defaults to DNM, but as you see in the source case above it can be overridden using the optional second parameter.

Provide Default values and fixed values

A default value can be provided:

$this->addFieldMapping('field_age_range', 'source_age_range')
     ->defaultValue(t('Unknown age range'));

This indicates that for every source row without the property "source_age_range", the resulting field_age_range should be set to 'Unknown age range'. To use default values for NULL or '' or array() values, be sure to remove the item with unset() in your prepareRow() function.

By omitting the source field, you can hard-code a value to be applied to every migrated row - in this case, we ensure that every migrated user has the "authenticated user" role:

$this->addFieldMapping('roles')
     ->defaultValue(DRUPAL_AUTHENTICATED_RID);

Convert source strings to arrays

Multiple values for a field can be migrated directly:

$this->addFieldMapping('field_tags', 'tags')
     ->separator(',');

If the source field tags contains a comma-separated list of tags (e.g., "music,tv,radio"), field_tags will be assigned an array of the individual tags (array('music', 'tv', 'radio')).

Subfields

When a destination field has options which can be set to control its behavior, or requires multiple pieces of data, these can be set using subfields. For example, to set a body and its summary:

    $this->addFieldMapping('body', 'body');
    $this->addFieldMapping('body:summary', 'excerpt');

The subfields are exposed on the migration detail pages just like regular fields, and all the power of field mappings can be applied to them.

For more complex fields (such as Name or Address), it might not be obvious what should be mapped to the "primary" field component. By default, the first subfield is used as the primary field, but this can be overridden by individual modules. Consult the specific module / field documentation in those cases.

sourceMigration

A key feature of Migrate is the ability to maintain relationships among imported objects, even when their primary keys have changed during import. For example, a user account may have had the unique ID 123 on the legacy system, but the Drupal user account created from that legacy account data gets the uid 456. The legacy system may thus have had an author ID of 123 on that account's content, which we want to store as a uid of 456 on the Drupal nodes. This is accomplished by using sourceMigration.

$this->addFieldMapping('uid', 'author_id')
     ->sourceMigration('Users');

What this is saying is that the author_id value (e.g., 123) should be used to query the Users migration's map table to retrieve the corresponding Drupal ID (456), which will then be assigned to the node uid field.

For some fields (e.g. taxonomy term references), you will also need to set the source_type argument for the mapping to succeed:

$this->addFieldMapping('field_term_ref', 'term_id')
     ->sourceMigration('OracleTags');
$this->addFieldMapping('field_term_ref:source_type')
     ->defaultValue('tid');

Multiple sourceMigrations for a single value

Migrate also supports the possibility that the Drupal Id for a given field may have been generated by any number of separate migrations. For example, the Author of the node in our example above could have been generated by our 'Users' migration, but it also might have been generated by an 'Authors' migration which also creates users.

In this case, you can simply pass an array to sourceMigration() like so:

$this->addFieldMapping('uid', 'author_id')
     ->sourceMigration(array('Users', 'Authors'));

Migrate will loop through the listed source migrations in the order given, and will stop at the first destination id that it finds.

Callbacks

Sometimes you need to perform some validation transformation on a single field. You could do this in prepareRow(), but the simplest thing to do is add a callback to the field. This isolates the manipulation from any more complex work going on in prepareRow(), lets you apply standard PHP functions without adding anything outside of the field mapping, and also lets you share a function among multiple fields.

As shown, each callback may be either a standalone function or an object and method. (These are passed directly as the first argument to call_user_func().)

  // Incoming text is ISO-8859; we want to convert it to UTF-8.
  $this->addFieldMapping('field_text_stuff', 'source_text')
    ->callbacks('utf8_encode');
  $this->addFieldMapping('field_computed_value', 'field_base_value')
    ->callbacks(array($this, 'computeValue'));
  // Pass more parameters to callbacks() to specify more callbacks.
  $this->addFieldMapping('field_email_address', 'source_email_address')
    ->callbacks('trim', array($this, 'validateEmailAddresses'));
  
  // ...
 }

  protected function computeValue($value) {
    $value = ($value * 10) + 28.3;
    return $value;
  }

  protected function validateEmailAddresses($value) {
    // If there may be multiple values in the source data for this field, $value
    // may be a scalar (single) value, or an array of values. Your code should
    // take this into account. Here's a simple way: cast $value as an array. If
    // $value is already an array, it will have no effect; otherwise it creates
    // an indexed array with $value as its only item.
    $value = (array) $value;

    // Now continue as if you had an array all along.
    return array_filter($value, function($address) {
      return filter_var($address, FILTER_VALIDATE_EMAIL);
    });
  }

Note that the callbacks are applied to the source $row field after prepareRow() is called.

Callbacks are applied after any sourceMigration(s) is/are applied as well. So if you need to manipulate a field's values based on the source IDs, you'll have to use prepareRow() instead of a callback to get at them before they are replaced with the destination's corresponding IDs.

Deduping

In some cases, Drupal may require a given field be unique for every object, but the source system may not have had the same requirement. You can automatically generate deduped values in these instances. In this example, while Drupal requires that usernames be unique, the source system we're coming from did not:

$this->addFieldMapping('name', 'username')
     ->dedupe('users', 'name');

The dedupe() arguments are the Drupal table and column to check for duplicates. When a migrate-import is run, if the first record has a username of 'mike', a Drupal user with name 'mike' is created. If later another record has a username of 'mike', the dedupe handling will query users.name and discover it already exists, so it will modify the name value to 'mike_1'. It will also save an informational message to the migration's message table, so you know when values have been modified (for example, you might use the message table to send emails to users letting them know their usernames have been changed).

Documenting mappings

The above methods cover all the methods that affect the actual operation of data import. There are a number of other methods and techniques, however, which aid in documenting your migration. If you have enabled the migrate_ui module, clicking on a migration name on the Migrate dashboard brings you to a page documenting the migration - the source, the destination, and especially the mappings. Regular review of this page can help ensure nothing gets missed.

Every field mapping has an Issue Group associated with it - on the migration information page, each group is rendered on a vertical tab named "Mapping: <group>". If you don't provide a group explicitly using issueGroup() (see below), it defaults to "Done".

Now, it's a rare migration where you use every available source field, and populate every available Drupal field directly from a source field. Usually some source data will simply be ignored, while some Drupal fields will be allowed to fallback to their defaults. You could simply not map these at all, but it is a good practice to make these decisions explicit - to clearly indicate that someone has consciously decided these fields will not be mapped. A helpful feature of migrate_ui is that it highlights on the Destination and Source tabs any fields that have no mappings - by explicitly marking fields that are not to be migrated, you can distinguish the fields you know you don't want to migrate from those you haven't dealt with yet, or new fields that have appeared and need to be dealt with. So, a good convention is to create NULL mappings within a group named "Do Not Migrate" (or "DNM", if you're a lazy typist).

// Unmapped destination fields
$this->addFieldMapping('status')
     ->issueGroup(t('DNM'));
$this->addFieldMapping('uid')
     ->issueGroup(t('DNM'));
// Unmapped source fields
$this->addFieldMapping(NULL, 'obsolete_data')
     ->issueGroup(t('DNM'));

Now, developing a complex migration process may take weeks or months - you are not going to know exactly what is being migrated and how when you initially write your migration class. Consider this example:

$this->issuePattern = 'http://drupal.org/node/:id:';
...
$this->addFieldMapping('field_ingredients')
     ->issueGroup(t('Client issues'))
     ->description(t('Where will the ingredient data come from?'))
     ->issuePriority(MigrateFieldMapping::ISSUE_PRIORITY_MEDIUM)
     ->issueNumber(770064);

The Migrate module was created in the context of consultants developing migrations for clients, so our usual convention was to have a "Client issues" group for anything the client needs to address (usually providing information on how or whether they want a field to be migrated) and an "Implementor issues" group for mappings which are fully specified but haven't been completely implemented yet. Note that descriptions (displayed on the migration detail page) can be added to any mapping, and are recommended for all but the most trivial cases. The issuePriority defaults to MigrateFieldMapping::ISSUE_PRIORITY_OK - if any other value, it will be highlighted on the detail page. The values for priorities are:

  • ISSUE_PRIORITY_OK
  • ISSUE_PRIORITY_LOW
  • ISSUE_PRIORITY_MEDIUM
  • ISSUE_PRIORITY_BLOCKER

Finally, note the issuePattern setting on the migration class and the issueNumber on the field mapping. This can be used to link to a ticket or issue in an issue-tracking system such as Unfuddle or Jira - the issueNumber is plugged into the :id: placeholder in the issuePattern to generate the link.

Removing field mappings

Usually, you will be implementing multiple Migration classes which share some common attributes (such as $this->issuePattern). To share such commonality, you would derive an intermediate abstract class directly from Migration, then derive each of your concrete classes from the intermediate class. In some cases, you may even be able to share a number of field mappings among multiple migrations by including them in the intermediate class. But, what if you have a mapping that applies to 9 out of your 10 migrations (e.g., you have a taxonomy reference field_tags on all but one content type)? What you can do is add the field mapping in the common constructor, then remove it in the one exception:

abstract class CommonMigration extends Migration {
  public function __construct() {
    parent::__construct();
...
    $this->addFieldMapping('field_tags', 'tags');
...
  }
}

class PageMigration extends CommonMigration {
  public function __construct() {
    parent::__construct();
...
  $this->removeFieldMapping('field_tags');
...
  }
}

Comments

jaimecalvo’s picture

I have a problem with migrating over a field.
I want it to be a boolesk, but i have male, female and the ones that haven't choosen.
I want it to not do anything if the gender isn't set, but all it does is trying to set it to an empty string and the 'row' doesn't get imported.
Anyone got an idea ?
Here is my code:

$this->addFieldMapping('field_gender', 'gender')
->defaultValue(NULL) // empty = unknown
->callbacks(array($this, 'fixGender'))
->description(t('Gender'));

public function fixGender($gender) {
$gender = trim(strtolower($gender));
if($gender == "female" ) return "0";
if($gender == "male" ) return "1";
return false;
}

jaimecalvo’s picture

i needed a prepareRow.

public function prepareRow($row) {
$gender = trim(strtolower($row->gender));
if(empty($gender)) unset($row->gender);

}

a.milkovsky’s picture

Can I map multiple fields into one using MigrateSourceSQL? I can do migration logic in prepareRow() but in migrate UI I will get message like:

"event_data" was used as source field in the "event_data" mapping but is not in list of source fields

justice4world’s picture

Hi,

I'm new to Drupal and have performed a D5 to D7 migration and everything migrated over just fine except for forum content. I found this code here:

$this->addFieldMapping('taxonomy_forums', 'objectid');
$this->addFieldMapping('taxonomy_forums:source_type')
     ->defaultValue('tid');

on a comment to this post here:

https://www.drupal.org/node/1609160

My question is, where in the migrate module should this code be inserted? Possibly in this include file here:

/modules/migrate/includes/field-mapping.inc

Or, somewhere else?

The errors I am receiving during the migration are:

for Nodeforum:

reset() expects parameter 1 to be array, null given File /forum.module, line 331

This is line 331 in forum.module:

$forum_terms[] = $term->tid;

and for Commentforum:

No node ID provided for comment

Thank you in advance. :)

schuffr’s picture

Hi all,

I am having a problem migrating address fields. Everything else in my migration works, but address field simply is blank. I have searched for many hours and havent found a definitive answer that works. I am using Drupal 7.37, Migrate 7.x-2.7, Address Field 7.x-1.1. I have set the default value for address to US and I am using the : subfield notation. Any ideas why this wouldn't work?

many thanks!

My migration class is as follows:

    <?php
    class DOCInfoMigration extends Migration {
      public function __construct($arguments) {
    
        parent::__construct($arguments);
    
        $this->description = t('Loads early profiles data to DOCInfo profile');
            
       	/********* Source *********/
       	// MySQL database as source
        $query = Database::getConnection('default', 'default')
                 ->select('DOCInfo', 'u')
                 ->fields('u', array('uid',
                                     'first',
                                     'last',
                                     'phone',
                                     'phonetype',
                                     'dob',
                                     'gender',
                                     'membertype',
                                     'year',
                                     'make',
                                     'model',
                                     'new_used',
                                     'tshirt_it',
                                     'street',
                                     'apt',
                                     'city',
                                     'state',
                                     'zipcode'));
                 
        $this->source = new MigrateSourceSQL($query);
    
    		$this->destination = new MigrateDestinationProfile2('doc_info'); // use machine name of profile
    
    		/*********** Map **********/
    		// Create a "map" which is used to translate primary keys*/
        $this->map = new MigrateSQLMap($this->machineName,
          array(
            'uid' => array(
              'type' => 'int',
              'alias'=> 'u'
            ),
            ),
          MigrateDestinationProfile2::getKeySchema()      
        );
    
        /*********** Connect DOCInfo to user **********/
        $this->addFieldMapping('uid', 'uid');
    #         ->sourceMigration('DOCInfo')  // If user migration class was named 'MyUserMigration', the string is 'MyUser'
    #         ->description(t('The assignment of DOCInfo source data to the respective DOCInfo fields'));
    
        /******* Field mappings ******/
        $this->addFieldMapping('language')->defaultValue('en');
        $this->addFieldMapping('field_fname','first');
        $this->addFieldMapping('field_fname:language')->defaultValue('en');
        
        $this->addFieldMapping('field_lname','last');
        $this->addFieldMapping('field_lname:language')->defaultValue('en');
    
        $this->addFieldMapping('field_home_phone','phone');
        $this->addFieldMapping('field_home_phone:language')->defaultValue('en');
        $this->addFieldMapping('field_phone_type','phonetype');
    
        $this->addFieldMapping('field_dob','dob');
        $this->addFieldMapping('field_doc_gender','gender');
    
        $this->addFieldMapping('field_doctype','membertype');
        $this->addFieldMapping('field_docbikeyear','year');
        $this->addFieldMapping('field_docmake','make');
        $this->addFieldMapping('field_docmodel','model');
        $this->addFieldMapping('field_docmodel:language')->defaultValue('en');
        $this->addFieldMapping('field_docnewused','new_used');
        $this->addFieldMapping('field_italian_t_shirt','tshirt_it');
    
    
        $this->addFieldMapping('field_address')->defaultValue('US');
        $this->addFieldMapping('field_address:thoroughfare','street');
        $this->addFieldMapping('field_address:premise','apt');
        $this->addFieldMapping('field_address:locality','city');
        $this->addFieldMapping('field_address:administrative_area','state');
        $this->addFieldMapping('field_address:postal_code','zipcode');
    
    
        /*** Unmapped destination fields ***/
        $this->addUnmigratedDestinations(array('revision_uid',
    #                                           'field_address',
                                               'field_address:sub_administrative_area',
                                               'field_address:dependent_locality',
                                               'field_address:sub_premise',
                                               'field_address:organisation_name',
                                               'field_address:name_line',
                                               'field_address:first_name',
                                               'field_address:last_name',
                                               'field_address:data',
                                               'field_dob:timezone',
                                               'field_dob:rrule',
                                               'field_dob:to',));
      }
    } 
stevesmename’s picture

give this a try, this is how I had recently implemented it. Please report back what worked.

  /* you may just need this */
  function prepareRow($row) {
    $row->country = array("US");
  }

  /* otherwise try it this way */
  $arguments = array(
       'thoroughfare' => array('source_field' => 'street'),
       'locality' => array('source_field' => 'city'),
       'premise' => array('source_field' => 'apt'),
       'administrative_area' => array('source_field' => 'state'),
       'postal_code' => array('source_field' => 'zipcode'),
    );

    $this->addFieldMapping('field_address', 'country')
          ->arguments($arguments);


jag1500’s picture

Hello everyone, Really need your opinion and help on this. I installed the location module to do a location search within certain miles of a zipcode. I also want users to be able to select a city however the city field that comes with locations does not have autocomplete option. I have Restaurant content type and I am using the zipcode field from location as CCk field and also have my own CCK field using Taxonomy to list cities with autocomplete option and street CCK field. The city field lists city as City, State (Portland, OR) with autocomplete.

Can i use the mapping function addFieldMapping to map my own street and city cck field to location module's street and city fields when the node is submitted or saved? (I may have to split the city from the comma and insert the first part in city and second part in state). If so, how would the code look like and where would i insert the code? Thank you very much for your help in advance.