Last updated December 8, 2011. Created by mikeryan on April 5, 2011.
Edited by edb. Log in to edit this page.
While the Migrate module was originally developed for the purpose of performing complete site migrations, where the source data is always the system of record, it can be used for other scenarios. For example, you might define a migration process for the purpose of updating only specific fields on existing content with new data, or after initially performing a full migration of new content you might want to remigrate the legacy data while preserving some changes you've made on the destination side. To control this behavior, the Migration class supports setting the system of record for the migration.
<?php
/**
* Indicate whether the primary system of record for this migration is the
* source, or the destination (Drupal). In the source case, migration of
* an existing object will completely replace the Drupal object with data from
* the source side. In the destination case, the existing Drupal object will
* be loaded, then changes from the source applied; also, rollback will not be
* supported.
*
* @var int
*/
const SOURCE = 1;
const DESTINATION = 2;
protected $systemOfRecord = Migration::SOURCE;
?>So, by default the system of record is the source data - if you remigrate existing data (e.g., with --update on your drush migrate-import command), the Drupal content is completely replaced by the content from the source. However, in the constructor for your migration you can set
<?php
$this->systemOfRecord = Migration::DESTINATION;
?>If you do this, then when you run drush migrate-import only the fields that you have mapped in the migration will be replaced on the Drupal side - all other fields will retain their existing values.
So, when you are defining a migration whose purpose is updating existing content, just set systemOfRecord and add field mappings for the specific fields you want to update, leaving the rest unmapped.
If you are updating content from a previous migration you must map the NID to the primary key it relates to and tell Migrate what the initial migration was called so it knows what node to update.
<?php
$this->addFieldMapping('nid', 'key')
->sourceMigration('Beer');
?>If you'd like to run an initial migration to create content from an external source, and also be able to run subsequent migrations that update existing content but preserve some Drupal-side changes, you need to define separate Migration classes for the latter purpose with systemOfRecord set to Migration::DESTINATION.
Comments
Important distinction between
Important distinction between System of Record and use of the HighWater field.
System of Record (Destination) is typically used when updating specific fields for previously imported content. Example: You've made changes to a CCK Select list and want to update only that changed field for all of your content, but not erase anything you've previously imported.
HighWater You're performing an incremental migration and you've just been given 200 new articles that weren't present when you initially performed the migration. You'd specify a 'highwater' field (typically a date reflecting time changed, i.e node changed) to only update content that is = or > the value in your 'highwater' field.
I have data coming from two
I have data coming from two different sources. I need to run one source at node create and other to be import at node update. migrate provide to map nids with some sources. but i have same Id in both sources. I dont want to map nids with source id but I have mapped source id in node create import in title. now I want to map title(instead of nids) with source id on node update import.
Does some one have ideas?
Add extra info from second source
If you need to update the information you have added from a migration already with new data from a second migration, you will need to set the system of record as above:
<?php$this->systemOfRecord = Migration::DESTINATION;
?>
Adding to this, you need to define the primary key you are using and the migration it relates to. For instance, if you are updating a node item:
<?php$this->addFieldMapping('nid', 'someKey')
->sourceMigration('FirstMigration');
?>
Where FirstMigration is obviously the machine name of the migration you ran to get the data in the first time round.
Hmm, interesting. So you can
Hmm, interesting. So you can have one migration piggyback off a secondary migration? That is pretty darn cool! How do rollbacks work in such a scenario?
yeah, very cool! Looks like
yeah, very cool! Looks like you can't roll it back, see comments above:
Kind of makes sense. In a roll back you are deleting previously added records. If you would want to roll back updated records you would have to keep track of the previous values. Maybe doable with using revisions ...
actually, in a
actually, in a (systemofrecord=DESTINATION) update migration it will still build a map table and keep track of which records have been updated (or processed). When you do a rollback it just removes those records (or the last x records).
For example, do a UserUpdate migration to set the status to 0 (block a user).
Import the first 10 users to set status to 0.
Now change your script to set status to 1 and roll back and import 5 users.
The last 5 users now get changed to status 1 and the first 5 are still 0.