Hi,
I read that the migrate module can map a unique key from the source to a unique key on the destination database, and will remember these mappings to correctly map foreign keys in the future.
However, I am still unclear how exactly this works.

Assuming you have a source table A with primary key A.id.
The destination table is B with primary key B.id (auto-increment).

At the time I start the migration, table B has only 4 rows with id = 3 .. 6, and the auto_increment counter remembers 7 as the last insert id. Yep, that's not identical with the count, because 1, 2 and 7 have been deleted already (just to make it more interesting).
Source table A contains 50 rows, with id = 1 .. 50. All of these are to be migrated to B.

The migrate logic looks at row A[1], and sees that the id is 1.
Now it could follow one of two strategies:

  1. Check if the slot at B[1] is free. Yeah, it is! So, save as B[1]. This time, we have A.id and B.id match up, at least for this one row. This continues to go well for A[2] -> B[2]. Then we get to A[3], which clashes with the existing row at B[3]. So for this row, we let auto_increment do the job: A[3] -> B[8]. Then A[4] -> B[9], etc. For those cases where we used auto_increment, we have to remember the mapping.
  2. Don't care about B[1], and always do auto-increment. So, we get A[1] -> B[8], A[2] -> B[9], etc. No match-up of A.id and B.id. Still, the associations are remembered in the mappings table.

Honestly, option (1) scares me like hell. This could result in 100 record ids being matched up 1:1, and then the 101st has a mismatch. In the meantime, one developer got so used to those two the ids matching up 1:1, that some code depends on it. Bad, bad, bad! I'd prefer if they all mismatch from the beginning.

Option (2) is much better for my taste.

-----------

I tried to find out, what is the default strategy that migrate module uses, or how I can influence that strategy.
At some point I will probably need to write sth like this:
$this->addFieldMapping('id', 'id');
But, some of the docs can be understood like this would activate option (1).
What happens if I don't write the above? Will this still fill the mapping tables?

Comments

donquixote’s picture

donquixote commented
Issue summary: View changes

associations still remembered in case (2).

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Pittsfield, MA, USA
commented
Status: Active » Fixed

The migrate module calls the normal Drupal APIs such as node_save() to store the destination objects - these produce the destination IDs in the normal way, and Migrate saves those in the map table. You do not need to (and indeed should not) make an explicit mapping of old ID to new ID, this just works.

To maintain the original key as the Drupal ID requires support on the destination side - in Drupal 7, nodes and users support this. Here's an example from migrate_example's beer.inc:

    // To maintain node identities between the old and new systems (i.e., have
    // the same unique IDs), map the ID column from the old system to nid and
    // set is_new to TRUE.
    $this->addFieldMapping('nid', 'bid')
         ->description(t('Preserve old beer ID as nid in Drupal'));
    $this->addFieldMapping('is_new')
         ->defaultValue(TRUE);
donquixote’s picture

donquixote commented

Ah, this is useful to know!

So, Drupal already knows that B.id is the primary key of B, and that it is auto-increment.
Then I tell MigrateSQLMap which is the primary key of A, via constructor argument.

Finally, no need to mention A.id nor B.id with addFieldMapping.
This makes me feel a lot better :)

donquixote’s picture

donquixote commented
Status: Fixed » Needs work

Hm, one more question :)

What if two of my migrations write to the same entity?
Example: Migration A creates user entities.
Migration B updates those user entities with additional information, such as, group memberships.

Ok, in this special case I'd probably rather create a new destination type plugin for "group membership".
But in general, how do I build this second migration? How do I map the primary keys?

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Pittsfield, MA, USA
commented
Status: Needs work » Fixed

Migration B, since it wants to update the users without affecting any data in the user record other than the specific fields being changes, needs to set $this->systemOfRecord = Migration::DESTINATION;. Assuming it's querying the same source keys as were used in Migration A, you would have an explicit mapping of the keys, with ->sourceMigration('MigrationA'). See WineUserUpdatesMigration in wine.inc for a concrete example.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

Anonymous’s picture

(not verified) commented
Issue summary: View changes

clarify a bit, about why i dislike option (1).