Posted by RaF on March 17, 2011 at 2:57pm
8 followers
Jump to:
| Project: | Migrate |
| Version: | 7.x-2.x-dev |
| Component: | Code |
| Category: | feature request |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | active |
Issue Summary
Migrate comes a long with a few great examples. They helped me enormously.
Unfortunately, there's nowhere information to be found on the process of creating stubs. Any chance someone can copy/paste some code on how this actually works? Haven't been able to wrap my head around it yet. Could also be used to fill up the documentation page on http://drupal.org/node/1013506.
Comments
#1
I second that motion! I read about it here: http://drupal.org/node/915102 but then when I went to do it, I realized I had no idea how to actually go about writing the code for it. An example of this would be greatly appreciated.
#2
This is FAQ. We need some docs on it.
I could have sworn that stub nodes were created automatically by the nodereference integration in migrate module. I don't see that code anymore. Instead, there is an API where migrations can implement stub support. Start at Migration::handleSourceMigration method.
However, usually you don't need any stub nodes. If you are migrating music albums and tracks, you should simply import your albums and in a subsequent migration, import your tracks. That way, assuming the node reference points from track to album, your referenced nodes are guaranteed to already exist. Stub nodes come into play when you have a self-reference like when albums have a node reference called 'related albums'. You can't easily assure that all the related albums exist by the time you create a given album.
Hope this helps.
#3
This is quite similar to my situation!
Thank you for pointing out Migration::handleSourceMigration. Unfortunately, it is not clear to me how to use it to create stub node(s). Are there plans to put an example in the documentation?
#4
Yes, we will be filling out the documentation over time. The missing piece here is that you need to define a createStub() method in the destination migration (i.e., the migration corresponding to the sourceMigration() argument), to create the stub node and return its nid.
<?php/**
* Create a stub node for a so-far-unresolved node reference.
*/
protected function createStub() {
migrate_instrument_start('create stub');
$node = new stdClass;
$node->title = t('Stub');
$node->body = t('Stub body');
$node->type = $this->destination->getBundle();
// Default to admin account, unpublished
$node->uid = 1;
$node->status = 0;
node_save($node);
migrate_instrument_stop('create stub');
if (isset($node->nid)) {
return array($node->nid);
}
else {
return FALSE;
}
}
?>
#5
Perfect. This is exactly what I was looking for. Many thanks!
#6
Awesome, thanks, it works! If it's OK by you, I will fill up the documentation a bit (once I find the time).
1 more question though (hope this is the correct place - it's about stubs).
I'm importing a node reference field which can reference nodes of various content types (TypeX, TypeY, TypeZ). See code below, which works (XML source btw - sourceid's of the to-reference nodes are XML attributes).
<?php$this->addFieldMapping('field_related_nodes', 'RelatedNodes')
->sourceMigration(array('TypeXNode', 'TypeYNode', 'TypeZNode'));
?>
<?phppublic function prepareRow(&$current_row) {
if(!empty($current_row->xml->RelatedNodes)) {
$related_nodes= array();
foreach($current_row->xml->RelatedNodes->RelatedNode as $key => $value) {
$attributes = $value->attributes();
$related_nodes[] = (string) $attributes['Id'];
}
$current_row->RelatedNodes = $related_nodes;
}
}
?>
Stubs for nodes that haven't been imported yet are being created. The problem I'm currently facing is that the stubs are always of type 'TypeX' (handled by TypeXNode). How do you define wether the stub-create function in TypeXNode, TypeYNode or TypeZNode should be used?
Been digging in the code for some time now, no luck so far. Not giving up though!
#7
Looks like a bug to me. The Stub system has no way to know which source_migration won. From handleSourceMigration():
if ($destids = $source_migration->createStubWrapper(array($source_key), $migration)) {Would be good if Mike confirmed this.
#8
It's true, the handling of multiple sourceMigrations will call the first sourceMigration's createStubWrapper when the ID is not found in any of the map tables. I don't see how handleSourceMigration() can choose in a general way - presumably there is something in the outer migration's source query that indicates what destination type (and thus what sourceMigration) to use, but it's very likely that the value (if it is indeed a single field) is not the name of the desired sourceMigration.
So, how do we get the necessary information into handleSourceMigration? Perhaps the field mapping sourceMigration() method could have an optional second argument, $stub_migration_field - if present, then the value in the source row field of that name would be taken by handleSourceMigration as the name of the migration to call for creating stubs. With clever SQL you may be able to map your source data values into migration names, but more likely you would use prepareRow() to translate into that field. Let's see how this might look in practice:
<?php
...
$this->addFieldMapping('field_related_nodes', 'RelatedNodes')
->sourceMigration(array('TypeXNode', 'TypeYNode', 'TypeZNode'), 'RelatedTypes');
...
public function prepareRow($current_row) {
$type_mappings = array(
'type_x' => 'TypeXNode',
'type_y' => 'TypeYNode',
'type_z' => 'TypeZNode',
);
if (!empty($current_row->xml->RelatedNodes)) {
$related_nodes = array();
$related_types = array();
foreach ($current_row->xml->RelatedNodes->RelatedNode as $key => $value) {
$attributes = $value->attributes();
$related_nodes[] = (string)$attributes['Id'];
$related_types[] = $type_mappings[(string)$attributes['Type']];
}
$current_row->RelatedNodes = $related_nodes;
$current_row->RelatedTypes = $related_types;
}
}
?>
Thoughts?
#9
Looks good to me!
#10
To solve this problem, we filtered the source migrations based on the node type by implementing handleSourceMigration in our abstract migration class. We store the source node type in $this->sourceNodeType on each migration.
<?php
/**
* Look up a value migrated in another migration.
*
* @param mixed $source_migrations
* An array of multiple source migrations, or string for a single migration.
* @param mixed $source_values
* An array of source values, or string for a single value.
* @param mixed $default
* The default value, if no ID was found.
*/
protected function handleSourceMigration($source_migrations, $source_values, $default = NULL) {
// Filter out $source_migrations based on the source node type. Otherwise we
// end up with stubs being created with an incorrect node type. To do this,
// first query the source database for the node type.
$query = self::getSourceConnection()
->select('node', 'n')
->fields('n', array('type'))
->condition('n.nid', $source_values);
$result = $query->execute();
if (($source_node_type = $result->fetchField())
&& $source_migration = self::migrationFromSourceNodeType($source_node_type, (array) $source_migrations))
{
// If we have a node type, and that node type has an associated migration,
// set the $source_migrations to that one migration.
$source_migrations = array($source_migration);
}
return parent::handleSourceMigration($source_migrations, $source_values, $default);
}
/**
* Return the migration associated with a source node type.
*
* @param string $source_node_type
* The source node type.
* @param array $migrations
* An array of migration machine names to check. If empty, all migrations will
* be checked. Not recommended for performance reasons.
* @return mixed
* The string machine name of the migration, or FALSE if none was found.
*/
public static function migrationFromSourceNodeType($source_node_type, $migrations = NULL) {
static $map;
if (!isset($map[$source_node_type])) {
migrate_instrument_start(__FUNCTION__);
if (!isset($migrations)) {
$migrations = array_keys(migrate_migrations());
}
foreach ($migrations as $machine_name) {
$migration = MigrationBase::getInstance($machine_name);
if (!empty($migration->sourceNodeType)) {
$map[$migration->sourceNodeType] = $machine_name;
}
}
migrate_instrument_stop(__FUNCTION__);
}
return isset($map[$source_node_type]) ? $map[$source_node_type] : FALSE;
}
?>
#11
Hello everyone,
sorry for reopening this thread. I'm a brand newbie of Drupal AND Migrate and I've to say that these are really great softwares. I don't know if what I'm trying to do is possible but I thought it's a good place to post a question about it.
I'm dealing with a table in a database that references itself (e.g. a table of individuals that may reference other individuals). Till there, it's nothing new, and it has been covered widely here and there. Stubs do the job, and do it quite well, as long as records reference OTHER records in the table, but not THEMSELVES. As an example of use case, individuals may be insured by other individuals, or may be their own insurants.
I've noticed that with my naive implementation of a stub (taking simply the code sample given above or in the chicken and eggs page), self-referencing nodes usually do not reference themselves, but create a dummy stub that stay in the Drupal database (one dummy stub being created per self-referencing node). There is an exception to that though : it's when the self referencing node was already referenced by a previous node and doesn't need to create its stub. In that case, the self-referencing node seems to be built correctly.
I know that my use case may not very common and may seem a bit contrived, but if anyone has a solution for this, I would be very glad to hear about it.
Many thanks to everybody
#12
#13
#14