Hi,

When i migrate a bunch of nodes that optionally reference eachother, all goes well until i migrate nodes that *don't* have the reference and try to index them with Search API. In my Migration, i have defined a fieldMapping as such:

$ref_ca_occ_mapping = $this->addFieldMapping('field_ca_occ_ref_ca_entity', 'ref_ca_entity');
$ref_ca_occ_mapping->sourceMigration(array($source));
$this->dependencies = $this->dependencies + array($source);

Since the reference is optional, sometimes the Source will not specify the 'ref_ca_entity' field. I have tried all kinds of things: feeding it an empty array, passing NULL or FALSE, removing the field etc, but for some reason I can't get it to work. The reference field on the imported node always looks like this:

field_ca_object_ref_ca_entity
   und (Array, 1 element)
      0 (Array, 1 element)
         target_id (NULL)

The target_id => NULL causes all kinds of EntityMalformedExceptions, hinders indexing by Solr and causes this kind of warnings:
Warning: array_flip() [function.array-flip]: Can only flip STRING and INTEGER values! in DrupalDefaultEntityController->load()

The only way i got around this is by patching the MigrateEntityReferenceFieldHandler. This is a minor patch which simply avoids adding NULL as a target_id. I'm sorry if this is not the right place to do this, and i am open to suggestions. However, i think this patch can't do any harm and isn't obtrusive at all, so i'd like to get it in.

Patch follows in next post

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

thimothoeye’s picture

thimothoeye’s picture

Assigned: thimothoeye » Unassigned
Status: Active » Needs review

Forget to set this to needs review...

Damien Tournoud’s picture

Title: Entityreference + Migrate + Search API + Solr: empty reference fields break indexing » Handling NULL values during the migration of Entity Reference fields
Project: Entity reference » Migrate
Version: 7.x-1.x-dev » 7.x-2.x-dev
Status: Needs review » Active

There are a couple of related issues in there.

One of them is that Entity Reference defines the target_id column as nullable by default. I just opened an issue to consider changing this: #1569144: Consider making target_id NOT NULL.

The second issue is that Migrate doesn't seem to deal well with non-defined values. During the mapping, Migrate does:

// If there's a source mapping, and a source value in the data row, copy
// to the destination
if ($source && property_exists($this->sourceValues, $source)) {
  $destination_values = $this->sourceValues->$source;
}
// Otherwise, apply the default value (if any)
elseif (!is_null($default)) {
  $destination_values = $default;
}

Which means that if there is a value in the data row, even NULL, it will be considered.

I'm not sure that modifying the Field handler of Entity Reference is the way to go. I looked at the fields implemented in Migrate, and most of them do *not* filter out empty values (except the node reference and user reference ones, which means that you cannot reference the anonymous user in an user reference field!).

I'm moving this to the Migrate queue for advice on how to proceed.

mikeryan’s picture

Project: Migrate » Entity reference
Version: 7.x-2.x-dev » 7.x-1.x-dev
Status: Active » Needs work

I think the patch is the right approach (although it should use !is_null rather than != NULL). Or, you could echo the node and user reference handlers and derive from MigrateSimpleFieldHandler, setting skip_empty to TRUE.

At mapping time, Migrate can't know whether NULL values make sense for the given field - that's up to the field handler. Interesting point about referencing the anonymous user, I think the skip_empty check should allow empty strings and zeros through, it should only filter out NULLs, I'll open an issue for that.

Damien Tournoud’s picture

Status: Needs work » Fixed
FileSize
1.36 KB

Ok, I committed the following.

thimothoeye’s picture

Guys, thanks for looking into this so quickly!

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

adrupaler’s picture

I have the exact same problem. at first everything worked fine but after 3 test importing , the migrate module started to show this error even when my entity reference has value!

<?php
An AJAX HTTP error occurred. HTTP Result Code: 200 Debugging information follows. Path: /drupal/batch?id=340&op=do StatusText: OK ResponseText: stdClass Object ( [nid] => 9673 [vid] => 9673 [language] => und [title] => اخترشناسی [uid] => 16 [status] => 1 [created] => 1353915907 [changed] => 1365242715 [comment] => 0 [promote] => 1 [sticky] => 0 [tnid] => 0 [translate] => 0 [field_author_target_id] => 3405 ) {"status":true,"percentage":"100","message":"SQLSTATE[23000]: Integrity constraint violation: 1048 Column \u0026#039;field_translator_target_id\u0026#039; cannot be null: INSERT INTO {field_data_field_translator} (entity_type, entity_id, revision_id, bundle, delta, language, field_translator_target_id) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5, :db_insert_placeholder_6_target_id); Array\n(\n [:db_insert_placeholder_0] =\u0026gt; node\n [:db_insert_placeholder_1] =\u0026gt; 5562\n [:db_insert_placeholder_2] =\u0026gt; 5562\n [:db_insert_placeholder_3] =\u0026gt; product_display\n [:db_insert_placeholder_4] =\u0026gt; 0\n [:db_insert_placeholder_5] =\u0026gt; und\n [:db_insert_placeholder_6_target_id] =\u0026gt; \n)\n (C:\\wamp\\www\\drupal\\modules\\field\\modules\\field_sql_storage\\field_sql_storage.module:451)\u003Cbr \/\u003EProcessed 1 (0 created, 0 updated, 1 failed, 0 ignored) in 0.8 sec (72\/min) - done with \u0027ProductDisplay\u0027\u003Cbr \/\u003E"}

?>

any body can help me why the entityreferencefieldhandler behave like this? also I modified the prepare() function to make sure the value isn't empty but nothing changed.

<?php
if($value != ''){
//if($value != null){
//if($value != NULL){
//if(!empty($value)){
       $return[$language][$delta]['target_id'] = $value;
        $delta++;
}
?>