I'm importing and updating records into nodes. Fields whose values are completely removed in an update do not get removed from the destination node. My "system of record" is the default, Migration::SOURCE, so the node should have the same data in each field as the source--even if that data is null.

This happens because when a node is saved with some fields absent, the existing values for those fields are not removed from the node. If there is no source data (or the source data is NULL) for a particular field mapping, Migrate doesn't set any value for the field, so Drupal uses the existing values. This prevents a migration from removing values from previously set fields.

There's code in Migration::applyMappings() that suggests that it will set the destination value to NULL if a source value isn't set, but I think that is checks for the wrong system of record--this code only runs if the system of record is "destination", which doesn't make sense if the destination values are supposed to override the source values.

There is corresponding code in MigrateDestinationNode::import() that allows explicit NULL source values to override the destination values when the system of record is "destination".

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

becw’s picture

Status: Active » Needs review
FileSize
1.16 KB

Here's a patch that sets empty source fields to null in the destination if the system of record is Migration::SOURCE, and otherwise only sets explicitly null-valued source fields to null.

becw’s picture

Title: Updating records with null values » Updating records with null values (patch)
becw’s picture

The issue with this patch is that most migrate destination plugins treat NULL as a field value, meaning you get empty text field values stored where there should be no field value stored at all.

becw’s picture

Since the patch in #1 resulted in empty fields on nodes, here's another approach where MigrateDestinationNode::import() sets missing fields to NULL if the "system of record" ($migration->getSystemOfRecord()) is Migration::SOURCE--this means that any fields that are NULL or not present in the source are set to an empty field in the destination node, so that the destination node contains the exact same data as the source.

This only solves the issue when migrating into nodes; if the issue exists for taxonomy terms, users, or other entity destinations (because it's specific to fieldable entities), then this patch would need to be extended to address those cases.

mikeryan’s picture

Status: Needs review » Postponed

Thanks for the patch, that does seem to make sense. I'm going to postpone for now, however - I don't have time to extend it to the other entities, and would rather not commit only the node support for 2.3 (thus having the destination plugins behave inconsistently, not to mention the D6 version). After 2.3 is finalized, we can try to complete this for 2.4.

mikeryan’s picture

Status: Postponed » Active
aidanlis’s picture

I'm confused as to what the current / desired behaviour is.

What I'm seeing is: Destination fields that are NULL mapped (via addUnmigratedDestinations), end up blank after a migration even when the destination node originally had data. That's definitely not the behaviour that I want ... has a patch for this already gone through?

aidanlis’s picture

Ignore my comment, I was confused about the systemOfRecord.

mikeryan’s picture

Status: Active » Needs work
shmox’s picture

Hi everybody, i am importing my own (self made) site to drupal 7...
I have a node email field (from email module), the problem is when source email field is empty (null), the field is imported with blank value, that caused problem when clicking on the email field (front office) which leads to not found page (404) because the field was imported and it's value is empty.

So what i want to know is : is there a way to import field conditionnaly so that if the source field value is empty the field is not imported at all istead of importing it with empty value ?

Thanks and sorry for my bad english.

PS : I'm new to drupal world ,sorry if my question is a beginer one.
PS: i've also tried the two patchs above which did not solve the problem
========================================================
EDIT:
I've foud a walkaround by unsetting the email field (when source email field is null) in prepare function :-D

smichel’s picture

I'm having what looks to me like this issue, but with users.

I have a number of text fields on the user entity, added via the field api, saved to features, etc. I'm reading values from the profile tables in Drupal 5.

I have mappings set up for fields, like:

$this->addFieldMapping('field_profile_first_name', 'profile_first_name');
$this->addFieldMapping('field_profile_last_name', 'profile_last_name');

... etc. There are a couple dozen of them. In prepareRow, I add, say, the profile_first_name field to the $row item. At the end of prepareRow, I print_r($row) and only see items I've created. Say, if there was no profile record for a user's first name, then it doesn't show in the dump of $row.

But when the record was created, I still see rows in the field_data_field_profile_first_name table, where the value column is empty.

If I dump $entity in prepare(), it shows this for that first_name field:

[field_profile_first_name] => Array
        (
            [und] => Array
                (
                    [0] => Array
                        (
                            [value_format] => plain_text
                            [format] => plain_text
                            [value] => 
                        )

                )

        )

I'm running the 7.x-2.4-beta1+8-dev of migrate. It's doing this for all text fields. Other fields, such as date fields, link fields, entity_references, etc. don't get the extra records.

mikeryan’s picture

@smichel - what you observe is unrelated to this issue, this issue is about updating fields that have values failing to clear those values when they're cleared on the source. If the empty fields are causing you difficulty, please open a separate issue.

smichel’s picture

Thanks. I realized my issue isn't about updating at all, it's about creating new users. I'll dig into it over the next couple days.

pifagor’s picture

Issue summary: View changes
Status: Needs work » Closed (outdated)