Migration from references module

I've written some code that replaces the references fields by entityreference fields. First tests were successful but I bet there are unknown issues ;)
To test it place the code in an update hook of a module.
If the approach is verified I'd be happy to integrate it into entityreference itself.

@das-peter,
Thanks! I think that instead of hook_updae_N() we should have it as PHP file and people can execute it with drush (drush scr references_migration.php)

Maybe something like this?

@Amitaibu: That's indeed a good idea. I initially even thought we could add this into the module itself on an admin page or whatever. But I guess your approach is the nicer place for such code.
Now the question is if Damien is happy with the way the migration works. :)

Maybe worth a different issue, but we should also think if there's a way to help people migrate their views as-well?

That's a good idea - unfortunately I've no clue how to do that :)

I think we should have something a little bit more user friendly if we want to be able to completely deprecate the References module.

I'm thinking about a UI that allow you to execute this process, and only shows while there are fields from the References module.

Migrating Views is a nice idea. It's going to be mostly renaming fields, arguments and filters, so it should not be that hard.

@Damien: I'm absolutely willing to help creating such an UI, but I'd prefer to have confirmed the approach I chose for the migration first. If the approach / code is reviewed and confirmed all the other stuff is decoration. :)

This code works for me. It didn't select the correct widget for the fields in use, but that didn't bother me.

I feel better about exposing this code as a drush command, and making it available to invoke as a function in a hook_update_N implementation (or the like). So that's what's attached!

A little of topic, but not really -- In OG7 we have og-migrate module, that allows defining migration plugins (via CTools). I wonder if it wouldn't make sense to have a generic "module-migrate" module, that allows any other module to declare it's migrations.

I have been working on this as well and to try and tried to split it up so people can migrate certain fields at a time if need be (I haven't figured out the views migration yet). I have placed the code so far at http://drupal.org/sandbox/btmash/1404542. It could use a fair level of cleanup but it mostly consists of what was already in here. I've added in a web ui so the user gets a form to select which fields they wish to migrate (should probably include a 'check all' in there), and added in the migration bit for the field formatters to map over. Though if we go over via a CTools route, I'm happy to help in that way too.

Just wondering if more has been done with migrating things from References. We are currently using References and would like to move over to this module in order to keep up with development.

@rbruhn, see #12. The actual fields conversion is pretty much there. I need more info on completing the views migration (as none of my sites do complicated things with the views and references so I was able to fill out their mapping).

@BTMash
Tried out your migration script and it seems to work well for node references.
The only issue I had was with a User references. My field named "contributor" is a User Reference, and while trying to migrate my Views it was not showing up right. When I viewed the Content type the field is associated with, I saw Contributor was indeed an Entity Reference now, but the Target was set as a Node instead of User.

I added the target type in your function then ran the migration and it appeared to work alright

function _references_migrate_references_field_to_entityreference_field_create_field($field_info) {
//..................

$entityreference_field = array(
    'field_name' => $field_name,
    'type' => 'entityreference',
    'module' => 'entityreference',
    'entity_types' => array(),
    'foreign keys' => array(),
    'indexes' => array(
      'target_entity' => array(
        '0' => 'target_id',
      ),
    ),
    'settings' => array(
      'handler' => 'base',
      'handler_settings' => array (
        'target_bundles' => $target_bundles,
      ),
      'handler_submit' => 'Change handler',
      'target_type' => (($field_info['type'] == 'node_reference') ? 'node' : 'user'), //Added this line
    ),
//.....................

I've got a project with a few dozen node reference fields migrated from D6 so I'm looking forward to this project and migrating to entity references.

I downloaded the sandbox module at #12 and enabled and tried to run on one field, ex field_author_organization. On the batch API screen, I get the following error:

An AJAX HTTP error occurred. HTTP Result Code: 500 Debugging information follows. Path: /batch?id=32&op=do StatusText: Service unavailable (with message) ResponseText: PDOException: SQLSTATE[42000]: Syntax error or access violation: 1072 Key column 'field_author_organization_target_type' doesn't exist in table: CREATE TABLE {field_data_field_author_organization} ( `entity_type` VARCHAR(128) NOT NULL DEFAULT '' COMMENT 'The entity type this data is attached to', `bundle` VARCHAR(128) NOT NULL DEFAULT '' COMMENT 'The field instance bundle to which this row belongs, used when deleting a field instance', `deleted` TINYINT NOT NULL DEFAULT 0 COMMENT 'A boolean indicating whether this data item has been deleted', `entity_id` INT unsigned NOT NULL COMMENT 'The entity id this data is attached to', `revision_id` INT unsigned NULL DEFAULT NULL COMMENT 'The entity revision id this data is attached to, or NULL if the entity type is not versioned', `language` VARCHAR(32) NOT NULL DEFAULT '' COMMENT 'The language for this data item.', `delta` INT unsigned NOT NULL COMMENT 'The sequence number for this data item, used for multi-value fields', `field_author_organization_target_id` INT unsigned NULL DEFAULT NULL COMMENT 'The id of the target entity', PRIMARY KEY (`entity_type`, `entity_id`, `deleted`, `delta`, `language`), INDEX `entity_type` (`entity_type`), INDEX `bundle` (`bundle`), INDEX `deleted` (`deleted`), INDEX `entity_id` (`entity_id`), INDEX `revision_id` (`revision_id`), INDEX `language` (`language`), INDEX `field_author_organization_target_entity` (`field_author_organization_target_type`, `field_author_organization_target_id`), INDEX `field_author_organization_target_id` (`field_author_organization_target_id`) ) ENGINE = InnoDB DEFAULT CHARACTER SET utf8 COMMENT 'Data storage for field 69 (field_author_organization)'; Array ( ) in db_create_table() (line 2688 of <siteroot>/includes/database/database.inc).

and a message on the top saying 'proceed to the error page'. This link takes me back to the migrate references interface with a message that the 'migration was successful' with my field now no longer in the list (haven't checked into the data yet.. )

Still debugging this and trying to figure out where its going wrong but wanted to report on status.

Belay that last order ;) Just found the updated thread on this issue here: "target_type column was removed from entity reference" -> http://drupal.org/node/1441636

Patched files and can now confirm that this module is working for me on 40+ node reference fields.

Its strange that you had to apply the patch since I had committed it a few weeks ago with @martysteer's patch. I"ll take a look and see what is happening.

@rbruhn, I'll also add in your change tomorrow. Thank you :)

@rbruhn, I added in your piece so it'll hopefully fix up the issue though it needs some more testing.

@BTMash, It appeared to work fine for me after that change.

The only strange thing I'm running into right now is it appears my Views are still working even after the change. There are a few relationships in my Views I used to get User info, or related nodes, and everything is still appearing in my Views. Even after flushing cache, etc. Not sure why, but I'm not complaining.. lol

I posted a few of the things I went through while migrating from Reference to Entity Reference that might be useful for some of you: References vs Entity Reference & Migrating Obviously the script BTMash provided was of significant help. Thanks!

Don't know if this will affect anyone else, but I ran into the temp table
$temp_table_name = 'entityreference_migration_' . $table_name;
being too long for MySQL. I cheanged it to:
$temp_table_name = 'e_m_' . $table_name;
And it worked smoothly.

@rbruhn, that is because I've done some basic conversion of views used in references to convert over to entityreference (I've tackled arguments as far as I remember)

@codeelegance, I just committed a patch that changed the table name to 'er_m_'. $table_name. And I do a further trimming down to 64 chars in the event that the table name is still long. Thanks to you and @batje for letting me know of the issue.

I'm attempting to use this migration code as a module and wonder if this should work? (I have drush installed on my windows desktop but not git.)

I've downloaded the code at http://drupalcode.org/sandbox/btmash/1404542.git/tree/HEAD and enabled at a standard drupal module.

Navigated to http://localhost/admin/content/migrate-references

I chose a node reference that I would like to migrate to an entity reference and got:

An AJAX HTTP error occurred. HTTP Result Code: 500 Debugging information follows. Path: /batch?id=136&op=do StatusText: Service unavailable (with message) ResponseText: PDOException: SQLSTATE[42000]: Syntax error or access violation: 1072 Key column 'field_screen_video_artist_target_type' doesn't exist in table: CREATE TABLE {field_data_field_screen_video_artist} ( `entity_type` VARCHAR(128) NOT NULL DEFAULT '' COMMENT 'The entity type this data is attached to', `bundle` VARCHAR(128) NOT NULL DEFAULT '' COMMENT 'The field instance bundle to which this row belongs, used when deleting a field instance', `deleted` TINYINT NOT NULL DEFAULT 0 COMMENT 'A boolean indicating whether this data item has been deleted', `entity_id` INT unsigned NOT NULL COMMENT 'The entity id this data is attached to', `revision_id` INT unsigned NULL DEFAULT NULL COMMENT 'The entity revision id this data is attached to, or NULL if the entity type is not versioned', `language` VARCHAR(32) NOT NULL DEFAULT '' COMMENT 'The language for this data item.', `delta` INT unsigned NOT NULL COMMENT 'The sequence number for this data item, used for multi-value fields', `field_screen_video_artist_target_id` INT unsigned NOT NULL COMMENT 'The id of the target entity.', PRIMARY KEY (`entity_type`, `entity_id`, `deleted`, `delta`, `language`), INDEX `entity_type` (`entity_type`), INDEX `bundle` (`bundle`), INDEX `deleted` (`deleted`), INDEX `entity_id` (`entity_id`), INDEX `revision_id` (`revision_id`), INDEX `language` (`language`), INDEX `field_screen_video_artist_target_entity` (`field_screen_video_artist_target_type`, `field_screen_video_artist_target_id`), INDEX `field_screen_video_artist_target_id` (`field_screen_video_artist_target_id`) ) ENGINE = InnoDB DEFAULT CHARACTER SET utf8 COMMENT 'Data storage for field 83 (field_screen_video_artist)'; Array ( ) in db_create_table() (line 2685 of C:\xampp\htdocs\sawvideo\includes\database\database.inc).

In the drupal site the node reference is gone, but the entity reference was not created.

How does one use this code from the drush command line?

Do I need to setup a field before the migration?

Thanks for any comments!

argh...I though the issue was now taken care of...I'll take a look and see what is going on.

@mlecha, do you have the latest code from the code repository? You should use the 7.x-1.x branch and not HEAD (I'm going to update the codebase to reflect this).

The module will automatically convert the reference field into a entity reference field of the same name (so you do not need to configure another field). It won't migrate the data over into another field (I could see a module for it though it would be errily similar to this).

@BTMash Checked out 7.x-1.x branch and it worked fine. Thank you for the code and for time to look into my issue and make a reply. Much appreciated.

@mlecha, thank you for letting me know to do this :)

I've now converted this over to an official project with a namespace - http://drupal.org/project/entityreference_migration. Not the best name but it should work in this case (and leaves room for a entityreference_migrate should we go down a path with migrate and handlers to move over fields).

Lets continue over on the issue queue of BTMash's module: http://drupal.org/project/entityreference_migration