Update MigrateFieldHandler to work with migrate-7.x-2.4 and later

I'm looking into this, at the moment, on 7.

Here's a patch for 7.x. I imagine the version for 6 wouldn't be a whole lot different, so it seemed wasteful to start a different issue.

How should we use this patch to migrate a link field

This patch worked fine. The version for the migrate module is 7.x-2.1 and the version for the link module is 7.x-1.0-alpha3.

We need to add the following line in our migration script to migrate links in drupal 7
$arguments = MigrateLinkFieldHandler::arguments(array('source_field' => 'link_name'));
$this->addFieldMapping('field_link', 'link_url')
->arguments($arguments);

Subscribe (I'll update the Migrate Extras project page when this is committed).

Has anyone migrated multiple links for a single field, each with their own titles? I'm currently attempting to hack together a couple different solutions, but I'd love it if someone has already done the work for me.

UPDATE: I've posted more of my situation over in a Migrate issue, here.

This version of the patch has been re-rolled to apply to 6.x-2.9. However it needs to be rerolled for 6.x-2.x-dev.

Oops! That was the patch from #2. this patch is for 6.x-2.9.

I think I must have been confused above and meant 7.x-dev.

This version of the patch adds the ability for multiple URLs to be set AND for there to be a unique title per URL. The title source should be an array. E.g.

function prepareRow($row) {
  $links = $titles = array();
  foreach (array('LinkedIn', 'Twitter', 'Facebook', 'Other', 'Profile') as $type) {
    $key = strtolower($type);
    if (!empty($row->{"{$key}_url"})) {
      $links[] = $row->{"{$key}_url"};
      $titles[] = $type;
    }
  }
  $row->link_urls = implode("\n", $links);
  $row->link_titles = $titles;
}

Allowing the title field to be an array is a bit of a hack since it requires use of ->prepareRow to create the array. I have suggested what could be a better, cleaner and simpler solution as a feature request for the Migrate API: #1222668: Can I add onto a unlimited-value node reference field when updating?.

In case anyone else needs it, I have created a patch for link-6.x-2.x that has the same functionality as #2.

This has no enhancements over #2, but just a backport to 6.x.

I didn't backport the patch from #9 because that didn't seem to be solved or agreed upon as a good solution with this issue and #1222668: Can I add onto a unlimited-value node reference field when updating?.

@eporama: Nice one. Thanks!

@eporama Thanks a bunch for the patch! It works quite nicely, but I'm having one minor issue: If a link source is empty, but a static title is set, when the data is imported, the link field will no longer be empty (versus saving a node with an empty link field in which case the link field is truly empty). Hopefully that was clear as mud.

I'm still trying to fully understand the Migrate module/process, so I'm not sure where/how to handle this issue. Any pointers would be greatly appreciated. (I can fix this at the theme level, but I'd have to do this for all my links, which is not desirable if another workaround exists.)

Is there any way to check if the url is empty() for the import data, and if so, don't import the filed for that particular node?

Thanks,

Dan

@daneesia, I tested the process with a static title and it behaved as I expected. It left NULL link sources and didn't assign them titles. Are you trying to import the static title? If you have that set on your cck field, you shouldn't need to set the title attribute in the migrate class.

If you addFieldMapping for the link field, but don't apply the title attribute in the class, you should get the desired behavior.

The patch from #2 doesn't apply when using Link module 7.x-1.0-alpha3.

The error is "error: patch failed: link.module:896". The link.module file from 1.0-alpha3 doesn't go up to line 896, and so the git apply command fails.

Any chance someone could reroll this patch for 7.x-1.0-alpha3?

Thanks!

Subscribing.

Any chance someone could reroll this patch for 7.x-1.0-alpha3?

@ digitalcarla and all others

Please use latest --dev of link module from now. It is strongly recommended to use it rather than alpha3, since in the latest --dev push are already some of known issues fixed (html tags f.e. with static title, etc) and please go from there to report back for better overview and any leftovers.

Any reports and patches are much much appreciated in the moment, since we have still a lot to catch up to get right for final D7 release. I am appointed in here since yesterday to push things forward -

more info here: #1269718: Blockers to a 7.x-1.0 release?

Perhaps it is time for another alpha release then? Or a beta?

exactly. beta is in plan 4 next days. as I sad somewhere else in the queue, we will put some of the most important patches together to get away from the latest dev + patches. and to get feedback for final release schedule.

beta is out. :) 2 night shifts ... goto go sleepin ...

please start again with retrieving the patch for beta1 and a new issue! Would be much appreciated!

thanks for all contribution til here!

One point I'd like to mention is that the title should be importable as well. While attributes could be treated as equal for all items a title is directly related to an url.

Thus I wrote earlier this spring the attached migration handler.
This handler requires a joined string that contains the url and the title for a link. Thus it let's you define the separator to split the joined string and define which value is the title and which the url.

Another approach could be to implement MigrateLinkFieldHandler->fields() to provide a dedicated field which holds the link title. MigrateLinkFieldHandler->prepare() joins then the values of the mapped url field and the special link title field.

Rerolled #9 to work with drush make

This patch never got added as far as I can see.

patch fails ...

fatal: git diff header lacks filename information when removing 1 leading pathname components (line 5)

INFO: patch will be committed immediately when it passes without errors.

thanks for your effort. Awesome contribution!

Here's a reroll of the patch in #21 to make it apply cleanly. The patch was just missing the git a/ b/ so git apply -p0 worked fine. This patch applied cleanly with git apply.

git commit -m "Issue #1010850 by jcfiala, quartsize, raman2385, Bevan, eporama, das-peter, smithmilner, Digidog: Fixed MigrateFieldHandler to work with Migrate Module"

 3 files changed, 67 insertions(+), 0 deletions(-)
 create mode 100644 link.migrate.inc

Counting objects: 8, done.
Writing objects: 100% (5/5), 1.05 KiB, done.
Total 5 (delta 3), reused 0 (delta 0)

committed to latest --dev

@ all: Awesome contribution! ROCK!

Dear contributers and patchers: please help us and take a look into one of our other main trouble makers:

#1319520: Gathered: Internationalized domain names (punycode)
#1295160: Gathered: options and tools to create Digg like site with link module
#1318938: An all-embracing attempt for URL validation and Input / Output Form Filtering
#1325112: Gathered: Integration of Link module working with Views module (6.x and 7.x)

woo hoo.

Any chance we can get the patch from #10 http://drupal.org/node/1010850#comment-4912598 into 6.x? ;-) Do you want a diff issue for that?

Since most of the discussion run here, I think its ok to point to here for that. Let me check if the patch still fits to latest --dev

git commit -m "Issue #1010850 by jcfiala, quartsize, raman2385, Bevan, eporama, das-peter, smithmilner, Digidog: Fixed MigrateFieldHandler to work with Migrate Module"

3 files changed, 59 insertions(+), 0 deletions(-)
create mode 100644 link.migrate.inc

Counting objects: 8, done.
Writing objects: 100% (5/5), 1018 bytes, done.
Total 5 (delta 3), reused 0 (delta 0)

committed to latest 6.x-2.x-dev

@ all: Awesome contribution! Please take a look into one of our other main trouble makers too:

#1319520: Gathered: Internationalized domain names (punycode)
#1295160: Gathered: options and tools to create Digg like site with link module
#1318938: An all-embracing attempt for URL validation and Input / Output Form Filtering
#1325112: Gathered: Integration of Link module working with Views module (6.x and 7.x)

Woohoo!

@Bevan (or anyone)

I am trying to migrate links with unlimited values. Your example doesn't address migrating unlimited titles, just a very specific case. In mine only one of the links imports. I've been trying for days to get this to work. Here's my code related to the links:

    $linkArg = MigrateLinkFieldHandler::arguments(array('source_field' => 'field_source_title'));
    
    $this->addFieldMapping('field_source', 'field_source_url')->arguments($linkArg);

Any advice on how to do this? The migrate documentation is good but it doesn't expose the api, at least I haven't found that. I really hope someone can guide me a little, thank you in advance.

Bradford

It was a while ago that I worked on this. I seem to remember that being a really really difficult problem to attempt to solve given Migrate's API (my problem wasn't quite that complicated). And that Migrate's hook that cleans up each "row" just after mapping but before inserting in Drupal was the only place you could build complicated multi-valued field values from flat & non-SQL sources (such as CSV). Sorry. :(

Hi @elBradford, did you ever figure out how to handle multiple link titles? Seems to me if the argument handling code could somehow be adjusted to handle an array, I think I have the rest solved.

@webdrips For the moment I'm focusing on other parts of migrating my site to drupal 7 - kind of letting this one sit for a bit. But I will probably return to this one in the next couple weeks.

Seems like it's been solved for filefield, but I admit I haven't tried it just yet: http://drupal.org/node/1005090

I was wrong about how filefield handles it; now it's much easier and cleaner using JSON.

Here is how I got multiple files/titles/descriptions added:

In my constructor:

    // Add Media to Events
    $query->leftJoin('events_media', 'em', 'em.events_id = ev.id');
    $query->addField('em', 'title');
    $query->addField('em', 'file');
    $query->addExpression("GROUP_CONCAT(DISTINCT em.title SEPARATOR '|')", 'media_titles');        
    $query->addExpression("GROUP_CONCAT(DISTINCT em.file SEPARATOR '|')", 'media_files');        

...
   // Note the arguments for path, title, and description are all NULL in the argument as it's handled in prepareRow()
    $arguments_events_media = MigrateFileFieldHandler::arguments(NULL, 'file_copy', FILE_EXISTS_REPLACE); 
    $this->addFieldMapping('field_event_media_attachment', 'media_files')
         ->arguments($arguments_events_media);   

In Prepare Row, do this:

    $media_titles = explode('|', $current_row->media_titles);
    $media_files = explode('|', $current_row->media_files);

    $current_row->media_files = array();
    $count = 0;
    foreach($media_files as $mf) {
      $file_data = array(
        'path' => 'sites/mysite/files/formigration/media/events/' . $media_files[$count],
        'title' => $media_titles[$count],
        'description' => $media_titles[$count],        
      );
      $current_row->media_files[] = drupal_to_js($file_data);
      $count++;
    } 

I can probably look at creating a patch that applies similar logic to links, but I won't be able to get to it for several weeks.

I came across this issue when I was trying to figure out how to migrate some data that had multiple links per node, each with their own title. I used the code from #9 as a starter, but discovered that didn't really work for me. Here's what I did in my prepareRow() (I queried a resources table which contained both file and url resources, so I attached them to the appropriate destination fields)

	foreach ($result as $row) {
		if ($row->type == 'File') {
			if (!empty($row->filename)) {
				$currentRow->filename[] = "resources/{$sourceId}/{$row->filename}";
				$currentRow->resource_title[] = $row->title;
			}
		} else {
			if (!empty($row->url)) {
				$links[] = $row->url;
				$titles[] = $row->title;
			}
		}
	}
	if (!empty($links)) {
		$currentRow->url = $links;
		$currentRow->url_titles = $titles;
	} else {
		$currentRow->url = NULL;
	}
	return TRUE;
}

I couldn't figure out how to keep an empty record being created when there were no links associated with a node, but that is for another issue.

This was a pain to get working but I learned more about Migrate in the process. This is my code:

In the constructor I have the following:

$source_fields = array(
    ...
    'links' => t('The source URL'),
);

...

$linkArg = MigrateLinkFieldHandler::arguments(array('source_field' => 'titles'));
$this->addFieldMapping('field_source', 'links')->arguments($linkArg);

"titles" and "links" are defined in prepareRow shown here:

public function prepareRow($current_row) { 

    ...
   
    $source_id = $current_row->nid; //get the node id of the current node

    $result = db_select(ISP_MIGRATION_DATABASE_NAME . '.content_field_source', 's')
      ->fields('s', array('field_source_url', 'field_source_title', 'nid'))
      ->condition('s.nid', $source_id, '=')
      ->execute(); 
    
    $current_row->links = array();  //links, that is used above
    $current_row->titles = array();  //titles, used above
    
    //loop through all the results (for multiple urls) and add them to the links and titles arrays
    foreach ($result as $row) {

        $current_row->links[] = $row->field_source_url;
        $current_row->titles[] = $row->field_source_title;

    }
    return TRUE;
  }

The thing that confused me was that I didn't know that prepareRow can set up a new source, in my case "links" and "titles", so I can feed those arrays right into MigrateLinkFieldHandler.

Hopefully this can help someone who was as stuck as I was.

Migrate 2.4 changed how it handles field arguments. Instead of using ->arguments you can map subfields. So link, link:title, and link:attributes would all show up in the destination field list.

To make that happen, link's field handler needs to implement fields(), which returns an associative array of field => description. I've attached mine, although I'm sure some of you will provide more interesting descriptions.

Following up from #41, here's a complete patch for Migrate v2.4 that can be applied to the 7.x-1.0 release of Link.

Changing status.

#42 did not apply, new patch attached.

Patch from #42 with #45 blended in worked great for me today for migrating in data on migrate 2.4-dev. Thanks guys! This should get committed into the project, very very helpful.

And here's a clean patch for the combination of #42 and #45. Applies against Link 7.x-1.0. Works for me with migrate 7.x-2.4.

Thanks jpklein and Jelle_S.

@elstudio: patches should be made against latest dev ;-)

The patch in #45 does apply cleanly to dev 7.x-1.x.

The comments here about multiple URLs and titles are probably best held in #1222668: Can I add onto a unlimited-value node reference field when updating?.

I have tested this patch and it works fine.

The patch in #45 does apply cleanly to dev 7.x-1.x.

The comments here about multiple URLs and titles are probably best held in #1222668: Can I add onto a unlimited-value node reference field when updating?.

I have tested this patch and it works fine.

I can confirm that the patch in #47 applies to Link 7.x-1.0 just fine, and after adding the new migrate.inc and clearing my site's Drupal cache, I'm able to successfully migrate D6 Link fields into D7.

Thanks a million!

I've just used the 7.x-1.x-dev version of Link to migrate a link field from drupal 6 to drupal 7. Both of the following code snippets worked to migrate a link field that only had a URL in it (field_website_link is the source field, field_link is the destination field):

$arguments = MigrateLinkFieldHandler::arguments(array('source_field' => 'field_website_link'));
$this->addFieldMapping('field_link', 'field_website_link')
     ->arguments($arguments);

or

$this->addFieldMapping('field_link', 'field_website_link');

The migration also worked when applying the patch at #47 to Link 7.x-1.0

Patch #45 doesn't consider if URL is NULL and we always have empty record in link field.

Fixed patch attached.

Rerolled against current -dev.

Using #54 the NULL links don't come across any more (normal links still work fine). Patch looks good to me!

If you've already been migrating links and have a bunch of NULL links in your database, you can get rid of them with a manual query (assuming your destination link field is called "field_link"):

delete from field_data_field_link where field_link_title is null and field_link_url = '';

Re-rolled the patch fixing the implementation of the MigrateFieldHandler::fields() function for Migrate 7.x-2.5.

This patch consolidates the previous patch with patches from #1832754: Class registration for Migrate 2.5 or later and #1715378: Document available subfields in migration support (there were a couple different takes on fields() among them).

@mikeryan:

As usual, you are a few steps ahead of me. I applied the patch from #57 and my links get pulled in like magic. Thanks!

Rerolled #56's patch for ff518b60113f29885a8f358e8b0fa4499b0c608d