Support for Media destinations

Mike just added media module support to migrate_extras - http://drupal.org/cvs?commit=468544

I've updated the media destination to set the entity type and bundle appropriately, and implemented fields() so the available fields are presented in the migrate UI. Actually saving the fields is still broken - see my comment on #1018290: API for creating Media instances.

OK, I've committed a fix to fields not being saved (added a complete() method to call media_save()).

On the media field handler, let's reuse the filefield implementation as much as possible. Is this workable?

class MigrateMediaFieldHandler extends MigrateFileFieldHandler {
  public function __construct() {
    parent::__construct();
    $this->registerTypes(array('media'));
  }

  public function prepare(stdClass $entity, array $field_info, array $instance, array $values) {
    // Do media-specific stuff here...

    // Do standard filefield stuff
    return parent::prepare($entity, $field_info, $instance, $values);
  }

I've fixed the options order, thanks.

basename won't work right if you have a situation like

http://example.com/images/2010/12/my_image.jpg
http://example.com/images/2011/01/my_image.jpg

It's not at all unusual in practice for basenames to be duplicated within the file hierarchy.

It seems to me we had something to trim a specified prefix from the URI - it must've been in a particular project, it would be worth generalizing in MigrateDestinationFile.

Oh, another (perhaps even better) reason to preserve the full path is to preserve the URL of the files, so when you move your old domain to the new server links still work as-is.

so, how is it going with MigrateMediaFieldHandler ?

subscribe.. will be testing out migrate_extras + media soon.

A couple things I noticed so far: 1) re: MigrateDestinationMedia, media module no longer has media entities, it just has file entities, and 2) re: MigrateMediaFieldHandler, media fields are pretty similar to file fields, except that a media field contains a file object, while a file field contains a file array. So it won't be quite as simple as returning MigrateFileFieldHandler::prepare() as suggested in #9 Nevermind, media fields have a file object when loaded but not when saved.

Here's a first stab at MigrateMediaFieldHandler. I couldn't reuse MigrateFileFieldHandler::buildFileArray() as is, but more code could be reused if buildFileArray() were broken up into pieces.

BTW, I think file_fast in MigrateFileFieldHandler is broken: there are a couple instances of $fids[$source] which doesn't make sense as $source is an object.

I committed this from Amtrak the other day, and now realize the connection dropped before my reply went through... Testing was minimal, if any problems are found please open a new issue. Thanks!

Hello!

Based on my experimentation with this functionality this afternoon, it looks as though the patch above was designed back when the Media module exposed its own media entities instead of just making the D7 core file entity fieldable (cf. #1086958: Switch from Media Entity to File Entity for a very extensive discussion of the issue; it was also briefly referenced in this thread at #18). My evidence is that migrate_extras/media.inc still attempts to make use of the media_save() function, which no longer exists.

Unfortunately I'm a bit new to the structure of all these modules, so I'm not entirely sure what would need to change here to ensure compatibility with Media's new approach …that is, although I'm sure that media_save() can no longer be used, I'm not sure that file_save() will be sufficient.

Does anyone have any ideas on this?

Thanks!
Adam

For some reason, I thought I had committed a patch dealing with the changes, but apparently not... OK, I will need to review this when I have time.

Thanks.

yes MigrateDestinationMedia is broken as mentioned in #18. The patch above only dealt with MigrateMediaFieldHandler

Subscribing...

Anyone put in any further work on this or tested the patch from #19?

Subscribe

I should be fixing this up within a couple of days.

I started looking at this today and ended up pretty much starting from scratch. We needed a Media destination for migration, and that was not part of the previous update. I'm attaching two patches, one that replaces all of the media.inc functionality with just a working MigrateDestinationMedia plugin and a second which adds the working destination to the existing media.inc. I'm assuming the field handler will need refactoring as well, so I'll let you decide which to apply.

Looked over your patch, and the condition of the field handler, and I think the Migrate module can support most of the need with just a little generalization: #1240928: META: Refactoring of file destination/field handlers.

Well, as mentioned at #1240928: META: Refactoring of file destination/field handlers, I've made a couple of tweaks to the Migrate file handler support that were helpful, and I've just now pushed a new media.inc that should be semi-useful for the moment - at least it works for image files, although no luck with youtube references. I've concluded that the Migrate file handling needs some refactoring, but I'd like to get 2.2 released before tackling that.

Hi MIke,

Any hints on what you're seeing with Videos? Looks like we're running into the same thing. I'll check out your changes and see what I can do.

Jeremiah

Well, I've hacked things up a bit to get it to work here for video. I'm attaching a patch.

Key things in this patch

1) Your updates to plugins/destinations/file.inc in Migrate module didn't include some key code from my previous patch that allow for migration of additional meta data into the file entity. Here's the relevant code from my modified prepare() method:

    $migration = Migration::currentMigration();
    // Call any general entity handlers (in particular, the builtin field handler)
    migrate_handler_invoke_all('Entity', 'prepare', $entity, $source_row);
    // Then call any entity-specific handlers
    migrate_handler_invoke_all($this->entityType, 'prepare', $entity, $source_row);
    // Then call any prepare handler for this specific Migration.
    if (method_exists($migration, 'prepare')) {
      $migration->prepare($entity, $source_row);
    }

Second, to get videos importing I found I need to call API functions from file_entity and media modules in order to get a more well defined file object. Here's the relevant bit from my modified import() method:

    if ($this->copyFile) {
    ... 
   }
    else {
      // For external sources where we aren't copying the file build a more complete object.
      $file->uri = media_parse_to_uri($file->uri);
      $file = (object) array_merge((array) file_uri_to_object($file->uri), (array) $file);
    }

Feels dirty, but it works in my limited testing. I'm sure there's a cleaner way to handle this, perhaps your re-write of the file handling will address it. Let me know if you have some thoughts on this.

Yes, that's the issue, media needs to do stuff where the file handler has firm control. I'm not going to spend time (of which I have none:-) trying to work around things as they are - I'm looking ahead to refactoring the file handlers, particularly with media in mind.

Hi MIke,

Sounds good - I'll keep posting updates here if we run into issues in our own migrations in case someone else needs a stop-gap solution.

Jeremiah

I use:
Drupal 7.10
Migrate 7.x-2.x-dev (2011-Dec-16)
Migrate Extras 7.x-2.x-dev (2011-Oct-26)

I get the error-message:

Strict warning: Declaration of MigrateDestinationMedia::options() should be compatible with that of MigrateDestinationFile::options() in _registry_check_code() (Line 3004 of G:\xampp\htdocs\abt\includes\bootstrap.inc).

I hope, this error belongs to this issue.

Please help.

Thanks for this very excellent module!

wusel (#36):

I also get that error, it just seems to be because MigrateDestinationFile hasn't been written with easy extension in mind. It isn't important - just lower the error reporting level in your settings.php or in admin/config/development/logging.

The error has not gone...

Please open distinct issues in the queue for distinct problems you see, let's not keep tacking everything media-related on to this issue.