I want migrate files first and then use migrated files in node file/image fields. All files (AttachmentsMigration) are migrated and registered to drupal file_managed table without problem. But if i use rollback for my nodes (ArticlesMigration) my files are always deleted from disk and file_managed database.

I try use

$this->addFieldMapping('preserve_files')
         ->defaultValue(TRUE);

but without success. I try debug in file.inc and for some reason there is no entry in file_usage table.

Here is relevant part of migration class:

class AttachmentsMigration extends HNMigration {
  public function __construct($arguments) {
    parent::__construct($arguments);

    $this->description = t('Migration Attachments from mongodb to Drupal');
    $this->dependencies = array('PrepareAttachment', 'PrepareImage');

    // We instantiate the MigrateMap
    $this->map = new MigrateSQLMap(
      $this->machineName,
      array(
        'old_id' => array(
          'type' => 'int',
          'unsigned' => TRUE,
          'not null' => TRUE,
          'description' => 'ID z ceskeho API.',
        ),
      ),
      MigrateDestinationFile::getKeySchema()
    );

    $query = db_select('hn_migrate_attachments', 'ma')
             ->fields('ma')
             ->orderBy('old_id', 'ASC');

    $this->source = new MigrateSourceSQL($query);
    $this->destination = new MigrateDestinationFile('file');

    $this->addFieldMapping('value', 'data')
         ->callbacks(array($this, 'convertDataToURL')); // create url from MongoDB grid_fs ObjectId
    $this->addFieldMapping('destination_dir', 'old_id')
         ->callbacks(array($this, 'getSubDir'));  // create subdir
    $this->addFieldMapping('destination_file', 'filename')
         ->callbacks(array($this, 'cleanupFileName')); // transliterate filename
    $this->addFieldMapping('file_replace')
         ->defaultValue(MigrateFile::FILE_EXISTS_REUSE); // reuse files
    $this->addFieldMapping('preserve_files')
         ->defaultValue(TRUE); // preserve files
  }

  /**
   * Prepare entity before import begin.
   *
   * @param type $entity
   * @param type $row
   */
  public function prepare($entity, $row) {
    // nastavime povodne meno suboru
    $entity->filename = $row->filename;

    // is file image type?
    if ($image_info = @getimagesize($entity->value)) {
      $entity->type = 'image';
      $entity->filemime = $image_info['mime'];
    }
    else {
      // use file_entity file_get_type() function for determine type of file
      $entity->type = file_get_type($entity);
      $entity->filemime = $row->mime_type;
    }
  }
}

Thanks for advice.

Comments

havran’s picture

I still have no success. I go to through code in plugins/destinations/file.inc and if i try comment out line 661 (in import() method from MigrateDestinationFile class) here:

    // Don't pass preserve_files through to the file class, which will add
    // file_usage - we will handle it ourselves in rollback().
    $file->preserve_files = FALSE; // <<< try comment out

Then migrate module create entry in file_usage table (as i expected).

I try debug rollback() mehod from MigrateDestinationFile and this seems value from $mappings['preserve_files'] is not taken from my field mapping: $this->addFieldMapping('preserve_files')->defaultValue(TRUE);

      // If we're preserving files, roll our own version of file_delete() to make
      // sure we don't delete them. If we're not, make sure we do the job completely.
      $migration = Migration::currentMigration();
      $mappings = $migration->getFieldMappings();
      if (isset($mappings['preserve_files'])) {
        // Assumes it's set using defaultValue
        $preserve_files = $mappings['preserve_files']->getDefaultValue();
      }
      else {
        $preserve_files = FALSE;
      }
mikeryan’s picture

Status: Active » Postponed (maintainer needs more info)

It's rolling back the node migration, not the file migration, that's deleting the files, right? What does the file field mapping look like in the node migration?

mikeryan’s picture

Category: support » bug
Status: Postponed (maintainer needs more info) » Active

OK, I hacked around with the wine.inc example to reproduce this - it is the node rollback that removes the file, because the only file_usage it sees is the one for the field.

The reason we try to avoid adding file_usage is that post-migration, file_usage rows held by migrate prevent manual deletion of files. In this case, though, the only way to prevent the implicit file deletion caused by the field deletion is to add a file_usage row. So, how can we make sure files don't get deleted as a side-effect of migration, but allow manual deletion? This needs a bit more thought...

havran’s picture

Thanks for answer. Yes, it's about node migration - rollback articles delete files migrated by AttachmentsMigration. Here is relevant code from ArticlesMigration class:

    // images in article
    $this->addFieldMapping('field_image', 'images')
         ->sourceMigration('Attachments');
    $this->addFieldMapping('field_image:file_class')
         ->defaultValue('MigrateFileFid');
    $this->addFieldMapping('field_image:alt')->defaultValue('');
    $this->addFieldMapping('field_image:title')->defaultValue('');
    $this->addFieldMapping('field_image:preserve_files')
         ->defaultValue(TRUE); 

I try preserve_files here but files are still deleted.

mikeryan’s picture

Issue tags: +Migrate 2.6

Tagging for Migrate 2.6.

mikeryan’s picture

Status: Active » Fixed

Addressed simply by adding preserve_files support for MigrateFileFid. We're already adding file_usage rows for MigrateFileUri, so this isn't introducing anything really new there.

havran’s picture

Thank you!

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

marcus178’s picture

Is there a solution to this without using 2.6 as I'm migrating from Ubercart as well and I don't want to have to go through the pain barrier of figuring that out to work with 2.6

criznach’s picture

I got this working in 2.5 (so far) by extending MigrateFileFid with my own class

class InewsFilePreservedFid extends MigrateFileFid {
  public function __construct() {
    $this->preserveFiles = TRUE;
  }
}

I'm not sure if that's the normal method, but it seems to work...

Now I can do a rollback and import of both files and nodes, followed by rollback and import of just the nodes. All image fields are re-populated correctly.

marcus178’s picture

@criznach you're a life saver, that's fixed it for me.