Image_attach migration

Is this still true? thanks!

Yup. Someone needs to write a migration plan for it -- shouldn't be too hard, just define an entityreference field and migrate the data to that.

discovered this one after having created #1643396: image_attach D6 -> D7 upgrade
That's incredible such a common D6 module didn't get its migration path !

It is, yes, but it would seem that nobody using it is either in a position to upgrade their site to D7, or has used other means to upgrade, or is not able to write the code for this.

Just did 378 nodes for a total of 841 attached images. using the following (absolutely alpha-quality) 100-lines script:
[edit1]

/**
   Raphaël Droz, 2012, WTFPL

   image_attach.convert.php:
   Attempt to fixes Drupal critical bug #1226062:
   - Converts the {image_attach} table from D6 to D7
   - Switch from image_node to Image or File field

   * iid is mapped to the corresponding image_node
   * image_node is mapped to the corresponding File
   * file is added to a field of the node (nid in {image_attach})
   * node is saved
   * record is removed from {image_attach}

   * Files order is preserved
   * Field's cardinality is checked
   * The node's destination field can be autodetected (image or generic_file)
   * The node's destination field can be fixed on a per-bundle basis using $newfields

   usage: backup your DB then run:
   $ drush scr image_attach.convert.php

   **/

// customize here: "bundle" => "field_name"
// or comment for autodetection
global $newfields, $nid;
$newfields = array(
  'report' => 'node_image',
  'announcement' => 'node_image',
);

$res = db_query('SELECT DISTINCT nid FROM {image_attach}');

foreach($res as $obj) {
  $nid = $obj->nid;
  $node = node_load($nid);
  if(!$node) {
    drush_log("can't convert node $nid: missing", 'warning');
    continue;
  }

  if(!($destfield = _c_check_field($node->type))) continue;
  $fieldsettings = field_info_field($destfield);
  $card  = intval($fieldsettings["cardinality"]);

  $res2 = db_query('SELECT iid, weight FROM {image_attach}'
		   . ' WHERE nid = ' . $nid
		   . ' ORDER BY weight DESC');

  // stores iid alone + fid alone + couple (iid, weight)
  $ok = array();

  foreach($res2 as $oimg) {
    if( ($file = _get_legacy_image($oimg, $nid, $ok)) ) {
      if(!isset($file->display))
	$file->display = 1; // TODO ?
      $node->{$destfield}[LANGUAGE_NONE][] = (array)$file;
    }
  }

  if(!$ok) continue;
  // -1 means unlimited
  if(count($ok['i']) > $card && $card >= 0) {
    drush_log("can't convert node $nid: $destfield cardinality of $card < " . count($ok['i']), 'warning');
    continue;
  }
  
  drush_log("ok: $nid :  " .
	    "iid: " . implode(' ', $ok['i']) . ' => ' .
	    "fid: " . implode(' ', $ok['f']),  'status');


  node_save($node);
  foreach($ok['ok'] as $v) {
    db_query(
      sprintf('DELETE FROM {image_attach} where nid = %d AND iid = %d AND weight = %d',
	      $nid, $v[0], $v[1]));
  }
}

function _c_check_field($bundle) {
  global $nid, $newfields;
  if(!empty($newfields[$bundle])) {
    $newfield = $newfields[$bundle];
    $fins = field_info_instance("node", $newfield, $bundle);
    if(!$fins) {
      drush_log("can't find " . $newfield . " in {$bundle}", 'warning');
      return FALSE;
    }
    return $newfield;
  }

  // guess a destination field
  $fields = field_read_instances(array("bundle" => $bundle));
  $match = array();

  foreach($fields as $field) {
    if($field['widget']['module'] == 'image' ||
       $field['widget']['type'] == 'file_generic')
      $match[] = $field['field_name'];
  }
  if(! $match) {
    drush_log("can't find a destination image field for nid $nid ($bundle)", 'warning');
    return FALSE;
  }
  if(count($match) > 1) {
    drush_log("more than 1 possible destination field for nid $nid", 'warning');
    return FALSE;
  }
  return $match[0];
}

/**
   warn !!
   every iid in {image_attach} is a reference to the nid
   of an image_node.
   This image_node should have been migrated thus having
   already a valid fid (pointing in {file_managed}
   That's what we are looking for.
**/
function _get_legacy_image($oimg, $nid, &$ok) {
  /* never ever do this the following, you may grab a wrong file !
     $file = file_load($oimg->iid); if($file) goto ok;*/

  // fallback on the image_node principles
  $image_node = node_load($oimg->iid);
  if($image_node) {
    if($image_node->type == 'image' &&
       count($image_node->node_image[LANGUAGE_NONE]) == 1 &&
       ($file = file_load($image_node->node_image[LANGUAGE_NONE][0]['fid'])))
      goto ok;
    
    drush_log("$nid/{$oimg->iid} problematic image_node", 'warning');
    return FALSE;
  }

  $b = db_query("SELECT * FROM files where fid = " . $oimg->iid)->fetch();
  if(!$b)
    drush_log("$nid/{$oimg->iid} not even an entry in the files table", 'warning');

  if(in_array($b->filename, array('_original', 'preview', 'thumbnail')) &&
     drush_get_context('DRUSH_VERBOSE'))
    drush_log("$nid/{$oimg->iid} thumbnail => skipped", 'status');
  // db_query("SELECT * FROM image where fid = " . $oimg->iid)->fetch();
  // drush_log("$nid/{$oimg->iid} leftover in the files table: $b->filename", 'status');
  return FALSE;

ok:
  $ok["i"][] = $oimg->iid;
  $ok["f"][] = $file->fid;
  $ok["ok"][] = array($oimg->iid, $oimg->weight);
  return $file;
}

Thanks, Raphaël, for your work on the image_attach migration nightmare!

Would you mind to elaborate a bit what the script does, especially for those too dumb to read the actual code below the comments? How are you handling the re-use of image nodes in D7?

Hmm. I kind of figured that the same framework that handles the main image node migration could be used too...

Alternatively, I'm wondering whether Migrate module would be a way to do this, as I've recently started using it for something else and it seems really powerful and easy to use.

for each SELECT DISTINCT nid FROM {image_attach}')
  for each SELECT iid FROM {image_attach} WHERE nid = $nid
      $file = _get_legacy_image($iid)
      $node->{$destfield}[LANGUAGE_NONE][] = (array)$file;
  node_save($node);
  DELETE FROM {image_attach} where nid = $nid

should the image_node been deleted too is another question.
but I've to work on getting other image_node (non-image_attach) into D7 field first.

My server is running PHP 5.2.6, so the goto operator in line 133 is causing a syntax error

Hi. I have modified the above script, in order to make the images from " image_attach " to be entered as Media fields.

How to:

0. Install and enable media module (Tested with Media version 1). You will also need drush
1. Go the the http://website/admin/structure/types/manage/story/fields , assuming that the story type is the one that has the attachments
2. Add a new field with the desired name.
Field type [Multimedia Asset] or [Image] or [File].
Widget: Media file selector
(repeat steps 1,2 for every node type that used image_attached)
3. Using SSH or SFTP create a file on your server home folder with the following content, named ~/image_attach.convert.php
4. Use the commandline to go to drupal/sites/mysite (or drupal/sites/default) and run
drush scr ~/image_attach.convert.php

Here's the modified code

/**
   Raphaël Droz, 2012, WTFPL , modified by RadioBuzzer

   image_attach.convert.php:
   Attempt to fixes Drupal critical bug #1226062:
   - Converts the {image_attach} table from D6 to D7
   - Switch from image_node to Image or File field of the media module

   * iid is mapped to the corresponding image_node
   * image_node is mapped to the corresponding File
   * file is added to a field of the node (nid in {image_attach})
   * node is saved
   * record is removed from {image_attach}

   * Files order is preserved
   * Field's cardinality is checked
   * The node's destination field can be autodetected (image or generic_file)
   * The node's destination field can be fixed on a per-bundle basis using $newfields

   usage: backup your DB then run:
   $ drush scr image_attach.convert.php

   **/

// customize here: "bundle" => "field_name"
// or comment for autodetection
global $newfields, $nid;
$newfields = array(
  'report' => 'node_image',
  'announcement' => 'node_image',
);

$res = db_query('SELECT DISTINCT nid FROM {image_attach}');

foreach($res as $obj) {
  $nid = $obj->nid;
  $node = node_load($nid);
  if(!$node) {
    drush_log("can't convert node $nid: missing", 'warning');
    continue;
  }

  if(!($destfield = _c_check_field($node->type))) continue;
  $fieldsettings = field_info_field($destfield);
  $card  = intval($fieldsettings["cardinality"]);

  $res2 = db_query('SELECT iid, weight FROM {image_attach}'
           . ' WHERE nid = ' . $nid
           . ' ORDER BY weight DESC');

  // stores iid alone + fid alone + couple (iid, weight)
  $ok = array();

  foreach($res2 as $oimg) {
    if( ($file = _get_legacy_image($oimg, $nid, $ok)) ) {
      if(!isset($file->display))
    $file->display = 1; // TODO ?
      $node->{$destfield}[LANGUAGE_NONE][] = (array)$file;
    }
  }

  if(!$ok) continue;
  // -1 means unlimited
  if(count($ok['i']) > $card && $card >= 0) {
    drush_log("can't convert node $nid: $destfield cardinality of $card < " . count($ok['i']), 'warning');
    continue;
  }
 
  drush_log("ok: $nid :  " .
        "iid: " . implode(' ', $ok['i']) . ' => ' .
        "fid: " . implode(' ', $ok['f']),  'status');


  node_save($node);
  foreach($ok['ok'] as $v) {
    db_query(
      sprintf('DELETE FROM {image_attach} where nid = %d AND iid = %d AND weight = %d',
          $nid, $v[0], $v[1]));
  }
}

function _c_check_field($bundle) {
  global $nid, $newfields;
  if(!empty($newfields[$bundle])) {
    $newfield = $newfields[$bundle];
    $fins = field_info_instance("node", $newfield, $bundle);
    if(!$fins) {
      drush_log("can't find " . $newfield . " in {$bundle}", 'warning');
      return FALSE;
    }
    return $newfield;
  }

  // guess a destination field
  $fields = field_read_instances(array("bundle" => $bundle));
  $match = array();

  foreach($fields as $field) {
    if($field['widget']['module'] == 'media') //    ||   $field['widget']['type'] == 'file_generic')
      $match[] = $field['field_name'];
  }
  if(! $match) {
    drush_log("can't find a destination image field for nid $nid ($bundle)", 'warning');
    return FALSE;
  }
  if(count($match) > 1) {
    drush_log("more than 1 possible destination field for nid $nid", 'warning');
    return FALSE;
  }
  return $match[0];
}

/**
   warn !!
   every iid in {image_attach} is a reference to the nid
   of an image_node.
   This image_node should have been migrated thus having
   already a valid fid (pointing in {file_managed}
   That's what we are looking for.
**/
function _get_legacy_image($oimg, $nid, &$ok) {
  /* never ever do this the following, you may grab a wrong file !
     $file = file_load($oimg->iid); if($file) goto ok;*/

  // fallback on the image_node principles
  $image_node = node_load($oimg->iid);
  if($image_node) {
    if($image_node->type == 'image' &&
       count($image_node->node_image[LANGUAGE_NONE]) == 1 &&
       ($file = file_load($image_node->node_image[LANGUAGE_NONE][0]['fid']))) {
       $ok["i"][] = $oimg->iid;
       $ok["f"][] = $file->fid;
       $ok["ok"][] = array($oimg->iid, $oimg->weight);
       return $file;
    }else{
      drush_log("$nid/{$oimg->iid} problematic image_node", 'warning');
      return FALSE;
    }
  }

  $b = db_query("SELECT * FROM files where fid = " . $oimg->iid)->fetch();
  if(!$b)
    drush_log("$nid/{$oimg->iid} not even an entry in the files table", 'warning');

  if(in_array($b->filename, array('_original', 'preview', 'thumbnail')) &&
     drush_get_context('DRUSH_VERBOSE'))
    drush_log("$nid/{$oimg->iid} thumbnail => skipped", 'status');
  // db_query("SELECT * FROM image where fid = " . $oimg->iid)->fetch();
  // drush_log("$nid/{$oimg->iid} leftover in the files table: $b->filename", 'status');
  return FALSE;
}

Please bear in mind that this issue is on the Field converter project, therefore is about specifically using the Field converter framework to provide a UI upgrade path for image attach.

Though see also my comment at http://drupal.org/node/513096#comment-6182394.

will this be properly fixed before D8 ?
the number of knowledgeable people is inversely proportional to the number of APIs, and a few people knows about migrate.
where are we going ?
sooner or later, people will need a way to go from image nodes / image_attach to file fields.

Perhaps all these people could contribute to the ChipIn for the D7 upgrade path, which currently stands at an astounding $22!! See http://drupal.org/project/image.

as you may have noticed, my script was wrong at sorting weight, it should be ASC rather than DESC ... and other enhancements are possible : just drop me an email + $ 836.54 and I'll provide you with an updated update-script.

Oh wait, don't you known about the 21.12 bug ? you need to send $ 8500 @linus' paypal then obtain a patch in order be safe from kernel crash that day.

The point of using Migrate for this is that it would operate within an existing, solid, and I might add, very well documented framework, which would do things like provide a UI, and handle creation of and writing to Drupal objects cleanly.

And I resent your implication that in setting up a ChipIn, I'm somehow holding people to ransom. I am a volunteer, working on this and other modules in my free time, and the D7 migration is a very large undertaking that a lot of people seem to think will magically materialize from thin air. It won't. If everyone who needs it contributed a modest amount, I could take some of my working hours to get it done.

Kudos to drzraf for sharing his migration path! Though the requirement for drush is a bit of a pain in the butt it worked like a charm!

Thanks again!

-Ryan

I did exactly what radiobuzzer suggested in comment #12, and I am happy to say everything pretty much worked like I hoped. All the images are now displaying in my nodes. However, I have one problem. Before, with the image_attach module, the link for the image would take you to the image node. However, now, my only options for that image field under manage display is to either link to content or link to file. As you can guess, link to content basically links to the node that the image node appears in and not the image node itself.

Is there any place we can configure this?

this stuff convert image node to image field, how would you want to link to an image-node which does not exists anymore ?

Yup, you have to decide whether you want to migrate to:

A:

node -> image field

or B:

node -> image node -> image field

Yeah, sorry about that, that was a really dumb question. It's like watching the entire Twilight Saga and asking who 'Bella' was. Nevertheless, thanks a lot guys for the work on this. Didn't think the conversion would go so smoothly as it did. Really appreciate it!!!

Hi drzraf,

Nice to see this stuff, here i struck when migrate to manage files table (ie.) files-to-file_managed.txt i renamed to files-to-file_managed.php, When i run that file for me it shows blank page. Dont know why?
Can you explain me the steps what i should follow.

Any one have idea please help.

Thanks in advance.

install, configure, understand and finally use drush with eg :
drush scr image_attach.convert.txt

I would like to try radiobuzzer's approach to this field migration. I have all the steps outlined complete except for the final step of executing the file. Before I do this, I would like to get confirmation from anyone here that I am doing this correctly for how my server/account is set up.

I am in a shared hosting environment with this site. In our shared hosting account, we have one site in the root and several sites in subdirectories using add-on domains. The development site I am trying to implement this field migration on is in one of the subdirectories, not in the account root. I have placed the image_attach.convert.php file in the subdirectory for the site I'm trying to do this on. That subdirectory is the "root" for this single website.

If I understand the directions correctly, I must now navigate to the sites/default directory within this website and run the drush command from there. Within the command shown in the instructions above, I must replace the ~/ with the path leading to where I have the image_attach.convert.php file -- is that correct?

If I do the above, I just want to make sure this will only effect the single site within the subdirectory and that there is no way doing this will impact any of the other sites within this account. Can anyone here confirm this for me before I proceed. This is my first attempt at using drush for anything and I'm suffering from newbie intimidation where I'm afraid I'm going to royally mess things up if I do the wrong thing. So confirmation that I have everything right before I proceed will be most helpful. Thanks in advance to anyone that responds. I'm very glad to have found a solution to this problem that I hope will work. I will have to rebuild hundreds of Image Attach files if this doesn't work, so fingers crossed.

I have tried to run radiobuzzer's script using drush. I isolated this site from the other sites to make sure it didn't effect them. The script ran, but I got the following warning on each node where it attempted to work:

more than 1 possible destination field for nid

In looking at my content type structure, I think this is because I have already imported another image field for this content type. This other image field was one created using CCK and not the Image Attach module and I successfully migrated that data using the CCK Migrate module. So the following part of the script is seeing two possible fields as destinations:

 // guess a destination field
  $fields = field_read_instances(array("bundle" => $bundle));
  $match = array();
  foreach($fields as $field) {
    if($field['widget']['module'] == 'media') //    ||   $field['widget']['type'] == 'file_generic')
      $match[] = $field['field_name'];
  }
  if(! $match) {
    drush_log("can't find a destination image field for nid $nid ($bundle)", 'warning');
    return FALSE;
  }
  if(count($match) > 1) {
    drush_log("more than 1 possible destination field for nid $nid", 'warning');
    return FALSE;
  }
  return $match[0];
}

Instead of looking for fields with a media widget or image widget, is there any way this can be modified to use a specified field as the destination? The machine name of the field I would like to use as the destination is field_page_image. I don't know PHP code well enough to know how to do this. My best guess would be to change the line that says

$fields = field_read_instances(array("bundle" => $bundle));

to just

$fields = field_page_image

and get rid of the match code, but that's probably wrong. Can anyone assist with making this change as needed?

I attempted to modify the php file doing the best I could. I think I managed to define the destination field as needed, but now when I ran the modified version of the script, I got "problematic image_node" as the warning message for each image. From my limited code knowledge, it looks to me like there is a check for image node types defined as "image". The problem I'm seeing is that my image nodes do not seem to have anything defined for their content types. When I look at the Content page, where it used to say "Image" as the content type in D6, now that space is empty in D7. So I'm guessing that this script isn't working for me because it is not recognizing the image nodes defined as "image".

However, looking at my database in phpMyAdmin, I'm looking at my image nodes in the Node table. In here, the type is defined as "image". So I'm not sure what's wrong at all here.

Thank you drzraf. Your scripts worked like a charm.
A piece of advice for others like me:
1. Apparently you need drush, don't waste your time trying to edit the scripts to bootstrap Drupal. Didn't work for me anyway.
2. If you, like me, didn't use Drush before. No matter what the install instructions in the Git repo says. You don't need Composer. It's actually a simple install.
3. If you got a multisite install, drush will still try to load sites/default/settings.php
4. You need to run First files-to-file_managed.php, Then image_attach.convert.php
5. The images will now be 0-to-many uploaded files on your nodes. So you might need to do a little hands on in views fields etc.

This approach has been dropped in favour of using Migrate module as a framework.

Great!
Could you please post how one is expected to move from image_attach D6 to file fields D8 and which PHP migrate plugin does this (in which git tree)?
Thank you.