Notes for image module's migration.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

joachim’s picture

This works! :D

To test drive the conversion:

1. Set up a test environment. (You can't try a D6-7 core upgrade yet; core isn't ready.)

- install a fresh D7
- enable image_legacy module (within the 7--1 branch of 'classic' Image module)
- import the attached SQL file into the database
- unzip the attached zipfile into the sites/default/files folder, so you get a folder sites/default/files/images

Your site should now be (roughly!) in the same state as if you have taken a D6 site with image module data, and run a core upgrade to D7. You should have some image nodes that have no images visible and are listed with no type on the admin content page.

2. Run the conversion

- enable Field convert
- go to /admin/content/field_convert
- run the Image conversion.

Your image nodes should now be of type 'image', with a single-valued image field on them. The images from the zipfile should show as values in this field.

Not yet working:
- for some baffling reason, image_legacy_field_convert_object_pre_load() is not getting called; hence files are not getting cleaned up.

To do:
- convert old derivative sizes to new image styles.

Nice to have:
- enable ALT and title text on the new image field, and populate them with the node title. (see notes in the code on specifying complex property conversions...)

Scott J’s picture

Component: Code » Migration plans
Category: bug » task

I gave it a try. Image nodes are created, but I receive message,
"Notice: Array to string conversion in image_get_info() (line 123 of /var/www/drupal7/includes/image.inc)."

'Image' does not show up in admin/structure/types but is in the database node_type table.

joachim’s picture

> Image nodes are created

Do you mean by the SQL import? The conversion process doesn't actually create any nodes...

> "Notice: Array to string conversion in image_get_info() (line 123 of /var/www/drupal7/includes/image.inc)."

Hmm... no idea how that could happen. Did you get that just once or several times?

Scott J’s picture

Sorry, maybe I don't know what I'm doing.

> The conversion process doesn't actually create any nodes...
I mean that this works:

INSERT INTO `node` (`nid`,`vid`,`type`,`language`,`title`,`uid`,`status`,`created`,`changed`,`comment`,`promote`,`sticky`,`tnid`,`translate`)
VALUES
	(1,1,'image','','Box of Hammers',1,1,1270051633,1270051633,2,0,0,0,0),
	(2,2,'image','','archway',1,1,1270051633,1270051633,2,0,0,0,0),
	(3,3,'image','','bus2',1,1,1270051633,1270051633,2,0,0,0,0),
	(4,4,'image','','chair',1,1,1270051633,1270051633,2,0,0,0,0),
	(5,5,'image','','daffodils',1,1,1270051633,1270051633,2,0,0,0,0),
	(6,6,'image','','trees',1,1,1270051633,1270051633,2,0,0,0,0);

And I see the nodes at admin/content.

joachim’s picture

> INSERT INTO `node` (`nid`,`vid`,`type`,`language`,`title`,`uid`,`status`,`created`,`changed`,`comment`,`promote`,`sticky`,`tnid`,`translate`)
VALUES

That's not the conversion process. That's setting up the test environment... :)
At what point do you get the error message?

I'll update the comment above with the instructions.

Scott J’s picture

Thanks Joachim,
I get the error after step 1, but before I go any further I should probably just confirm that image_legacy module only needs the 3 files at http://drupal.org/cvs?commit=349214 ???

joachim’s picture

Aha. I see it now :)

Weird. By the time I've enabled devel module to debug it, the error message has gone!
I'm away for a few days, but will try and look more at this when I get back.

In the meantime, I don't think it affects the conversion.

quicksketch’s picture

Following, great work joachim.

joachim’s picture

Update:

- {file} table has been renamed. I've updated the code for image module to account for this. Here's a new SQL dump of dummy D6 import data with that change accounted for.
- the "image_get_info() (line 123" error is because the nodes are of type 'image'. Before conversion has run, these are not yet registered as node module nodes, so their base type is 'image', and NodeController's attachLoad() tries to run image_load() for them -- which on D7 is another function in image.inc. The conversion process takes care of changing the image node type to being handled by node module (ie like an admin-created type) and this warning then goes away. I'm not sure we can do anything about it prior to conversion.

jenyum’s picture

where is this image_legacy module of which you speak?

Scott J’s picture

libre fan’s picture

Hello everyone,

What's the situation like now Drupal7 has been released? Can we use the Field converter as is? -- though the version is rather old.

Cheers,

joachim’s picture

I've not tried it in ages.

If you have a use for it, please try it and report any problems in the issue queue :)

libre fan’s picture

Hello,

I upgraded to Drupal 7 and installed Field converter. When I open the page admin/content/field_convert, this is what I can see;

Article field demo
DEMO: converts an article field in a fictitious table to a numberfield.
This conversion plan has already run.

No mention of Images there.

joachim’s picture

You need the plan definition supplied by Image Legacy module -- it is up to each module to provide its own plan.

libre fan’s picture

You need the plan definition supplied by Image Legacy module

So, is this plan on this page? http://drupal.org/cvs?commit=349214

If so, I had a look at the links in the comment on April 1st, 2010:

Commit #349214 by joachim at 19:29
Image: /modules/image/image_legacy.field_convert.inc 1.1
Image: /modules/image/image_legacy.info 1.1
Image: /modules/image/image_legacy.module 1.1

The first link opens a page which shows a more recent file: Image: /modules/image/image_legacy.field_convert.inc 1.2
So I suppose this is the one.

The two other links seem to have no more recent updates.

Do we simply copy those files in the image module in your Drupal /sites/all/modules?

Well, I have so many issues since my upgrading to Drupal 7 that I'm not sure I'll succeed in the image conversion. I'll report sucess if any.
Cheers,

Scott J’s picture

libre fan,
Yes, make yourself a module by copying those 3 files into /sites/all/modules/image_legacy

joachim’s picture

Or just do a check-out of the 7 branch of image module... IIRC I've removed the actual image.module file so there won't be a clash with core.

libre fan’s picture

hello again,

Sorry to be that dumb. I made the image_legacy module, enable it in admin/.../modules, then clicked on Convert fields under the Field converter in /admin/index. What I get is the page /[drupal]/contributions/modules/image/image_legacy.field_convert.inc at the top of the web site, with all its content and buttons.

do a check-out of the 7 branch of image module

My problem is that I need first to have my Drupal6 images running in Dryupal7. I can see the nodes, but here's the error:

Notice: Undefined index: comment_node_image_form in drupal_retrieve_form() (line 736 of /.../includes/form.inc).
    Warning: call_user_func_array() [function.call-user-func-array]: First argument is expected to be a valid callback, 'comment_node_image_form' was given in drupal_retrieve_form() (line 771 of /.../includes/form.inc).

And if I click on the Edit tab, I get a blank page with just the node title. not edit the texts

joachim’s picture

> I made the image_legacy module

Don't understand what you mean by 'made'. Get a checkout with CVS.

> then clicked on Convert fields under the Field converter in /admin/index. What I get is the page /[drupal]/contributions/modules/image/image_legacy.field_convert.inc at the top of the web site, with all its content and buttons.

Don't understand what you are doing or getting here either.

> My problem is that I need first to have my Drupal6 images running in Dryupal7

I don't understand what you mean by this.

The procedure should be:

1. disable image module
2. upgrade Drupal core from 6 to 7
3. enable image_legacy
4. run conversion plan.

libre fan’s picture

Many thanks for your message Joachim.

I said "I made a module" just to echo your "make yourself a module by copying those 3 files into /sites/all/modules/image_legacy".

So I put the following CVS files in the rep:
/modules/image/image_legacy.field_convert.inc 1.2
/modules/image/image_legacy.info 1.1
/modules/image/image_legacy.module 1.1

I chose the more recent image_legacy.field_convert.inc 1.2 file and maybe that's wrong.

On my local install I used the three 1.1 files and though I got an error (which I may be able to correct), at least Field convertor showed the Image line which you can choose.

Okay I'll try again, and report success hopefully.

joachim’s picture

Best just use a full CVS checkout of image module to avoid any problems with versions!

julienbras’s picture

I'm trying to use this module to migrate from v6 to v7 the image content. I have successfully installed the image_legacy from cvs. But : it seems to migrate only 4 images on my 100 image node ?

And nothing is changed in my content.

When i run a second time the conversion, i've got an error :

An AJAX HTTP error occurred. HTTP Result Code: 500 Debugging information follows. Path: /batch?id=17&op=do StatusText: Service unavailable (with message) ResponseText: FieldException: Attempt to create field name <em class="placeholder">node_image</em> which already exists and is active. in field_create_field() (line 82 of /home/www/drupal7_root/modules/field/field.crud.inc).

As anybody have managed to migrate a live drupal6 to drupal7 with image content ?

joachim’s picture

> Attempt to create field name node_image which already exists and is active.

Yeah... running the plan a second time will do bad things like that :/
It's designed to run only once. Though we should maybe put in workarounds for while we're testing.

> it seems to migrate only 4 images on my 100 image node ?

Can you explain more?

julienbras’s picture

> it seems to migrate only 4 images on my 100 image node ?

Can you explain more?

I have retry the full migration from d6 (d6.20 and image 6.x-1.1). I have copied back my d6 DB ; and i have relaunch the core upgrade to d7. I add the 2 modules : image_legacy and field_convert.

I launch the image convert, and the blue bar says : Processed 4/4 and it go back to /admin/content/field_convert after the end of the migration (the migration seems very fast !! too fast ?)

The site have got about 100 image nodes.

Then I go to admin/content, and I see that 5 (yes, 5, not 4 !!) images have just been updated by the tool (update timestamp modified and new flag) ; but the node type is still blank. Others image have not been modified. Inside each 'old image node' i can't see any image. When I try to edit the node, i've got the following error :

    * Notice: Undefined index: image_node_form in drupal_retrieve_form() (line 736 of /home/www/drupal7_root/includes/form.inc).
    * Warning: call_user_func_array() [function.call-user-func-array]: First argument is expected to be a valid callback, 'image_node_form' was given in drupal_retrieve_form() (line 771 of /home/www/drupal7_root/includes/form.inc).

The tool have not created new content type (i think it's the case ?)

Mmmm... I don't think that everything is working well on my site :) I maybe will postpone my d7 site migration ;)

joachim’s picture

Ok confirming all those problems.

I don't know when I will next have time to look at this. This was all working before, so this should just be tweaks to account for things that have changed in core over the last few months. If anyone has time to look at any of this that would be great.

joachim’s picture

Here's the steps for using the plan:

A NOTE ON TERMINOLOGY:
- 'Image D6' means the project at http://drupal.org/project/image
- 'Core image' means the module named 'image' in core
- 'Image legacy D7' means the CVS HEAD checkout of the project at http://drupal.org/project/image. Within this project is a module named image_legacy.

These are all very different!

0. If you're setting up a test site
- install Drupal 6
- install Drupal 6 image
- create some image nodes

1. Upgrade Drupal core
- Follow the instructions in Drupal 7's UPGRADE.TXT file

2. Download and enable field_convert

3. Download Image legacy D7 (see note above), and enable image_legacy module within this package.

4. Enable Core image.

4 Go to admin/content/field_convert and run the plan.

julienbras’s picture

Joachim,

thanks on the time spend on that, I have followed your workflow (except that i've use my own D6 site to test the migration, which is containing some images).

I will try now to work on this issue, i've got some time but not a lot of xp in drupal api :)

Regards,

Taxoman’s picture

Subscribing.

Taxoman’s picture

Title: Image module » Image module migration
asb’s picture

sub

izmeez’s picture

subscribing

crispinbailey’s picture

subscribing 0_0

salvis’s picture

Priority: Normal » Critical

This blocks upgrading sites with images, even though Image is now in core.

joachim’s picture

> Image is now in core.

Please could everyone take note: Image is NOT IN CORE. Imagefield + imagecache are in core, and they've been renamed to 'image'.

julienbras’s picture

I know it's not for the image conversion ; but i am trying the "Article field demo" conversion, which is directly inside field_convert.

And i've got the same issue i've got on my image conversion :

Processed 4/4 and it go back to /admin/content/field_convert

The 2 tables have been created (drupal_field_data_article_number and drupal_field_revision_article_number but nothing inside); drupal_field_config have been alimented (one more row).

Sound not related to image_legacy module, but more on filed_convert. I'm on field_convert.admin.inc :)

joachim’s picture

> Processed 4/4

Yeah, I dunno why it's only doing 4.

julienbras’s picture

4/4 are 'operations of the batch', not images !

in field_convert.admin.inc, lign 133 :

    'operations' => array(
      // Each operation is an array of callback and arguments.
      array('_field_convert_batch_process_pre_batch', array($plan)),
      array('_field_convert_batch_process_create_bundles', array($plan)),
      array('_field_convert_batch_process_create_fields', array($plan)),
      array('_field_convert_batch_process_migrate_data', array($plan)),
    ),

I am working on that. The third first seems OK. I'm on the last one. (migrate_data)

drupal module are soooo modulable :)

joachim’s picture

Oh duh! Yes, of course.

Thanks for looking into this. Hopefully I put enough comments in the code that people can figure it out :D

julienbras’s picture

yep :)

joachim,

is the field_convert module expect that the image node type is present on a fresh d7 update from d6 ?

in field_convert.admin.inc, l 213 :

            case 'convert':
              // Convert a node type defined by hook_node_info() into a user-defined
              // node type.
              // We need to change:
              // - custom: from FALSE to FRUE
              // - locked: from TRUE to FALSE
              // - base:  to 'node_content'
              $query = db_update('node_type')
                ->fields(array(
                  'base'   => 'node_content',
                  'custom' => 1,
                  'locked' => 0,
                ))          
                ->condition('type', $bundle['type'])
                ->execute();
              break;

it's only an update of the node_type table

in image_legacy.field_convert.inc, l61 :

      // We need to create a node type.
      'bundles' => array(
        'image' => array(
          'status'  => 'convert', 
          // can be one of:
          // - 'create' -- TODO
          // - 'convert' -- for modules that used hook_node_info() on D6.
          'type'    => 'image',
        ),
      ),

That's strange ?

joachim’s picture

Hmm... what does the node_type table look like immediately after the core d7 upgrade?

The situation with the process is this: on D6, image's hook_node_info() implementation, image_node_info() defines the image node type. Once you upgrade to D7, Image D6 is gone and so image_node_info() is gone too. However, last time I worked on this whole thing, the situation at this point was that the node_type table held a record for the 'image' node type. So the 'convert' operation simply had to amend this entry in the table to convert it into a user-defined node type -- that is, the sort of node type that you create in the admin UI.

It may well be here that Drupal core, or the upgrade process has changed since I last looked at this.

joachim’s picture

Yay, my hunch was correct!

This is from the database dump of the site, immediately after the core upgrade:

INSERT INTO `node_type` (`type`,`name`,`base`,`description`,`help`,`has_title`,`title_label`,`custom`,`modified`,`locked`,`orig_type`,`module`,`disabled`)
VALUES
	('page','Page','node_content','A <em>page</em>, similar in form to a <em>story</em>, is a simple method for creating and displaying information that rarely changes, such as an \"About us\" section of a website. By default, a <em>page</em> entry does not allow visitor comments and is not featured on the site\'s initial home page.','',1,'Title',1,1,0,'page','node',0),
	('story','Story','node_content','A <em>story</em>, similar in form to a <em>page</em>, is ideal for creating and displaying content that informs or engages website visitors. Press releases, site announcements, and informal blog-like entries may all be created with a <em>story</em> entry. By default, a <em>story</em> entry is automatically featured on the site\'s initial home page, and provides the ability to post comments.','',1,'Title',1,1,0,'story','node',0);

There's no image node type left -- I expect a cache clear during the core upgrade process removes the node type.

So what needs doing is:

- finish the case 'create' at line 192.
- change the image migration plan 'status' => 'convert',
- add data to the plan so the case create has machine name, human label and description for the new type to create.

joachim’s picture

Ok I've just made CVS commits to both modules (FC and Image) and now the image node type gets created.

Images still aren't appearing though :/

joachim’s picture

I'm off to bed.

So far: node type gets created, field gets created.

Line 341 might be worth investigating:

  // Invoke the hook_field_convert_object_pre_load() on each object id.
  // Plan modules should use this if they need to prepare the data prior to 
  // loading it into the object.
  // Example: image module needs to clean up the corresponding entries in the {file} table.
  // WTF. WHY IS THIS NOT GETTING CALLED FFS???????????

Looks like I found a problem sometime last year but didn't get to the bottom of it ;)

For people testing this, I strongly recommend making a database dump with something like phpmyadmin or SequelPro immediately before running the migration plan, and then to test it again just wipe the database and reimport it.

joachim’s picture

> There's no image node type left -- I expect a cache clear during the core upgrade process removes the node type.

Ok that's a core bug and a critical one: #895014: All fields of a node type are lost on module disable.

But the changes I made yesterday can stand as a workaround for now.

julienbras’s picture

whooo....

such work during my night ! thanks joachim.

I have updated my install with your cvs update ; i work like you : the image node have been created ; but no image visible.

Second point, it work only with 5 nodes, not all of image nodes.

i suspect this :

  // Set the range for the query: start at our pointer and load 5 objects at a time.
  $start = $context['sandbox']['progress'];
  $limit = 5;

to work only once. Do you have more than 5 image node on your test install ?

Regards,

joachim’s picture

Chased the problem down to _field_convert_get_object_ids().

Update CVS for other minor fixes; debug with this code in the execPHP block:

module_load_include('inc', 'field_convert', 'field_convert.admin');  
$conversion_plans = _field_convert_get_plans();
$plan = $conversion_plans['image_d6'];
_field_convert_prepare_plan($plan);
dsm($plan);

$ids = _field_convert_get_object_ids($plan, 0, 5);
dsm($ids);
joachim’s picture

Gah I hate typos in variable names!

Okay, that part is fixed. Now we have a whole new bug!

PDOException: SQLSTATE[42S02]: Base table or view not found: 1146 Table 'drupal-fc-6up.file' doesn't exist: SELECT f.fid AS fid FROM {file} f INNER JOIN {image} i ON f.fid = i.fid WHERE (i.nid = :db_condition_placeholder_0) ; Array ( [:db_condition_placeholder_0] => 1 ) in image_legacy_field_convert_object_pre_load() (line 195 of /Users/joachim/Sites/fc/d7/sites/all/modules/image-HEAD/image_legacy.field_convert.inc).

joachim’s picture

Fixed that one too -- CVS update everybody ;)

Okay, bug is now in image_legacy_field_convert_object_pre_load().

I'm busy for the rest of the day. If anyone wants to dig further, try this debug code in execPHP:

module_load_include('inc', 'field_convert', 'field_convert.admin'); 
$conversion_plans = _field_convert_get_plans();
$plan = $conversion_plans['image_d6'];
_field_convert_prepare_plan($plan);
dsm($plan);

$ids = _field_convert_get_object_ids($plan, 0, 5);
dsm($ids);

// try just one node!
$id = array_shift($ids);
image_legacy_field_convert_object_pre_load($id, $plan);

We need to figure out why the call to file_load_multiple() in image_legacy_field_convert_object_pre_load() is not returning anything.

julienbras’s picture

joachim,

file_load_multiple() is not returning anything ; because we put inside fid from the {files} tables :

  // Get all files for this image node.
  $query = db_select('files', 'f');
  $query->join('image', 'i', 'f.fid = i.fid');
  $query->condition('i.nid', $id, '=')
    ->fields('f', array('fid'));
  $result = $query->execute();

  $fids = $result->fetchCol();  
  $files = file_load_multiple($fids);

And file_load_multiple() return only for files inside the {file_managed} table :) [I have just tested] And on my d7 updated from d6 with image ; there is NO image file inside {file_managed}. Only in {files} :)

Is there any other function available ?

joachim’s picture

Yup, changed all that to direct queries, since there's now the {file_managed} table.

Next, and hopefully final step: figure out why the data isn't getting saved into the field.

joachim’s picture

Ok folks.

The situation is now this:

The plan runs without error messages, re-creates the node type (workaround for critical core bug), creates the field, tidies up the file data.
However, the field data isn't getting saved.

What's currently driving me up the wall is figuring this out, as after running the plan, this code WORKS in the execphp block:

$plan = _field_convert_get_prepared_plan('image_d6');
dsm($plan);

$node = node_load(1);
_field_convert_manipulate_object($plan, $node);
node_save($node);

... why that works directly but not while the plan is running beats me. Anyone care to take a look at it?

joachim’s picture

Okay, the first 5 nodes seem to work now. The remaining bug is a batch API problem, probably because my callbacks don't have the $context as a parameter... anyone care to roll a quick patch? (As I am off to bed and have a pretty busy few days ahead...)

joachim’s picture

Should all work now. Am off to bed now, really!

Can someone test please?

julienbras’s picture

get the last cvs from both image_legacy & filed_convert ; get a 'good' dump of my d7 upgrade ....

launch the conversion ....

and that works !

All images node are now OK ; image visible...

Joachim, you really rock ! Thanks for this code update.

Regards,

joachim’s picture

Brill! Thanks for trying it out.

joachim’s picture

Oh just one thing -- did the image files stay in the same folder, or did they get moved to the base files directory? Not sure what happened with my test site. It may be there's a small bug either in the plan definition, or in the file migration part of the code.

julienbras’s picture

hum, i say same folder : /files/images/IMG_3980.JPG

no move to base file directory.

Regards,

joachim’s picture

Yup, I checked my file_managed table again and they were fine for me too. Not sure how my site ended up with copies of the images in base /files...

Thanks for checking! And hurrah, it all works again!!! :D

rondev’s picture

As the commit system changed from CVS to git on the 24th, I found the files here:
http://drupalcode.org/project/image.git/tree/refs/heads/master

rightchoice2c_me’s picture

what about images attached to nodes? I have upgraded almost everything from D6 to D7 except those images which were attached to nodes. Any suggestions would be greatly appreciated. Thanks

joachim’s picture

There isn't a plan for Image Attach yet -- there needs to be, and I don't have time to work on it at the moment. If you have time to work on this, open up a new issue for it and I'll try to at least give you pointers. It shouldn't be that hard though -- just migrate image attach data to nodereference.

rightchoice2c_me’s picture

Thanks joachim for your reply,
I'll give a try, but lil bit busy in some other stuffs. If I am able to accomplish it, will post it out in new issue.

asb’s picture

Just tried image-7.x-1.0-alpha1 from 2011-Mar-18 with field_convert-7.x-1.0-alpha1 on a small site; a new content type is created ("image"), and it shows an image (without any size options).

Editing the node basically works, but throws a number of errors:

Error message
Notice: Undefined index: description in field_multiple_value_form() (line 156 of /var/www/drupal/modules/field/field.form.inc).
Warning: strlen() expects parameter 1 to be string, array given in drupal_validate_utf8() (line 1503 of /var/www/drupal/includes/bootstrap.inc).
Notice: Array to string conversion in filter_xss() (line 1335 of /var/www/drupal/includes/common.inc).
Notice: Undefined index: required in field_multiple_value_form() (line 178 of /var/www/drupal/modules/field/field.form.inc).
Notice: Undefined index: required in field_multiple_value_form() (line 207 of /var/www/drupal/modules/field/field.form.inc).

Image Attach is really missing, the incomplete nodes look really sad ;)

Next I need to learn about Imagecache on D7. Geez, looks like the D7 migration will give us a taste for the ultimate nightmare. Thanks, Joachim, for your incredible work!

Greetings, -asb

joachim’s picture

> Image Attach is really missing, the incomplete nodes look really sad ;)

There's another issue on this module to write a migration plan for image attach -- just haven't had time to look at it. Be great if someone else could get the ball rolling!

crispinbailey’s picture

+1 for an Image Attach upgrade path. Is anybody working on this yet? Asking here because I couldn't find the other issue mentioned in @joachim's most recent post (#65)...

joachim’s picture

Oh, looks like there isn't one... oops.

Feel free to start work on it -- I haven't enough time at the moment to devote to this.

Scott J’s picture

In case it helps someone, may I mention that the D6 script at http://drupal.org/node/432860 is supposed to migrate image_attach to imagefield.

crispinbailey’s picture

Tried it - twice - but I can't get it to work.

Well, ~something~ worked, as I got the following confirmation message:

- content_type_imagepage populated.
- content_field_imagefield populated.
- files table updated
- 2033 image_attach relationships were migrated.

However, there are no images visible anywhere - either in the nodes with attached images or in the newly created nodes of the new content type. The only thing it seemed to do was create 2033 new nodes with no images. :(

Maybe something has changed in one of the modules since the script was written over 2 years ago...

joachim’s picture

joachim’s picture

... though the current workaround of creating a node type from scratch in the conversion plan works.

I've just run this and my image nodes got converted fine.

> - content_type_imagepage populated.

What produces that message? It's nowhere in the image or field convert code.

crispinbailey’s picture

That was the message produced when I ran the script in imagefield_migrate.php (found here: http://drupal.org/node/432860 ). The content_type_imagepage was the new content type I created as per the instructions in the prerequisites, which I followed to the letter.

Like I said, the new nodes were created - maintaining all info that was in the original Image nodes - but the actual images didn't seem to get associated with the new imagefield field, either in the new nodes or in the nodes with attached images. I looked at the source code verified that it wasn't a question of bad paths. It appears the imagefield fields just didn't get populated by the script.

joachim’s picture

The imagefield_migrate.php script is *nothing at all to do with this issue*.

See the instructions in comment #27 above.

crispinbailey’s picture

Sorry fur the confusion @joachim. I was just offering feedback in response to comment #68 - which suggests using that script - since we've been discussing the migration of attached images in this thread as well. Perhaps a separate issue should be logged to specifically address how to migrate attached images...? As I understand it that's still/currently not possible.

asb’s picture

The last time I tried a migration, it was not yet possible to get images attached to nodes via image_attach to show up on D7, yes. However, if D7 image nodes are actually created, it should be relatively easy to change the attachment mechanism, for example, to a node reference field plus something like a display formatter. At least that's what I'm hoping for.

Btw, is this issue actually "critical"?

joachim’s picture

> Perhaps a separate issue should be logged to specifically address how to migrate attached images...? As I understand it that's still/currently not possible.

Yup, a separate issue might be an idea. (We can accumulate twice as much tumbleweed... ;)

What's needed for image attach:

1. Decide on what to migrate to. References module? Something else?
2. Write the migration plan.
3. Test it.

> Btw, is this issue actually "critical"?

Hmm nobody using image can upgrade to 7 until this is sorted, so maybe?

crispinbailey’s picture

I've created a new issue for Attached Images here: http://drupal.org/node/1226062

Unfortunately I don't possess the knowledge or skill to tackle this myself, but hopefully someone else who needs it as much as I do can make it happen. I'm certainly more than happy to help test. :)

hd’s picture

I successfully performed the conversion and ended up with all the image nodes. So far so good and thanks for the tools and procedures provided.

However, the image derivatives have been dropped and even the derivative image files have been deleted. As mentioned way above in comment #1:

To do:
- convert old derivative sizes to new image styles.

I assume this is still on the todo list? Or is the there any magic I have overlooked to also get the derivatives back?

joachim’s picture

I've opened a separate task for this: #1482754: migrate presets to image styles. Any help would be welcome!

b0b’s picture

It went through and created all the image nodes for me. But it only converted about 6 of the image fields. I found the following error in the logs, which is referring to the next image to convert.

"The file sites/default/files/images/jamis-bam.thumbnail.jpg was not deleted, because it does not exist."

joachim’s picture

Thanks for reporting back.
Does that file exist or not?

b0b’s picture

The file did not exist. Looks like a case where the the image was added between pulling the files and the database down. I'll try it again.

moskito’s picture

I did the following steps:

1. Upgrade Drupal core
- Follow the instructions in Drupal 7's UPGRADE.TXT file

2. Download and enable field_convert (master)

3. Download Image legacy D7 (see note above), and enable image_legacy module within this package. (master)

4. Enable Core image.

4 Go to admin/content/field_convert and run the plan.

And got ther following error during the conversion:

An AJAX HTTP error occurred. HTTP Result Code: 500 Debugging information follows. Path: /batch?id=25&op=do StatusText: Service unavailable (with message) ResponseText: PDOException: SQLSTATE[23505]: Unique violation: 7 ERROR: duplicate key value violates unique constraint "image_pkey": UPDATE image SET fid=:db_update_placeholder_0 WHERE (nid = :db_condition_placeholder_0) ; Array ( [target] => default [return] => 2 [already_prepared] => 1 ) in image_legacy_field_convert_object_pre_load() (line 246 of DRUPAL_DIR/sites/all/modules/image/image_legacy.field_convert.inc).

I guess it is not related, but I run this site in postgreSQL DB.

I got a error in system update #7061 while upgrading, so I removed the duplicated entries in upload table as suggested in http://drupal.org/node/1172466 . Is it the cause of the convertion error?

joachim’s picture

It could be that issue, or a similar one with image module having a ton of duplicates kicking around from an old (pre-D5) site upgrade. There is an issue in the Image module queue about it.

joachim’s picture

I just tried this again with a fresh D6 and it migrated perfectly. My guess is that it's legacy issues from D5.

drzraf’s picture

created #1643396: image_attach D6 -> D7 upgrade which is about the "image_attach" table
(the field conversion as it is correctly took care of the upload table)

drzraf’s picture

finally found #1226062: Image_attach migration, sorry for the noise here

drzraf’s picture

But this conversion should not create an Image bundle of the node entity but rather use the Image bundle of the file entity.

joachim’s picture

> But this conversion should not create an Image bundle of the node entity but rather use the Image bundle of the file entity.

No, because the point is to keep the same structure that you had on D6. If you want a different structure... you can write a different migration path :)

radiobuzzer’s picture

Hi. I had some problems with this module and I suggested a fix at #1577648: Upgrade to D7 fails: PDOException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry . Some reviewing is needed, there doesn't seem to be any response yet

dgorton’s picture

Thanks much for all the great work.

I ran into a situation where about 20% of the image.module image source files ('_original' files) had differing values in the files.filename field (something other than '_original'). I'm not sure what caused that to happen, but I was able to solve that problem by running the below PHP on the source D6 site before migrating.

<?php
  $query = "SELECT * FROM {image} WHERE image_size = '_original'";
  $result = db_query($query);

  while ($row = db_fetch_array($result, MYSQL_ASSOC)) {
    $fid = $row['fid'];
    $nid = $row['nid'];
  
    $file_query = "SELECT * FROM {files} WHERE fid = " . $fid; 
    $file_result = db_query($file_query);     
    $image_file =  db_fetch_array($file_result);  
    if ( $image_file[2] != '_original') {
      watchdog("Image Fix", "Fixing image node %nid for fid %fid", array('%nid' => $nid, '%fid' => $fid), WATCHDOG_NOTICE, l('view', 'node/'.$nid));
      // Un-comment the next line to actually run the query!
      // $fix = db_query("UPDATE {files} SET filename = '_original' WHERE fid = " . $fid); 
    }
  }
?>

Hopefully that helps someone else as well.

dswier’s picture

Thank you so much dgorton. I had been lost as to why all our images were not converting and your script totally solved my problem.

Mokey’s picture

Can someone explain what:
"- 'Image legacy D7' means the CVS HEAD checkout of the project at http://drupal.org/project/image. Within this project is a module named image_legacy."
means? Specifically, what command do I use to check out the project? I have git, but don't use it regularly. Isn't CVS a different revision control system? I'm completely lost.

dswier’s picture

The instructions say CVS because they were not changed after drupal.org switched from CVS to git. All version control is done with git now, and here are some pretty clear instructions on how to check out the project http://drupal.org/node/3256/git-instructions/7.x-1.x

Mokey’s picture

Thank you, dswier. That was easy.

er.pushpinderrana’s picture

Issue summary: View changes

I followed your instructions and also migrated most of images data but one table is still not migrated that build by image_attach module.

D6Image Attach: Allows you to easily attach image nodes to other node types.

Can someone please suggest, Do I need to make separate script for this or I am missing something from above instructions.

jessZ’s picture

I am not clear on the instruction on where to get the "legacy" image module at the CVS head. I have never used Git. I tried installing the image-7.x-1.x from the all versions page:
https://www.drupal.org/node/3256/release

http://ftp.drupal.org/files/projects/image-7.x-1.x-dev.tar.gz

I tried to install this module on a 7.3 test site using the install from a url feature in D7.
I get the following error.

ImageMagick Advanced Options is already installed.

Do I have to install this manually?
"Within this project is a module named image_legacy."
MMH ok i've decompressed it.

image_legacy.info

image_legacy.field_convert.inc
image_legacy.module
Tried zipping them up and upload--doesn't work. Will upload via ftp.

OK now it shows up as a module: enabled

I go to test/admin/content/field_convert

check box doesn't work. If i click convert
Image module conversion to image field.
This conversion plan has already run.
may need to have some valid /dummy content. Now will try this on an actual D6> D7 upgrade site that has the tables.

OK now on site with real content. Checkbox works. click checkbox hit convert. Process starts.

Runs for 4 minutes.

Breaks all html inserted images that referenced file address of derivative images. But it does attach to the image to the node.
creates following error on editing image node.
Notice: Undefined index: required in field_default_form() (line 109 of /xxx/public_html/new/modules/field/field.form.inc).

Appears to have done a mass delete of all the derivative images. (preview, thumbnail etc except for those that are called .image.assist.properties.

Will look in actual image module to see how image thumbnail and preview creation is handled.

Image styles

Image styles commonly provide thumbnail sizes by scaling and cropping images, but can also add various effects before an image is displayed. When an image is displayed with a style, a new file is created and the original image is left unchanged.

Add style

Style name Settings Operations
Thumbnail (100x100) Default edit
Medium (220x220) Default edit
Large (480x480) Default edit

mmh. how to get this to recreate "preview, thumbnail, etc. enmasse?

Ok I am working with the content type image and changing it's display of fields.

Sweet, set field to diisplay medium image on node for full content and teaser and viola a lame taxonomy based gallery function with undistorted images. Most html links still broken but i can qorkaround that by replacing the images in the image directory with images pre-file conversion.

MMMh thefilepath for the created images is different styles/large/public/images/
so images embedded using the Wysywig image insert using html are looking in the wrong directory even if an appropriately named image file has been regenerated by the core image module.

It appears to keep the filename the same but save it to a new directory. In the original images directory it seems to have created a copy of each file called .img.assist.properties.jpg.
OK looking in the .inc I see that converting image derivatives to images styles is not supported in fact it looks like it purposely delets all the derivatives. but for some reason it created thumbnail derrivatives called.image.assist.properties.

well this isn't the most elegant solution but i simply swapped out the converted files/images directory with the pre-conversion back-up. All of the legacy html inserted paths appear to work and the new image nodes are decent looking and good enough and it appears that making new derivative images is extremely fast so i can tinker with those. A code warrior could improve the conversion script and i see that in the .inc that handling the derivative images is on the TODO list. but i'm going to call it a day.

Unfortunately a few tables got screwed up when attempting to migrate cck content (extra long textfields can apparently be fatal so if i want those back i have to reimport or roll back to the dbase upgrade step.
d6> d7 upgrade experience =AWFUL!!!! I hope d7>8 will finally get drupal into some sort of decent relationship to upgrading. It makes using even the most popular and well supported modules a real gamble.