Download & Extend

D5->D6 upgrade: file paths not updated

Project:Image
Version:6.x-1.0-beta4
Component:documentation
Category:bug report
Priority:normal
Assigned:Unassigned
Status:active

Issue Summary

Hi,

I'm testing (once more) an D5->D6 upgrade and ran into some trouble I haven't experienced in previous upgrade attempts with the same site (and prior versions of 'image.module' for D6, I think).

Configuration and paths:

D5 site (image.module 5.x-2.0-alpha4):

  • Physical image location on the server: /var/www/drupal/files/bilder/
  • ./admin/settings/file-system: "files"
  • ./admin/settings/image/nodes: Default image path: "bilder" (Subdirectory in the directory files); Derivates: "Original", "Thumbnail", "Preview", "Icon", "Default"

D6 site (image.module 6.x-1.0-beta4):

  • Physical image location on the server: /var/www/migration/sites/default/files/bilder/
  • ./admin/settings/file-system: "sites/default/files"
  • ./admin/settings/image/nodes: Default image path: "bilder"; Derivates: "Original", "Thumbnail", "Preview", "Icon", "Default"

Both image locations match the physical path where the images are stored. Both installations run separately and in parallel ("migration" is my sandbox to test the upgrade). It seems not to matter how the paths at ./admin/settings/image/nodes are configured, no images uploaded into D5 are displayed; newly uploaded image uploads show up immediately, as usual.

This is where images are located according to the database:

mysql> select * from files where fid='18502';
+-------+-----+-----------+----------------------------------+------------+----------+--------+-----------+
| fid   | uid | filename  | filepath                         | filemime   | filesize | status | timestamp |
+-------+-----+-----------+----------------------------------+------------+----------+--------+-----------+
| 18502 |  30 | _original | files/bilder/odyssey_5_cover.jpg | image/jpeg |    38636 |      1 |         0 |
+-------+-----+-----------+----------------------------------+------------+----------+--------+-----------+
1 row in set (0.00 sec)

However, newly uploaded images are stored in another location:

mysql> select * from files where fid='18507';
+-------+-----+-----------+----------------------------------------+------------+----------+--------+------------+
| fid   | uid | filename  | filepath                               | filemime   | filesize | status | timestamp  |
+-------+-----+-----------+----------------------------------------+------------+----------+--------+------------+
| 18507 |   1 | _original | sites/default/files/bilder/scanner.jpg | image/jpeg |    34088 |      1 | 1262447264 |
+-------+-----+-----------+----------------------------------------+------------+----------+--------+------------+
1 row in set (0.01 sec)

I tried to fix this by issuing the following command:

mysql> UPDATE files SET filepath = REPLACE(filepath, 'files/bilder', 'sites/default/files/bilder') WHERE filepath REGEXP '^files';

That gives me a better looking database (fid 18507 is a D6 upload, fid 18502 is a D5 upload):

mysql> select * from files where fid='18502';
+-------+-----+-----------+------------------------------------------------+------------+----------+--------+-----------+
| fid   | uid | filename  | filepath                                       | filemime   | filesize | status | timestamp |
+-------+-----+-----------+------------------------------------------------+------------+----------+--------+-----------+
| 18502 |  30 | _original | sites/default/files/bilder/odyssey_5_cover.jpg | image/jpeg |    38636 |      1 |         0 |
+-------+-----+-----------+------------------------------------------------+------------+----------+--------+-----------+
1 row in set (0.00 sec)

mysql> select * from files where fid='18507';
+-------+-----+-----------+----------------------------------------+------------+----------+--------+------------+
| fid   | uid | filename  | filepath                               | filemime   | filesize | status | timestamp  |
+-------+-----+-----------+----------------------------------------+------------+----------+--------+------------+
| 18507 |   1 | _original | sites/default/files/bilder/scanner.jpg | image/jpeg |    34088 |      1 | 1262447264 |
+-------+-----+-----------+----------------------------------------+------------+----------+--------+------------+
1 row in set (0.00 sec)

Also I tried to clear all caches through 'admin_menu' ("Flush all caches"). As far as I can see, now I'm getting images that were uploaded/imported as images to D5; however, images uploaded as attachments are still not being displayed. Also I'm seeing a size mismatch between the D5 and the D6 image directory:

D5 site:

# du -h /var/www/drupal/files/bilder/
4,0K    /var/www/drupal/files/bilder/temp
1,7G    /var/www/drupal/files/bilder/

D6 site:

# du -h /var/www/migration/sited/default/files/bilder/
4,0K    /var/www/migration/sited/default/files/bilder/temp
1,4G    /var/www/migration/sited/default/files/bilder/

This is quite strange; after re-copying the "bilder" folder again, the sizes. On D5, I'm using Imagecache to manage the deriavtes, maybe that explains the difference? OK, it looks better after regenerating the image derivates with VBO. Maybe this should be mentioned in the documentation?

Also I mentioned that some images were still not stored at ./sites/default/files/bilder, but at ./files/ (probably uploaded somehow differently than the rest):

mysql> select * from files where filepath REGEXP '^files/';
[...]
476 rows in set (0.00 sec)

I tried to fix this by moving *.jpg and *.png from ./files to ./sites/default/files/bilder and issuing those two statements:

mysql> UPDATE files SET filepath = REPLACE(filepath, 'files/images', 'sites/default/files/bilder') WHERE filepath REGEXP '^files';
Query OK, 9 rows affected (0.33 sec)
Rows matched: 476  Changed: 9  Warnings: 0

mysql> UPDATE files SET filepath = REPLACE(filepath, 'files/', 'sites/default/files/bilder/') WHERE filepath REGEXP '^files';
Query OK, 467 rows affected (0.14 sec)
Rows matched: 467  Changed: 467  Warnings: 0

So far my site looks fine; so this is mostly a support request with two remaining questions:

  1. Is this behaviour to be expected, especially are customized paths like "bilder" instead of "images" not handled automatically when updating the databse?
  2. Is my approach to replace file paths directly in the database the recommended way to fix issues like this? Are there other tables that would have to be taken care of, e.g. the 'upload' table, to avoid later inconsistencies?

I sincerely apologize if this is a duplicate; I searched quite a while and found lots of similar, but no identical isses.

Thanks for your time & work on the 'image' module,
Greetings -asb

Comments

#1

There's nothing in the current code that updates the filepath.
Shouldn't a site upgraded from 5 to 6 keep its files in /files rather than /sites/default/files?

#2

Status:active» closed (fixed)

Thanks, joachim;

if this is intended behaviour ("by design"), everything is fine. I "fixed" the filepaths on my remaining D5 sites as described above, which made the upgrades significantly smoother for me. You wouldn't believe what strange filepaths I found on the older sites (started on D4)... If fumbling around in the database in the described manner doesn't do any harm, I'm happy.

If /files should be used insted of /sites/default/files on D6 I don't know; I assume it's similar to contributed themes and modules which are supposed to be put in the /sites/all folder hierarchy. However, I'm not sure if image.module should necessarily care about such housekeeping (it sits on top of Drupal's file storage, so this might be a task for Drupal core or a contributed housekeeping.module. Hm, 'housekeeping.module', nice idea, though ;)

Closing this support request to shorten the issue queue.

Keep up the great work on image.module!

Thanks & greetings, -asb

#3

Component:image.module» documentation
Category:support request» bug report
Status:closed (fixed)» active

Is there documentation on what to do about file locations when upgrading?
At the very least, our upgrade notes should make it clear that file locations aren't handled.