So, as promised over there... :-)

I have no easy access to Drush on that particular site and anyway, many potential users might not have it, either, so it would be very good to solve something as simple as file renaming within the confines of the module. I checked out how phpBB stores it's files and, it seems to me, it's not only a question of mangled filenames. The original files are longer than needed, when copied verbatim, they are corrupt (a PNG doesn't open as a PNG, for instance). The phpbb_attachments table also has a column of filesize, so, my guess is that the files, in addition to being renamed, should be truncated to that file size as well.

Iterating through a couple thousand files and renaming them, even copying with a specific file size is not a complicated task in itself but I don't really now the interaction with Migrate and were not willing to undertake it before hearing your opinion about it. But if you consider it really nothing more than a nice foreach instead of calling the drush script, I get down to work at once... :-)

CommentFileSizeAuthor
#2 phpbb3_core.inc_.patch3.89 KBdjg_tram

Comments

djg_tram’s picture

djg_tram commented

I started to go deep into Migrate. As far as I can see, it has no provisions to copy a file to a different name. I added a newFile field to MigrateDestinationFile but it is still a fixed, hardwired name, not the real_filename coming from the database, so all attachments would be copied to a single file. I don't yet know Migrate enough to see if the record field can be referenced here for the real name. And of course, that won't help with truncating the file (although, as un-Drupalish as it might sound, I'm perfectly willing to hack Migrate for this single project now).

Also, I hacked Migrate to drop the extra, deeply nested paths you also complain about in TODO's, the avatars went into the proper, shallow /pictures this time, however, the users no longer pick it up. Feels like stabbing in the dark, of course, but this is testing.

djg_tram’s picture

djg_tram commented
StatusFileSize
new phpbb3_core.inc_.patch3.89 KB

I'm ready to admit it's not a particularly nice one in full, it's more like food for thought. It solves two problems: uses complete() to post-process the files. This breaks rollback (only the file removal that can be done manually) but it does migrate. Also, I needed to put in a conversion for accented filenames, although this creates a dependency on PHP iconv. But there are also a couple of fixes about corrected setting of comment dates and similar.

Besides, I know this should be handled in your module, not hacking Migrate, but for my quick needs, this helped: file.inc, line 220:

$destination = $this->copyDestination . '/'. pathinfo($path, PATHINFO_FILENAME) .'.'. pathinfo($path, PATHINFO_EXTENSION);

But I don't want to say this is the proper way to do it. :-)

djg_tram’s picture

djg_tram commented
Status: Active » Needs review

IMPORTANT! There was a rename() or something in complete(). It has to replaced with:

$file = file_move($file, $real_path, FILE_EXISTS_REPLACE);

It took me several hours to pin it down, the simple rename() is not managed... :-(

darksnow’s picture

darksnow
Location Scotland
commented

It's being looked at here http://drupal.org/node/1240928

Ideally what I'd like to do is give the migration the source file name so it can pull the file from the correct location, and then during migration rename it to both strip the path and give it a valid name.

The bigger issue here is that without proper mime information, from the file suffix, a file field won't accept the attachement since it's not in the list of allowed extensions.

I do like the idea of renaming it in complete using the managed file API though. As you say, food for thought.

djg_tram’s picture

djg_tram commented

Yes, this seems to be a limitation in Migrate, they never thought of two ends of a file copy operation having different names. But anyway, with these hacks I could successfully port the forum in question, so thanks for your work, anyway. I'll return the favor when I get grown... :-)

darksnow’s picture

darksnow
Location Scotland
commented
Status: Needs review » Closed (won't fix)

Since this is a limitation of migrate, I'm going to close this as won't fix.

If it becomes an ongoing issue please add a bug report to the issue queue for the 7.x-2.x branch of phpbb2drupal and I'll use the ideas here as the basis of a potential work around.