copy folder structure bug

+1 Subscribing

(Thanks StuartDH for pointing me towards this issue).

Consolidating with #1120136: Option to flatten or preserve file hierarchies on import - the broader issue is that both the file destination and the file field handler should allow you to specify whether or not to maintain the existing hierarchy. This will all be part of the effort in Migrate 2.4 to refactor file handling: #1240928: META: Refactoring of file destination/field handlers.

thank you Stuart for posting all the examples, it really helped me with my photo migration!

@StuartDH -- Any idea how to create filefield entities within another entity? I ask because I'm trying to migrate my existing folder structure with filefieldhandler (which doesn't seem to work, nor will seem to work for the foreseeable future), but can't create entities directly -- I'm using the field collection field handler to populate field collections, within which are my image fields.

Any thoughts? Thanks!

@Stuart -- Great, thanks for the help! I ended up doing that, then using phpThumb to manually resize all the images. Here's the code I used, for anyone else needing to do this before the shiny redone file handling comes out in 2.4:

<?php
private function resize_thumbs($path) {
		$origPath = '/home/aendrew/d7/sites/default/files/' . $path;
		$large_path  = '/home/aendrew/d7/sites/default/files/styles/large/public/sites/default/files/' . $path;
		if (!is_dir(dirname($large_path))){
			mkdir(dirname($large_path), 0777, true);
		}
		
		$medium_path = '/home/aendrew/d7/sites/default/files/styles/medium/public/sites/default/files/' . $path;
		if (!is_dir(dirname($medium_path))){
			mkdir(dirname($medium_path), 0777, true);
		}
		
		$square_thumbnail_path = '/home/aendrew/d7/sites/default/files/styles/square_thumbnail/public/sites/default/files/' . $path;
		if (!is_dir(dirname($square_thumbnail_path))){
			mkdir(dirname($square_thumbnail_path), 0777, true);
		}
				
		$thumbnail_path = '/home/aendrew/d7/sites/default/files/styles/thumbnail/public/sites/default/files/' . $path;
		if (!is_dir(dirname($thumbnail_path))){
			mkdir(dirname($thumbnail_path), 0777, true);
		}		

		$path_stuff = pathinfo($origPath);
		$accepted_exts = array('jpeg', 'jpg', 'png', 'gif');
                 if (isset($path_stuff['extension'])) {
			if (array_search(strtolower($path_stuff['extension']), $accepted_exts)) {
				$options = array('resizeUp' => false, 'preserveTransparency' => true);
				try {
					if (!file_exists($large_path) || !file_exists($medium_path) || !file_exists($thumbnail_path) || !file_exists($square_thumbnail_path)) {
						unset($image);
						watchdog('glegacy_migrate', 'Now on: %path', array('%path' => $origPath), WATCHDOG_NOTICE);					
						$image = PhpThumbFactory::create($origPath, $options);
						if (!file_exists($large_path)) {
							$image = PhpThumbFactory::create($origPath, $options);
							@$image->resize(480, 480)->save($large_path); //large
							unset($image);							
						}
						if (!file_exists($medium_path)) {					
							$image = PhpThumbFactory::create($origPath, $options);
							@$image->resize(220, 220)->save($medium_path); //medium
							unset($image);							
						}
						if (!file_exists($thumbnail_path)){				
							$image = PhpThumbFactory::create($origPath, $options);							
							@$image->resize(100, 100)->save($thumbnail_path); //thumbnail
							unset($image);							
						}
						if (!file_exists($square_thumbnail_path)) {						
							$image = PhpThumbFactory::create($origPath, $options);
							@$image->adaptiveResize(180, 180)->save($square_thumbnail_path); //square_thumb				
							unset($image);							
						}

					}
				}				
				catch (Exception $e) {
					self::displayMessage('Problem with image at ' . $origPath);
				}

		} else { //extension not present
			watchdog('thumbnail_migrate', 'Extensionless image at %path', array('%path' => $origPath), WATCHDOG_NOTICE);
			return FALSE;
		}
}

public function prepareRow($current_row) {
if (!file_exists('/home/aendrew/d7/sites/default/files/' . $current_row->file )) { // my DB stores file paths as directory1/directory2/4-digit-year/file.jpg
			return false;
		} else {
			$this->resize_thumbs($current_row->file);
			$current_row->file = 'sites/default/files/' . $path;
		}
}
?>

(Not visible here: I included ThumbLib.inc.php in the constructor so it's available later on.)

Hope this helps someone!

(Edit 2012-04-14: fixed typo and changed the thumbnail creation routine to redo the phpthumb create method for each -- otherwise, square thumbnails try to resize the 100x100 thumbs and looks awful)

re post #13
@aendrew - been looking for a resize image function to use with Migrate. Just want to check before I spend too much time...

is the method...

$this->resize_thumbs($current_row->file);

referring to...

private function resize_thumbnails($path)

if so - is this just a typo?

@James -- Yes, that's totally a typo. Fixed in the above code; thanks!

Importing an image from your oldsite (on a different server) is made really easy with phpThumb (thanks to @aendrew for introducing it to me)

I am pulling database information and files from www.djmag.com NOT /home/djmag/www

In the main ...NodeMigration class I have created an external connection (have a look at cross-database migrations)

$query = Database::getConnection('default', 'djmag')
		         ->select('magazine', 's');

So database stuff comes from this connection

But I want to pull files as well - so I have created this file mapping (see the beer.inc example for a better explanation)

$arguments = MigrateFileFieldHandler::arguments(NULL, 'file_copy', FILE_EXISTS_RENAME);
		$this->addFieldMapping('field_image', 'images')
		     ->arguments($arguments);

and then I have created these few lines of code in the prepareRow() function

public function prepareRow($current_row) {
        require_once '/path/to/phpThumb/ThumbLib.inc.php';
  //  $current_row->image_name is '3327.jpg' in this particular case (have a look at http://www.djmag.com/images/thumb/3327.jpg)
		$orig_path = "http://www.djmag.com/images/thumb/" . $current_row->image_name;
		$save_path = "/home/djmag/www/drupal-7.12/sites/default/files/cover/djmag/thumb/" . $current_row->image_name;
//OMG how simple is this code below!
		$image = PhpThumbFactory::create($orig_path)
			->save($save_path)
			;
//$current_row->images below refers to the second argument $this->addFieldMapping('field_image', 'images') in the main class above.
		$current_row->images = $save_path;
		
		RETURN TRUE;

}

I really hope this helps people who don't want to copy files (in bulk) from their existing server to their new server in order to effect a migration. But want to be able to do it selectively and on the fly. There maybe many reasons for this... a) there is no need to, b) you may still be using the old server whilst the switch is being made, c) you may want the same information on two different servers...

I am new to Drupal and programming so please forgive some of my terms, but I hope this helps - I couldn't find information anywhere about this.

By the way (and drush will remind you of this) if you are using Migrate to pull info off another server remember to have the necessary migration tables on that other server - 'migtate_map_...' and 'migrate_message_...'

Lastly - thank you so much @mikeryan for this wonderful module

.