How would you go about it?

What would be the best way to bulk import thousands of images to existing nodes using node_images?

Do I write a script that reads the files from the filesystem and then use all the node_images hooks to process it?

The nodes exists.
The images are stored in folders names similarly to the title of the nodes.
About 170 nodes and 4000 images.

I'm currently thinking of mapping all the image folders (their path) to a node id manually and put that in a CSV file. Then write a script a bit like http://drupal.org/node/67887 that chews through the CSV file, read all the images in the corresponding folder and run them through a customized function based on node image's _node_images_upload function?

Then I would get the sizes I want. In the folders I want and all that would be inserted into the node_images table without me having to worry about that part?

Comments

petterw’s picture

Status: Active » Closed (fixed)

This is how I solved it. Be aware, I don't really know what I am doing yet in terms of Drupal development so use at your own risk.


// The problem: A lot of images in named folders. The folders has the same name
// as the nodes they are to be "attached to" with node_images.
// I have also customised the node_images module but have tried to edit out 
// anything related to those customisations in this script but if it doesn't
// work for you, be aware. For example I have made node_images to save images
// in a subfolder with the node id. 

// Ideas on how to write one off scripts for drupal came from
// http://drupal.org/node/67887 
// so cred to Victor Kane - victorkane at awebfactory dot com dot ar
// and Jeff Vogelsang - jeffvogelsang at pqa dot com
// I'm at petterw at kenandi dot com

// CONFIGURATION
// Change the value below to TRUE when you want to run the script After running,
// immediately change back to FALSE in order to prevent accidentally executing
// this script twice.
// NOTE: I find the best way to do this is to leave it false. When you are ready
// to execute, set to this variable to true, and then click Preview. The script
// will execute, but will not be saved with the value set to true. This prevents
// accidentally saving the script with the value set to true. Also, it prevents
// the unfortunate situation where the cron job runs while it's set to true,
// and you get double imports.
$active = false;
// Set the folder with the images to import. I used a drupal subfolder, seemed easier so 
$import_folder = 'sites/all/files/import/a';
// Importing a lot of images can take a while, especially if they are big and needs to 
// be scalled, so set the PHP script time limit, how long the script can run before it is
// terminated.
set_time_limit (120);

// Bootstrap Drupal
include_once "includes/bootstrap.inc";
include_once("includes/common.inc");
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);

// Check for the script being enabled.
if ($active) {
       import_images($import_folder);
} else {
       print "Variable active is set to false.";
}

function import_images($import_folder) {
  // Get all the files and folders of the import folder
  $folders = scandir($import_folder);
  // Iterate through all the files and folders  
  foreach($folders as $folder){    
    $folder_path = $import_folder . '/' . $folder;
    // Make sure it is a directory and not a file
    if (is_dir($folder_path) && $folder != "." && $folder != "..") {
      // Check if there is a node named the same as the directory
      if($node = node_load(array(title => $folder))){
        // Start inserting all images of that folder
        insert_images($import_folder,$folder, $node);
      }
    }
  }
}
    
function insert_images ($import_folder, $folder, $node) {
    $output = '<ul>';
    $folder_path = $import_folder . '/' . $folder;

    if ($handle = opendir($folder_path)) {
        // Set the destination directory
        $dest = _node_images_get_directory($node);

        // Create the destination folders
        $directories = explode("/", $dest);
        $dirs = array();
        foreach($directories as $directory) {
          $dirs[] = $directory;
          if (!file_check_directory(implode("/", $dirs), 1)) {
            mkdir(implode("/", $dirs));
          }
        }

        $i = 0;
        // Go through each file in that folder
        // Parts taken from the node_images upload function
        while (false !== ($infile = readdir($handle))) { 
            if ($infile != "." && $infile != "..") { 
                $i++;
                $file = new stdClass();
                $file->filename = $infile;
            
                // Create temporary name/path for newly uploaded files.
                $file->filepath = $folder_path . '/' . $file->filename;
            
                // Scale image
                $file = _upload_image($file);
                
                // Validation from upload.module
                $user = 1;
                $fid = 'upload_'.$user;
                $node->files = array($fid => $file);
                _upload_validate($node);
                if (!isset($node->files[$fid])) return FALSE;
             
                // Move the file so if it the file is no longer in the import folder you can
                // assume it worked. Otherwise just use the drupal file_copy function 
                if (file_move($file, $dest)) {
//                if (file_copy($file, $dest)) {
                  $thumb = _node_images_create_thumbnail($file->filepath);
                  $file->filesize = filesize($file->filepath);
                  db_query("INSERT INTO {node_images} (nid, uid, filename, filepath, filemime, filesize, thumbpath, thumbsize, weight, description, page, position)
                           VALUES (%d, %d, '%s', '%s', '%s', %d, '%s', %d, %d, '%s', %d, %d)",
                  $node->nid, 1, $file->filename, $file->filepath, $file->filemime, $file->filesize,
                  $thumb->filepath, $thumb->filesize, 0, '', 0,0);
                }
            } 
        }
        $output .= '<li>' . $i . ' images for: ' . $folder . ' into nid: ' . $node->nid;
        closedir($handle);
    }
  $output .= '</ul>';
  print $output;
}

petterw’s picture

Just create a new page and put this in the body with input type set to php.