Mass import/upload?

dirkson - August 8, 2008 - 15:36
Project:FileField
Version:6.x-3.x-dev
Component:Code
Category:support request
Priority:normal
Assigned:Unassigned
Status:closed
Description

We have something like 1,500 files that need importing. What's the sanest way to do that?

#1

drewish - August 8, 2008 - 16:27

use the batch api and to avoid timeouts. that's not shown

here's a reasonably generic function to create nodes:

<?php
function _import_create_node($node_type, $files, $title, $caption, $taxonomy) {
 
// For node_object_prepare()
 
module_load_include('inc', 'node', 'node.pages');

 
$node = new stdClass();
 
$node->type = $node_type;
 
$node->uid = $user->uid;
 
$node->name = $user->name;
 
$node->title = $title;
 
$node->body = '';
 
node_object_prepare($node);

 
$node->field_caption[0] = array('value' => $caption);
 
$node->taxonomy = $taxonomy;

 
// Get the filefield module to do the saving and marking the record as permanent.
 
foreach ($files as $field_name => $file) {
    if (isset(
$node->$field_name)) {
     
array_push($node->$field_name, $file);
    }
    else {
     
$node->$field_name = array(0 => $file);
    }
  }

 
$node = node_submit($node);
 
node_save($node);

  return
$node;
}
?>

this will help you figure out where the widget wants it's files stored (it should probably become a function in filefield.module):

<?php
/**
* Determine the widget's files directory
*
* @param $field CCK field
* @return files directory path.
*/
function _import_widget_files_directory($field) {
 
$widget_file_path = $field['widget']['file_path'];
  if (
module_exists('token')) {
    global
$user;
   
$widget_file_path = token_replace($widget_file_path, 'user', $user);
  }
  return
file_directory_path() .'/'. $widget_file_path;
}
?>

And here's an example of calling it you'll need to validate then save the files then plug them into the node (I'm doing multiple imagefields) this would be the part that would be a single batch api operation:

<?php
 
// For field_file_save_file()
 
module_load_include('inc', 'filefield', 'field_file');

 
// Loop over all the field/filepath pairs and create files objects.
 
$files = array();
  foreach (
$args['files'] as $field_name => $file_info) {
   
// Load the field and figure out the specific details.
   
$field = content_fields($field_name, $args['node_type']);
   
$validators = array_merge(filefield_widget_upload_validators($field), imagefield_widget_upload_validators($field));
   
$files_path = _import_widget_files_directory($field);

    if (!
$file = field_file_save_file($file_info['filepath'], $validators, $files_path)) {
      return
FALSE;
    }
   
$files[$field_name] = $file;
  }

 
// Create the node object.
 
$node = _import_create_node(
   
$args['node_type'],
   
$files,
   
$args['title'],
   
$args['caption'],
   
$args['taxonomy']
  );
?>

#2

drewish - August 8, 2008 - 16:28
Status:active» fixed

#3

dirkson - August 13, 2008 - 10:18

Awesome. That's a ton of info, and should serve me well, whenever I get around to actually pulling all these files in. Very high thanks!

#4

grandcat - August 23, 2008 - 08:58

Oh, great, also very high thanks. I can need your code to do quite a good integration of ImageField (based on FileField) in Image FUpload (mass uploading via Flash =)

#5

grandcat - August 24, 2008 - 13:09

<?php
// Load the field and figure out the specific details.
   
$field = content_fields($field_name, $args['node_type']);
   
$validators = array_merge(filefield_widget_upload_validators($field), imagefield_widget_upload_validators($field));
?>

I used this code to get the suitable validators for my imagefield, but I find it curious that it isn't tested if this is a real image file, not a fake like a renamed PDF, for example. But this seems to be a bug of imagefield, right?

#6

Anonymous (not verified) - September 11, 2008 - 05:32
Status:fixed» closed

Automatically closed -- issue fixed for two weeks with no activity.

#7

mecano - November 10, 2008 - 14:14
Title:Mass import/upload?» Mass import/upload? imagefield_widget_upload_validators gone?

Where is this imagefield_widget_upload_validators function gone? I have not been able to locate such function in last imagefield rev (imagefield-6.x-3.0-alpha2)!

#8

anantagati - November 14, 2008 - 21:01
Title:Mass import/upload? imagefield_widget_upload_validators gone?» Mass import/upload?

Thank you Drewish. With your help (from #1) was easy to make batch with image upload.

#9

trogie - February 16, 2009 - 11:07

I have been fighting a few days now to find a way to add imagefield files to existing nodes (thus editing). I try to load the node (node_load) add the file array (after the validators and other checking) and submit and save the node...

In fact I'm trying to move my img_attached images on nodes to imagefields without 'destroying' the original images.

#10

greg.harvey - March 21, 2009 - 23:11
Status:closed» active

What format should $files take? No matter what I throw at this, the result is the field's array in the node object is totally empty and no files are imported. I can't get this (or any other) approach to work. This *should* be simple, but for some reason I cannot make filefield data stick to a new node in a script at all. It's driving me crazy. =(

#11

drewish - March 24, 2009 - 23:17

greg.harvey $files is an array keyed by the field name with a file (object?) as the value.

#12

greg.harvey - March 29, 2009 - 08:08

Thanks drewish! Btw, is there a reference for the Drupal 6.x file object anywhere? I couldn't find one - had to attach a file to a node and vardump the node object to get the structure!

#13

quicksketch - March 29, 2009 - 08:11

greg.harvey, the $file object is (rather lamely) a direct mapping of the "files" database table.

fid
uid
filename
filepath
filemime
filesize
status
timestamp

#14

greg.harvey - March 29, 2009 - 08:44

Ah, that's straight-forward enough! Thanks, quicksketch! =)

#15

quicksketch - April 12, 2009 - 02:06
Status:active» closed

I'm moving this back to closed. I'd highly suggest taking a look at Image FUpload module, which has implemented this method and added a Flash uploading interface for bulk uploads.

#17

davebv - October 14, 2009 - 10:35

Is this code importing just one file to one field or are you able to import several files to one node with a filefield with multiple values?

 
 

Drupal is a registered trademark of Dries Buytaert.