Community & Support

Inserting large amount of data with php script

I want to make a list of movies, and I've created a CCK content type. I'd like to enter them in fairly quick, and the web interface takes too long. So I wanted to quickly add the whole list thru an sql query, but it seems to get it to show up properly in drupal, its a lot more complicated than that.

After a long evening of messing around and trying things, I've got my code down to this:

[code]

<?php
require_once "includes/bootstrap.inc";
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL); // makes ur script an offical Drupal script

// Sample line from list.txt:  Blade Runner;1982;http://www.imdb.com/title/tt0083658/;Widescreen
$list = file('list.txt');
$node = new stdClass();

foreach (
$list as $line_num => $line) {

  list(
$title, $year, $imdb, $vformat) = split(";", $line, 4);

 
$node->type = 'content_movie';
 
$node->uid = '1';
 
$node->title = $title;
//$node->field_format_value = $vformat;
//$node->field_format_0_value = 'DVD';
//$node->field_year = $year;
//$node->field_imdb = $imdb;
 
$node['field_format_value'] = array (0 => array('value' => $vformat,),);
 
$node['field_format_0_value'] = array (0 => array('value' => 'DVD',),);
 
$node['field_year'] = array (0 => array('value' => $year,),);
 
$node['field_imdb'] = array (0 => array('value' => $imdb,),);

   
//Make sure the data is ok and save him
   
node_validate($node);
    if (!
form_get_errors()) {
     
$node = node_submit($node);
     
node_save($node);

     
//Could this return be exiting the foreach loop? do I need it?
      //return $node->nid;
   
}
    else {
        return
false;
    }

}
?>
[/code]

I dont get an error on the page, as I'm previewing a page in the create content menu, but I'm getting this error in the admin page:
"Cannot use a scalar value as an array in /home/nahalla/public_html/includes/common.inc(1175) : eval()'d code on line 23."

The node fields with the array() tags was copied out of Victorkanes's code snippet http://groups.drupal.org/node/2197 . His code I think does what I want it to do, but I honestly cant make heads or tails of it, or figure out how to convert it to my cause. Running the code above will create my first node, but won't add any info to the CCK, and then it stops. And I know the foreach and list/strip code works, that was all working before I added the $node stuff.

Comments

Importing to CCK

I can't debug all your code. Here are some comments:

1. Look:

$node->title = ...;
$node['field_format_value'] = ...;

First you use object syntax, then you use array syntax. Bad.

2. Have you tried to import just _one_ node? with no loop? did it work?

3. You should move $node = new stdClass(); inside the loop, or else only one node will be created (save_node() creates a 'nid' field, so subsequent calls to it will keep _updating_ this one node).

4. Try removing the call to node_validate() and node_submit() (and form_get_errors()) -- they are not strictly necessary.

5. The 'field_format_value' and 'field_format_0_value' fields look suspicious. Do you really have such fields defined (sans the 'field_' prefix)?

6. You may want to publish the nodes with $node->status = 1;

Got it Working

1. Yeah, I was using the first syntax, but it wasnt working, so I switched to try victorkane's syntax.

2. Uhh... no I havent, that would have been a good idea, but I dont know the code to just read one line at a time.

3. I think moving the stdClass() inside the loop made it work. I've got all the nodes in the system now :D

5. When I was creating the fields in CCK, thats what it called the fields for whatever reason.

6. Ah! yes. that is a good idea.

So, I commented out node_validate and all that, and put the stdclass() inside the loop, and now it worked. But it was still unable to input the data into the CCK. My log is now flooded with "Invalid argument supplied for foreach() in /home/nahalla/public_html/modules/cck/content.module on line 386." errors. I'm going to input the data into the CCK through a seperate php script now, and not thru drupal.

thanks mooffie!

Edit: Got it working. I put it in by generating the mysql code:

$list = file('list.txt');
foreach ($list as $line_num => $line) {
  list($title, $year, $imdb, $vformat) = split(";", $line, 4);
  $sql = mysql_query("SELECT `nid`,`vid` FROM `node` WHERE `title` = '".$title."'");
  $row = mysql_fetch_array($sql);
  echo "INSERT INTO `node_content_movie` ( `vid`, `nid`, `field_format_value`, `field_format_0_value`, `field_year_value`, `field_imdb_value` ) VALUES ( ".$row['vid'].", ".$row['nid'].", '".$vformat."', 'DVD', ".$year.", '".$imdb."') ;";
  echo "<br>";
  mysql_free_result($sql);
}

And then put the generated mysql code in through phpmyadmin. I'm sure its possible to do this thru the php code in a drupal page, but I dont know how to use drupal's db features. The only problem at this point, is I that my text file had spaces in front of all the movies, and I didnt use trim, so I've got a space in all the node titles. I'm rectifying this now with some quick php code.

Nahallan.com
If you ever think you've hit rock bottom, get the f*** off my shoulders.
"Indeed, I am so far OUT of sane that you appear a tiny blip on the distant coast of sanity" - Bucky Katt

Nahallan.com
If you ever think you've hit rock bottom, get the f*** off my shoulders.
"Indeed, I am so far OUT of sane that you appear a tiny blip on the distant coast of sanity" - Bucky Katt

...

"Invalid argument supplied for foreach() in /home/nahalla/public_html/modules/cck/content.module on line 386."

You should have quoted the lines around #386. We made some silly error, that's all.

but I dont know how to use drupal's db features

http://api.drupal.org/api/4.7/group/database

Easier than directly using mysql_*.

[...] so I've got a space in all the node titles. I'm rectifying this now with some quick php code.

Or execute two SQL queries:

update node set title = trim(title);
update node_revisions set title = trim(title);

woah!

Holy crap! that's a LOT easier than what I was trying to do! (Especially since what I was trying was refusing to work) Thanks man!

Line 386 was:
      foreach ($node_field as $delta => $item) {
And somehow I think it was something I was doing, not the cck code :(

But this has solved all my problems, its working fine now. Thanks!

Nahallan.com
If you ever think you've hit rock bottom, get the f*** off my shoulders.
"Indeed, I am so far OUT of sane that you appear a tiny blip on the distant coast of sanity" - Bucky Katt

Nahallan.com
If you ever think you've hit rock bottom, get the f*** off my shoulders.
"Indeed, I am so far OUT of sane that you appear a tiny blip on the distant coast of sanity" - Bucky Katt

entire script?

Hey Interitus,
Can you post your entire, finalized script? I'm trying to do the same thing - enter in a lot of data from another homegrown CMS and enter each record from it into a new CCK type I created.

node_validate() kills select fields

Firstly, many thanks for the document.

I am using this logic to convert from one CCK type (application) to another (accepted show profile). What I am seeing is that form_validate() is deleting all of the select fields from my object, i.e. when I do something like

<?php
$node_profile
->field_genre=$node_appl->field_genre;
node_validate($node_profile);
?>

which works for text fields, the field_genre gets deleted by node_validate(). Does anyone have an idea why? If I comment out the node_validate, it works as expected.

subscribe

subscribe