The module is here - http://drupal.org/project/node_import

Unfortunately not all modules are supported (yet).
Have a look under "What works?" - http://drupal.org/project/node_import.

You must follow the quite clear on-screen menus of node import - here is a summary from a live example:

  1. http://www.example.com/en/admin/content/node_import/add - Next
  2. http://www.example.com/en/admin/content/node_import/add - Step 1 of 8 - choose the Content Type
  3. same url - Step 2 of 8 - upload your CSV file (or similar) - or choose your file from previous uploads
  4. same url - Step 3 of 8 - "Set file options" "Enter the file format options for the file." then check the sample data output further down the page
  5. same url - Step 4 of 8 - "Map file columns" "For each content type field, select the columns in the file that contain the data for this field." - the better your headings in the csv files match the field names used in Drupal the better - and remember Node Import does not import data for all modules yet, a bit frustrating - but you might just be the person to contribute a missing link here.
  6. same url - Step 5 of 8 - "Set import options" - may not apply to the data you want to import.
  7. same url - Step 6 of 8 - "Set default values" - may not apply to the data you want to import. The defaults may be fine. If you have CCK fields set as "required" you must provide a default here - whether or not your CSV file contains actual data - you could write here for example "to be added", etc. If you use lists in CCK to choose from you need to use one of the options from that list - as far as I remember. You will get a red warning if something is wrong.
  8. same url - Step 7 of 8 - "Preview import" - here you can see if your data is read correctly form the csv file. If you can see a problem in the sample data (such as special characters) you can correct the csv file locally on your machine and then just reload the corrected csv file using FTP to the right place on the server (../import) - this saves you going through the whole process again. Then just press "Realod" and check the sample data again. - You may ant to increase the number of sample data records extracted - just press "Relaod" again.
  9. same url - Step 8 of 8 - "Start import" - final chance to see you sample data further down - if all is clear hit "Start Import" - wait - well the process will run till it is finished. You can watch it like grass grwoing or do something else. The process will run indepently from you being logged on

If there are problems you can at the end of the import download a csv / excel file / open the problem data in Excel and check what is wrong.
You can then just correct these lines, create a new csv file - like "repaired ...csv" and just add these lines of data now.

That's it - I suggest you run a few tests with a sample / test content type and a test csv file with some 10-50 rows of content. Takes a little while to get it going - take notes along the way for your memory as creating the csv file via Excel and other helpers will involve quite a few steps.

I have attached a reduced sample filet UTF-8 with the following headers:
Authored by
Mall Image
Title
Name of Mall
Country
Location name
City
State / County
Street location
Additional
Postal code
Latitude
Longitude
..... more not included

Title,Name of Mall,Country,Location name,City,State / County,Street location,Additional,Postal code,Latitude,Longitude
Loop 5,Loop 5,Germany,,Weiterstadt,Hessen,,,64331,49.90778,8.59528
Kaufpark,Kaufpark,Germany,,Dresden,Sachsen,Dohnaer Straße 246,,1239,,
Einkaufscenter Marler Stern,Einkaufscenter Marler Stern,Germany,,Marl,Baden-Württemberg,,,74172,,
Einkaufszentrum Hamburger Straße,Einkaufszentrum Hamburger Straße,Germany,,Hamburg,Hamburg,,,22000,,
Ostsee Park,Ostsee Park,Germany,,Rostock-Sievershagen,Mecklenburg-Vorpommern,Ostsee-Park-Straße 3,,18069,,
Nordwest Zentrum,Nordwest Zentrum,Germany,,Frankfurt/Main,Hessen,Limescorso 8,,60439,,
Südring Center,Südring Center,Germany,,Groß-Machnow,Brandenburg,,,15806,,
Riesapark,Riesapark,Germany,,Riesa,Sachsen,Riesapark 2,,1587,,
Lausitz Park,Lausitz Park,Germany,,Cottbus,Brandenburg,Madlower Chaussee 4,,3058,,
Hürth Park,Hürth Park,Germany,,Hürth,Nordrhein-Westfalen,Theresienhöhe,,50354,,

Please note: the sample above contains special characters and to import these and other special characters into Drupal you need to make sure your source (import) file is converted to UTF-8. I produce these lists in Excel and save them as csv. Then I use a text editor like Node Tab Pro to save the file in UTF-8.
This is only necessary for files containing special characters. If you produce lists just in plain English you do not need to convert the csv file.

Comments

jainrutgers’s picture

I really appreciate the effort you took in writing this down.
The module is still in development but it worked out very well for me.
(Following the above guidance)

Thanks

joelbox-Mondial-IT’s picture

How would you define an image import in the csv, would that be an url to the image?

tryitonce’s picture

I think it would be - question is which URL - absolute or relative.

I suggest you play with a test file a bit - and please report back. I haven't done it, but when experimenting node_import "told" me that the image needs to be uploaded via FTP to the right location.

willhowlett’s picture

For file / image uploads you just need the filename in the csv file, then you need to make sure that all the files are in the folder that the content type is expecting them to be. Any that do not contain the file will not be accepted in the import

Lowell’s picture

I have tried over and over and keep getting the same error, "You must upload an image." I have put the image in the images folder exactly as the image content type does when I add an image node manually. I tried putting it in the imports folder defined for the node_import settings. I have tried relative and absolute urls to both locations. I just can't seem to get it to work.

Any ideas will be greatly appreciated.

willhowlett’s picture

When you say you've tried relative and absolute urls, in the CSV file you shouldn't include any URL, just the filename. Could that be the problem?

Then put the images in the images folder where the image content type is set to save them.

Lowell’s picture

I was thrown by a bug on the settings page where it showed a path "sites/default/files" then a text box where I entered "imports"

In fact I was actually using "sites/mysite.com/files" and had it set accordingly in the files settings...and I did find that node_import actually does use the "sites/mysite.com/files" location, so the bug is only on the settings page display.

The "imports" value I entered only tells node_import where to find the csv files. It actually looks for the images in the sites/mysite.com/files" location.

Bonus...node_import works with sub-folders and even writes the images to sub-folders including, and then imagecache works perfectly with them as well. Simply define the image name with the path in your csv and include a path field cck field, then using this field with file field path alias when importing.

I start with a directory structure of images in the files folder, and end up with the same directory structure of images in the images folder.

YAY!!!

This might be worth a tutorial on a prominent Drupal site page, I have seen endless requests for this functionality.

tryitonce’s picture

... feel free to add the tutorial it in this section - you could put it on the following page.
Thanks for working out this process and for sharing it. ......

jayb1’s picture

Everything seems fine with the product import until I get to the final step to actually start the import, (Step 8). Instead of what I would expect to see (the importing progress), I am instead taken back to step one and no data has imported. Any ideas? I really need to get this working ASAP. Are there any logs that I can reference to see what happened any why the data did not import. I am using version: node_import-6.x-1.0-rc4

tryitonce’s picture

In step 3 and 7 you should see the input data displayed below the screens of Node Import module.

In step 7 the should be no mistakes / errors. You need to see data especially in step 7. If there is no data you are trying to import data that is not importable - because the scripts for these do not exist.
Note: In step 3 you will see data as this is just showing you the file. In step 7 it is a "test" import.

I suggest you create a file with 10-15 records form the set you want to import. An set the "test" import number in step 7 to that number. The test import in step 7 will show you all the records and any errors.

Alternatively for gettting used to the module:
1. Create a Content Type with just a few records - 3 or 4 of different types - try to use some that are definetly covered by Import module.
2. Create a CSV file with 10 - 15 sets. Do put a deliberate error into one of the records (if you like).
3. Import this file an see how you can succeed.

Finally it took me a lot of try and error to get this going - once done it should like run fine. But you should remember not all data can be imported - check the list - ;-)

.... and do report back how you succeeded eventually.

el56’s picture

Hi,

Thanks for this guide. This, plus the online help have given what I needed except for one area: importing dates.

My CSV file (actually, I us the pipe character as a delimeter) includes a date field to be brought in into a custom CCK content type.
I know the name of the date field I want to map to, yet none of date fields show up in the mapping list in Step 4.

At the time of the import, the system complains about bad date parsing, but otherwise imports the pages successfully while and defaulting the date fields. Any suggestions?

tryitonce’s picture

You need to check that the module for which you want to import data is supported.
Unfortunately not all modules are supported (yet).

Have a look under "What works?" - http://drupal.org/project/node_import.

This would require someone to write a filter and add it to the module. I have no idea how that works.

If there is enough interest for module specific filters you might get some sponsorship or if your need is big enough you might write it yourself or get someone to write it.

It would be good to have an "exchange" on Drupal.org where people could offer to write / sponsor / support the writing of modules.

You could post a requirement under the paid service forum and might get a simple straight answer saving you a lot of time with a small fee.

Do report back how you resolved this - thanks

mwbyrd’s picture

Do I have to create the field names in a content type before importing?

Or does the import create the field names?

tryitonce’s picture

Field names are referring to the use of CCK.

You do not need to have field names to import nodes.

If you use CCK to organise your nodes with different fields then you have to define the fields in the Content Type first.

Otherwise you need to import the author, title and body. Some of these standard node elements can be imported without adding them to the import file if say the author is always the same person, e.g. "Webmaster" or so - same for post date / time and some more - just see as you work your way through your first test runs.

mwbyrd’s picture

Everything seems to be working. I seed the data and am able to import it. But I can't seem to access the nodes via Views.

I looked for the table in my database to see if I could find the records. (I didn't have much time before work). But I didn't see the data actually loaded in the DB.

I did find one table that was looking for to IDs. I'm wondering if Node_input creates the proper IDs or if drupal takes care of this?

Are there any step by step examples on the 'net - from start to finish (ie contenty type to displaying data)? I did a quick search without much luck.

Mike

mrgoltstein’s picture

I had the same problem, but the comment of razkovnik fixed it (change $args on line 1842 of node_import.inc to &$args).

mwbyrd’s picture

I was able to get it to work, but am now wondering if Ubercart is the best solution for what I want to do. I've been able to use Node_import to get teh data to appear as I want in the webpage. Now I just need to figure out how to assign the correct taxonomy so I can leverage Taxonomy_menu to create an easy to use Menu.

THanks,

Mike

sneta’s picture

I registered but I still can't see the first two step...

razkovnik’s picture

It helped me a lot while trying to import a massive database in Excell CSV format to Drupal CCK node.
I just had to make little fix in node_import.inc on line 1842 in order to fulfill the import process:
I changed
$form = call_user_func_array('drupal_retrieve_form', $args);
with
$form = call_user_func_array('drupal_retrieve_form', &$args);
because in STEP 7 I received an warning message like this - warning: Parameter 2 to drupal_retrieve_form() expected to be a reference...
But now it is working properly.
Thanks a lot!

firifiri’s picture

Hi, I'm getting a "HTTP Error 500 (Internal Server Error)" when I click the "Next" button in the "Set file options (step 3 of 8)".

I'm trying the import with a simple content type (just Title and Description) and a simple comma separated .csv file with 3 records (I tried also with the page content type, but the same error appears)

Do you have an idea of what can be wrong?

roknrod12’s picture

If you've already imported content and you import the same csv file, will the data in the db simply be updated? Anyone know?

tryitonce’s picture

... unfortunately and logically no.
The db has no knowledge of your csv import. So, if you reimport an updated version of the csv file (or the same for that matter) you will get new nodes.
Node ..../red-tractor will be added a second time as ..../red-tractor-1 - and so on.
So, if you need to import an updated csv fiel you need to delete the old nodes first - quite a problem if they have been updated as well.
My be there is a clever trick and I am as keen to find out - but this has been my experience so far.
...

roknrod12’s picture

I was afraid that was the case. Thanks for the reply. I'll have to figure out a way to do it directly on the db then. thanks, rh

tryitonce’s picture

.. yes, that seems to be the trick.
When done let us know how - thanks .....

tim@digitelly.com.au’s picture

I have only recently started using Drupal and this module has been a great find; I have a massive (and I mean massive) amount of data to import, some of it which needs to be displayed in tables. I have enabled the tablefield module which creates a table in the node; but this table can only be populated via .csv after the node has been created.

So...is there a way to populate a table using node import?
ie at the import stage the node is created and fills in fields as well as data which is displayed within a table.

I'm reasoning it can be done via css perhaps but would be fiddly, any brighter ideas.

cutitaru’s picture

I have the “jump from step 7 to step 0” problem when I try to import a .csv file for Ubercart.
The only time when I saw the step 8 was when I had a problem in the .csv file and I got this message at the end:
Status:
Completed
Rows imported:
0 rows
Rows with errors:
2 rows
It’s true that first time when I tried I had a problem with the images path so, after I solved that, when I entered step 7 everything looked fine, I could see all the products that are gonna be imported, but after I clicked next I jumped to step 0. After that I tried 100 different settings but I had the same result.
I’m a beginner in Drupal so maybe I’m doing something wrong but it would be really nice if somebody could help me.

cutitaru’s picture

i start using firefox instead of internet explorer and everything worked fine

tryitonce’s picture

good choice and thanks for reporting back.

mhalpern’s picture

So, I went to this very odd "mallsandmore" link and it said I had to register to view the page that had the steps. Seemed really odd, but I did it because I really need to do this. It took several hours but I got a link back to register on the site. So, I filled out all the info and after all that, I submitted it and got this back:

Fatal error: Call to undefined function date_content_is_empty() in /home/mallscom/public_html/sites/all/modules/cck/content.module on line 906

So, I'm starting to wonder is this site is legit or has been hacked. Has anyone else been able to get in to see the steps and if so, could someone just send me these magic steps? thanks.

willhowlett’s picture

Mallsandmore is just an example URL from the instructions. You should install the module on your site then follow the instructions replacing mallsandmore's URL with your own.

tryitonce’s picture

thanks willhowlett, sorry mhalpern,

I added this some time ago and forgot to replace the example from the live site with "www.example.com".

But then who would give "anyone" access to the admin section - ..../admin/content/.... .

I corrected the links above and hope you are finding your way to resolving your issues.

On the fatal error point - it seems you either caught the site during some routine upgrade or some cck related issue that has since been fixed.

Btw. seeing fatal error message will not give you any indication about a sites legitimacy. You either need to really understand what it means, make sure you are doing legit things and beware of sites that run rather smoothly.
The base for illegitimate activities is coming from sites pushing cookies, apps, java script, etc. down to your computer. So, check your cookie and other security settings.

All the best and don't forget to report back solving your problems back here.

Ahhh, one more thing - if you ever google for "fatal error" messages - remove the parts in the link that relate to your login details. Leave a plank there between the "/".
And please do not publish them - neither yours nor those of others.
.....