Importing CSV files into Drupal

The node import was not ready yet for D6 so I went with Drupal 5. (though it will be ready shortly)

I had a content type with 30 or more fields.

These fields where a mix of CCK and Taxonomy - multi level.

The client handed me a large spread sheet and the first thing I did (after hours of doing it the wrong way) was made a quick mysql table to import it into. The client made Columns per Taxonomy and Sub for example 2 rows of content would look like.

Lost more fields but this is a good example.

eid was an external ID - CCK

Taxonomic_Group was three columns which I reduced to one via a mysql concat command (you could do this in the spreadsheet but I am more comfortable with mysql). Here is the query I used.

SELECT eid, CONCAT_WS(" | ", tax_group, tax_subgroup, genus, species) as tax,
`common_name`, CONCAT_WS(" | ", bio_tissue, bio_subtissue) as bio, CONCAT_WS(" | ", antomical_structure, antomical_sub_structure) as ant, `age`, `sex`, `condition`, `note`, CONCAT_WS(" | ", `mat_type`, `mat_sub`, `mat_sub_sub`) `coord`, `author1`, `author2`, `author3`, `author4`, `author5`, `author6`, `author7`, `author8`, `author9`, `author10`, `author11`, `author12`, `author13`, `author14`, `author15`, `year`, `journal_title`, `journal`, `volume`, `start_page`, `end_page`
FROM `biology_dept`

Common Name was just the title of the node.

Thing is node import would not import and create heirarchy taxonomy so I used Taxonomy CSV import.

To help clean things up when I messed up but did not want to do a full database recovery.

http://drupal.org/project/views_bulk_operations This creates and easy way to find and delete large amounts of nodes

http://drupal.org/project/taxonomy_manager This can help move taxonomy around or delete a quick amount of items.

Then of course mysqldump at the command line for quick backups.

Modules Used:

http://drupal.org/project/taxonomy_csv

http://drupal.org/project/taxonomy_manager