CSV files often contain a header row with fieldnames. Attached a patch that adds support for this. It does not count and process the first row if you specify in your migration class that there is a header row.

To use in your migration class, change from:
// Create a MigrateSource object, which manages retrieving the input data.
$this->source = new MigrateSourceCSV($this->csv_file(), $this->csv_columns(), array('delimiter' => "\t", ), $this->fields());
to:
// Create a MigrateSource object, which manages retrieving the input data.
$this->source = new MigrateSourceCSV($this->csv_file(), $this->csv_columns(), array('delimiter' => "\t", ), $this->fields(), $this->csv_with_header_row());

and add:
function csv_with_header_row() {
//return TRUE if first row of csv file
return TRUE;
}

CommentFileSizeAuthor
#2 csv_header_support.patch2.22 KBwouter99999

Comments

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Pittsfield, MA, USA
commented

Sounds promising, but there's no attachment - please try uploading again.

Thanks.

wouter99999’s picture

wouter99999 commented
StatusFileSize
new csv_header_support.patch2.22 KB

Oops! Attached this time...

stella’s picture

stella commented
Status: Needs review » Needs work

I couldn't get this to work. It still processed the header row of my csv file and in addition didn't adjust the number of records available to import when running "drush ms"

wouter99999’s picture

wouter99999 commented

Hi Stella,
strange... Could you open plugins/sources/csv.inc to see if the patch is correctly applied? did you add the extra value to your new MigrateSourceCSV() ? Note that the default is FALSE, so if you did not add the extra TRUE value, it will still take the first row.

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Pittsfield, MA, USA
commented

The option should be passed as an option (in $options, the third argument to the constructor) rather than a new constructor argument.

It'd be really nice to use the header row to initialize csvcolumns, so the calling migration doesn't have to explicitly do it.

Thanks.

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Pittsfield, MA, USA
commented
Status: Needs work » Fixed

I've committed my own version of the concept. Pass array('header_row'=>TRUE) as an option - the header will be skipped on import, and if you don't provide an explicit $csvcolumns list one will be generated from the header.

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Pittsfield, MA, USA
commented
Status: Fixed » Active

I'm going to expand this support a bit... I'm dealing with CSV files with a bunch of lines up front listing "search parameters" etc. So, rather than making 'header_row' a boolean, let's make 'header_rows' an integer which is the number of rows to skip, with the last "skipped" row used to generate the CSV columns when $csvcolumns is empty.

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Pittsfield, MA, USA
commented
Status: Active » Fixed

Committed. If you were already using the 'header_row' argument, you need to change it to 'header_rows', and if you had been passing TRUE it would be preferable to pass 1 instead.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.