This migration source plugin extends the Migrate Source CSV plugin to provide for a virtual key in the migration. The virtual key field SHOULD NOT exist in the source CSV file and is constructed by combining one or more columns from the CSV file.

Features

When a CSV file does not contain a unique column that can be used for the key, we would generally construct a composite key by creating a sequence of column names in the ids field of the migration YAML.
For e.g.
Suppose we have a CVS file that has the following columns:

"First Name","Middle Name","Last Name","Degree","Specialties","NPI"

We could define the key for a row based on the record_number_field value or something like this:

source:
  plugin: csv
  ids:
    - 'Last Name'
    - 'First Name'
    - NPI

The problems with this approach are:
1. In the migration lookup table, we have multiple source columns created to uniquely identify a row (in this case, 3 source columns). If we add to or remove a composite column from the ids, and run migrations again, it throws an error.
2. We cannot set a default value if a column is empty. In the example above, if NPI is empty, we cannot set it to 0

This module attempts to resolve the above shortcomings by:
1. Keep a single source key column in the migration lookup table. A virtual key is generated by combining columns.
2. A default value can be used in the key part if a column is empty

With this plugin, you can do this:

source:
  plugin: virtual_key_csv
  ids:
    - key
  virtual_key_fields:
    - 'Last Name'
    - 'First Name'
    - NPI
  virtual_key_default_values:
    NPI: 0
Supporting organizations: 
Provides development and maintenance time

Project information

Releases