This Cookbook shows, how you import one CSV file into a Drupal 7 site, creating new users, each with Profile2 fields.
The trick is to create two separate migrations from one source. The first creates the users, the second creates the profiles.
The second migration connects up the profiles it creates with the users that now exist by mapping the source unique key, MID, to the user uid. This is achieved by applying the line$this->addFieldMapping('uid', 'MID')
to the code.
Remark
The intention of this cookbook is, with detailed notes on this page and comments in the code to explain the function of each component of this migration as a working demo. For your first try use a fresh installed 'drupal 7 with default profile' and then start with step 1.
sites/all/modules and) enable these modules:
http://example.com/admin/structure/profiles':http://example.com/admin/structure/profiles/manage/memberlist/fields':
http://example.com/admin/people/permissions#module-profile2'
Create a containing directory for your module. For example, all later steps in this section assume that you're in the directory sites/all/modules/a_wusel_migration.
In your module directory, create the file a_wusel_migration.info containing the following:
name = "A Wusel Migration"
description = "Importing a CSV file as new users, each with Profile2 fields"
package = Migration
dependencies[] = migrate (>=2.8)
dependencies[] = migrate_extras
dependencies[] = profile2
; only for the 'Date'-field 'Birthday':
dependencies[] = date (>=2.8)
dependencies[] = date_api
; include the Classes file
files[] = a_wusel_migration.migrate.inc
; TIP: to show the migrate user interface delete the ";" at the beginning of the next line (or use drush)
;dependencies[] = migrate_ui
version = "7.x-1.2"
core = 7.x
Hints:
The line "dependencies[] = migrate (>=2.8)" enforces to have a module-version of the migrate module like 7.x-2.8 or newer (e.g. like 7.x-2.8+xx-dev).
Currently being tested with "7.x-2.8".
The line "dependencies[] = date (>=2.8)" enforces to have a module-version of the date module like 7.x-2.8 or newer.
Currently being tested with "7.x-2.8".
In your module directory, create the file a_wusel_migration.module[1].
/**
* @file
* THIS FILE INTENTIONALLY LEFT BLANK.
*
* Yes, there is no code in the .module file. Migrate operates almost entirely
* through classes, and by adding any files containing class definitions to the
* .info file, those files are automatically included only when the classes they
* contain are referenced. The one non-class piece you need to implement is
* hook_migrate_api(), but because .migrate.inc is registered using
* hook_hook_info, by defining your implementation of that hook in
* a_wusel_migration.migrate.inc, it is automatically invoked only when needed.
*/
This is all you require in your .module file. And you need this file!
In your module directory, create the file a_wusel_migration.migrate.inc, starting with the following code[1]:
/**
* @file
* Our own hook implementation.
*/
/**
* Implements hook_migrate_api()
*
* Returns 'api' => 2 for the 7.x-2.x branch of Migrate.
* Registers the migration classes for the 7.x-2.6 branch of Migrate (including
* 7.x-2.6+xx-dev).
*/
function a_wusel_migration_migrate_api() {
$api = array(
'api' => 2,
// Migrations can be organized into groups. The key used here will be the
// machine name of the group, which can be used in Drush:
// drush migrate-import --group=WuselMigrate
// The title is a required argument which is displayed for the group in the
// UI. You may also have additional arguments for any other data which is
// common to all migrations in the group.
'groups' => array(
'WuselMigrate' => array(
'title' => t('WuselMigrate Imports'),
),
),
// Here we register the individual migrations. The keys (Wusel_Step1_User,
// etc.) are the machine names of the migrations, and the class_name
// argument is required. The group_name is optional (defaulting to 'default')
// but specifying it is a best practice.
'migrations' => array(
'Wusel_Step1_User' => array(
'class_name' => 'Wusel_Step1_UserMigration',
'group_name' => 'WuselMigrate',
),
'Wusel_Step2_Memberlist' => array(
'class_name' => 'Wusel_Step2_MemberlistMigration',
'group_name' => 'WuselMigrate',
),
),
);
return $api;
}
Continuing with this same file, we then add an abstract class[1]:
/**
* Migration classes for migrating users and profiles
*
* based on: drupal.org/node/1269066#comment-4988994
* and: drupal.org/node/1190958#comment-4616032
*/
/**
* Abstract class as a base for all our migration classes
*/
abstract class Wusel_Basic_Migration extends Migration {
public function __construct($arguments) {
// Always call the parent constructor first for basic setup
parent::__construct($arguments);
// Avoid known line ending issue: "Invalid data value" at drupal.org/node/1152158#InvalidDataValue
ini_set('auto_detect_line_endings', TRUE);
}
}
You might find you don't need this abstract class, but it helps avoid a problem with line ending encodings that some people have reported. Basically, rather than using Migration as our base class below, we always use Wusel_Basic_Migration.
Continuing with this same file, we then add a class to migrate our users[1]:
/**
* User-only migration - not profile fields
*
* The data file is assumed to be in
* sites/all/modules/a_wusel_migration/data_sources/
*/
class Wusel_Step1_UserMigration extends Wusel_Basic_Migration {
public function __construct($arguments) {
parent::__construct($arguments);
$this->description = t('Import an CSV-file (only "Account"-fields)');
$columns = array(
// "Source": ('Fieldname', 'Description')
0 => array('MID', t('Member-ID (must be unique)')),
1 => array('mail', t('Email (Account)')),
2 => array('name', t('Name (Account)')),
3 => array('password', t('Password (Account)'))
);
// TIP: delete ", array('header_rows' => 1)" in the next line, if the CSV-file has NO header-line
$this->source = new MigrateSourceCSV(DRUPAL_ROOT . '/' . drupal_get_path('module', 'a_wusel_migration') . '/data_sources/drupaluser_import.csv', $columns, array('header_rows' => 1));
$this->destination = new MigrateDestinationUser();
$this->map = new MigrateSQLMap($this->machineName,
array('MID' => array( // this field is used to connect user und profile2
'type' => 'varchar',
'length' => 6,
'not null' => TRUE,
'description' => t('User\'s Member-ID') // description never used
)
),
MigrateDestinationUser::getKeySchema()
);
// Mapped fields
$this->addSimpleMappings(array('name'));
$this->addFieldMapping('mail', 'mail')
->defaultValue('')
->description(t('Email address'));
$this->addFieldMapping('init')
->defaultValue('')
->description(t('Email address used for initial account creation'));
$this->addFieldMapping('pass', 'password')
->defaultValue('asdf')
->description(t("User's password (plain text)"));
$this->addFieldMapping('is_new')
->defaultValue(TRUE)
->description(t('Build the new user (0|1)'));
$this->addFieldMapping('roles')
->defaultValue(DRUPAL_AUTHENTICATED_RID)
->description(DRUPAL_AUTHENTICATED_RID . t(' = "authenticated user"'));
$this->addFieldMapping('theme')
->defaultValue('')
->description(t("User's default theme"));
$this->addFieldMapping('signature')
->defaultValue('')
->description(t("User's signature"));
$this->addFieldMapping('signature_format')
->defaultValue('filtered_html')
->description(t('Which filter applies to this signature'));
$this->addFieldMapping('created')
->defaultValue(time())
->description(t('UNIX timestamp of user creation date'));
$this->addFieldMapping('access')
->defaultValue(0)
->description(t('UNIX timestamp for previous time user accessed the site'));
$this->addFieldMapping('login')
->defaultValue(0)
->description(t('UNIX timestamp for user\'s last login'));
$this->addFieldMapping('status')
->defaultValue(1)
->description(t('Whether the user is active(1) or blocked(0)'));
$this->addFieldMapping('timezone')
->defaultValue(t('Europe/London')) // 'America/Los_Angeles', 'Europe/Berlin', 'UTC', ... from drupal.org/node/714214
->description(t("User's time zone"));
$this->addFieldMapping('language')
->defaultValue(t('en')) // e.g.: 'en', 'fr', 'de', ...
->description(t("User's default language"));
$this->addFieldMapping('picture')
->defaultValue(0)
->description(t('Avatar of the user'));
// Other handlers
if (module_exists('path')) {
$this->addFieldMapping('path')
->defaultValue(NULL)
->description(t('Path alias'));
}
if (module_exists('pathauto')) {
$this->addFieldMapping('pathauto')
->defaultValue(1)
->description(t('Perform aliasing (set to 0 to prevent alias generation during migration)'));
}
// Unmapped destination fields
$this->addUnmigratedDestinations(array('role_names', 'data'));
}
}
Notice the use of addSimpleMappings: $this->addSimpleMappings(array('name'));
addSimpleMappings may be used when the source field and the destination field are assigned the same name/identifier. In cases where they have different names/identifiers you need to use addFieldMapping to map the source to the destination.
In the same file we also write a second class to migrate the Profile2 fields[1]:
/**
* Profile2 field migration
*
* The data file is assumed to be in
* sites/all/modules/a_wusel_migration/data_sources/
*/
class Wusel_Step2_MemberlistMigration extends Wusel_Basic_Migration {
public function __construct($arguments) {
parent::__construct($arguments);
global $user;
$this->description = t('Import an CSV-file with Profile2-fields ("memberlist"-fields)');
$columns = array(
// "Source": ('Fieldname', 'Description')
0 => array('MID', t('Member-ID (must be unique)')),
1 => array('mail', t('Email (Account)')),
2 => array('name', t('Name (Account)')),
3 => array('password', t('Password (Account)')),
4 => array('complete_name', t('Complete Name (for Memberlist)')),
5 => array('birthday', t('Birthday (for Memberlist)')),
6 => array('tel_1', t('Tel.#1 (for Memberlist)'))
);
// TIP: delete ", array('header_rows' => 1)" in the next line, if the CSV-file has NO header-line
$this->source = new MigrateSourceCSV(DRUPAL_ROOT . '/' . drupal_get_path('module', 'a_wusel_migration') . '/data_sources/drupaluser_import.csv', $columns, array('header_rows' => 1));
// Declare migration 'Wusel_Step1_User' a dependency in migration 'Wusel_Step2_Memberlist' to have them run in the right order, if needed:
$this->dependencies = array('Wusel_Step1_User');
$this->destination = new MigrateDestinationProfile2('memberlist'); // 'memberlist' is the "Machine name" of the profile2-"Profile type"
$this->map = new MigrateSQLMap($this->machineName,
array('MID' => array( // this field is used to connect user und profile2
'type' => 'varchar',
'length' => 6,
'not null' => TRUE,
'description' => t('User\'s Member-ID') // description never used
)
),
MigrateDestinationProfile2::getKeySchema()
);
$this->addFieldMapping('uid', 'MID') // Connecting the profile2 to the user using 'MID' - this row is "the trick"
->sourceMigration('Wusel_Step1_User') // If your user migration class was named 'MyUserMigration', the string is 'MyUser'
->description(t('The assignment of profile2-items to the respective user'));
// Mapped fields
$this->addFieldMapping('field_mnr', 'MID')
->defaultValue(0)
->description(t('The Member-ID (must be unique)'));
/* Delete this line, if you need the following:
$this->addFieldMapping('field_mnr:format')
->defaultValue('plain_text')
->description(t('The Text-Format of the Member-ID'));
/* */
$this->addFieldMapping('field_mnr:language')
->defaultValue('und')
->description(t('The language of the Member-ID<br />("und" = no language)'));
$this->addFieldMapping('field_username', 'name')
->defaultValue('')
->description(t('The login name'));
/* Delete this line, if you need the following:
$this->addFieldMapping('field_username:format')
->defaultValue('plain_text')
->description(t('The Text-Format of the login name'));
/* */
$this->addFieldMapping('field_username:language')
->defaultValue(t('en'))
->description(t('The language of the login name'));
$this->addFieldMapping('field_name', 'complete_name')
->defaultValue('')
->description(t('The complete name (for Memberlist)'));
/* Delete this line, if you need the following:
$this->addFieldMapping('field_name:format')
->defaultValue('plain_text')
->description(t('The Text-Format of the complete name'));
/* */
$this->addFieldMapping('field_name:language')
->defaultValue(t('en'))
->description(t('The language of the complete name'));
$this->addFieldMapping('field_birthday', 'birthday')
->defaultValue('') // empty = unknown
->callbacks(array($this, 'fixTimestamp'))
->description(t('The birthday (for Memberlist)') . '.<br />' . t('Format') . ': "YYYY-MM-DD" <br />' . t('or') . ' "MM/DD/YYYY" <br />' . t('or') . ' "DD.MM.YYYY"');
/* Delete this line, if you use version 7.x-2.6+24-dev of the module Date or newer
$this->addFieldMapping('field_birthday:timezone')
->defaultValue('UTC') // !!! NO time conversion !!!
->description(t('The timezone of the birthday field'));
$this->addFieldMapping('field_birthday:rrule')
->defaultValue(NULL)
->description(t('Rule string for a repeating date field [this field is not present]'));
$this->addFieldMapping('field_birthday:to')
->defaultValue(NULL)
->description(t('End date date'));
/* */
$this->addFieldMapping('field_tel_1', 'tel_1')
->defaultValue('')
->description(t('The main telephone-number (for Memberlist)'));
/* Delete this line, if you need the following:
$this->addFieldMapping('field_tel_1:format')
->defaultValue('plain_text')
->description(t('The Text-Format of the main telephone-number'));
/* */
$this->addFieldMapping('field_tel_1:language')
->defaultValue('und')
->description(t('The language of the main telephone-number<br />("und" = no language)'));
// Other handlers
/* Delete this line, if you need the following:
if (module_exists('path')) {
$this->addFieldMapping('path')
->defaultValue(NULL)
->description(t('Path alias'));
}
/* */
// some internal fields
$this->addFieldMapping('revision_uid')
->defaultValue($user->uid)
->description(t('The user ID of the user, who started the migration'));
$this->addFieldMapping('language')
->defaultValue(t('en'))
->description(t("The default language of the user (e.g. 'en', 'fr', 'de')"));
// Unmapped fields (this fields are in core and not needed as profile2-fields)
$this->addUnmigratedSources(array('mail', 'password'));
}
public function fixTimestamp($date) {
// enable empty (= unknown) birthday-string:
if (strlen($date) > 0) {
$date = substr($date, 0, 10) . 'T12:00:00'; // we add 'twelve o'clock in the daytime' for automatic compensation of a website time zone difference to UTC
}
return $date;
}
}
It is necessary to add a hook_disable() function to your module to deregister the Migration Classes when you uninstall your custom migration module using two Migration::deregisterMigration lines.
So along with your other module files, create a_wusel_migration.install, containing the following[1]:
/**
* @file
* Implements hook_disable().
*
* the migration module should deregister its migrations
*/
function a_wusel_migration_disable() {
// based on: drupal.org/node/1418350#comment-5557772
Migration::deregisterMigration('Wusel_Step1_User');
Migration::deregisterMigration('Wusel_Step2_Memberlist');
}
Create the directory "sites/all/modules/a_wusel_migration/data_sources/"[4].
If not present, download and install to your modules-path like sites/all/modules these modules:
Then enable your own module "A Wusel Migration"! Because of the dependencies[] in the .info file,
it should also automatically enable:
dependencies[] = migrate_ui' in the .info file)Your next steps are:
drush help --filter=migrate. After enabling the module, run drush mreg to register your Classes. Then check that they are displayed with drush mshttp://example.com/admin/content/migrate' and click the "Configure" option then the "Register statically-defined classes" button.Background operations' on 'http://example.com/admin/content/migrate/configure'. To enable running operations in the background with drush, (which is recommended), some configuration must be done on the server. See the documentation "Running imports and rollbacks from the UI via drush" on drupal.org.
Prepare the CSV file[2] and name it drupaluser_import.csv, e.g.:
"member_nr","email","username","password","complete_name","birthday","tel_1"
"1001","new.tester@example.com","new tester","test","New Tester","07/13/1999","00000 12345-213"
"1002","old.tester@example.com","old tester","test","Old Tester","12.07.1945","00000/54321-0"
"1003","another.tester@example.com","another tester","test","Another Tester","1985-07-11","00000 678901"
"1004","","incognito user","test","Incognito User","",""
This file shows the possible forms of the birthday field (ten or none signs!) for testing the import.
The allowed date formats in the CSV-file are:
"YYYY-MM-DD" or "MM/DD/YYYY" or "DD.MM.YYYY".
The delimiters are different for each format and have to be used properly!
This is only for the import!
Within e.g. a view, you can chose the format of the output.
The columns of this CSV file have to follow the order of both
$columns = array(
...
);-code-parts in the file 'a_wusel_migration.migrate.inc'!
Ideally you may use the same names for the source fields in the class arrays as the column labels in the csv. The labels you use for the columns in the csv source file are ignored during the import via the array('header_rows' => 1) attribute, they are purely for visual inspection. The classes assign names/identifiers to each field in the arrays - so in your arrays you can assign the source fields any arbitrary name, as long as your array follows the order of the columns in the csv from left to right.
Store the CSV-file drupaluser_import.csv from Step 4 in the place as specified (sites/all/modules/a_wusel_migration/data_sources/drupaluser_import.csv).
Clear your cache!
There are two alternatives for the rest of this step:
step 1 to create the user accounts:
drush mi Wusel_Step1_User
step 2 to create the profiles:
drush mi Wusel_Step2_Memberlist
hint: get a list of the registered Class Names with drush ms
If you named the Migration Class "Wusel_Step1_UserMigration" the class name will be "Wusel_Step1_User" (without "Migration").
Or:
If you have enabled the module migrate_ui:
Go to Administration > Content > Migrate and click on "WuselMigrate Imports" or go to http://example.com/admin/content/migrate/groups/WuselMigrate.
Check the box for the first migration Wusel_Step1_User, select "Import" or "Import immediately" and click on "Execute".
Check the box for the second migration Wusel_Step2_Memberlist, select "Import" or "Import immediately" and click on "Execute".
Visit 'http://example.com/admin/people'.
[1]: Some sub-notes:
[2]: Some sub-notes:
ini_set('auto_detect_line_endings', TRUE); [3]: You can translate this enabled module using the Locale module (in core). Or use 'Add a translation' (http://drupal.org/node/1524254).
[4]: It's convenient to have the sample data files (drupaluser_import.csv) in the module directory for the example module, but bad practice with real data. Don't try this at your real live server! Your source data should be outside of the webroot, and should not be anywhere where it may get committed into a revision control system.
[5]: To be able to store dates before "1901-12-14" (12/14/1901), never use the field type "Date (Unix timestamp)"!
Good luck!
This Cookbook shows, how you can add a help page to the module A Wusel Migration (http://drupal.org/node/1285276).
This Cookbook shows, how you can add a translation of the module A Wusel Migration (http://drupal.org/node/1285276).
This Cookbook shows, how you can avoid to import a user in case of errors in the e-mail address and then to add a notice in the drupal error
Drupal is a registered trademark of Dries Buytaert.
