This Cookbook shows, how you import one CSV file into a Drupal 7 site, creating new users, each with Profile2 fields.
The trick is to create two separate migrations from one source. The first creates the users, the second creates the profiles.
The second migration connects up the profiles it creates with the users that now exist by mapping the source unique key, MID, to the user uid. This is achieved by applying the line$this->addFieldMapping('uid', 'MID')
to the code.
Remark
The intention of this cookbook is, with detailed notes on this page and comments in the code to explain the function of each component of this migration as a working demo. For your first try use a fresh installed 'drupal 7 with default profile' and then start with step 1.
Cookbook:
Step 1: Before we start
- If not enabled, (download, install to your modules-path like
sites/all/modules
and) enable these modules:- Profile2
- Entity API
- (optional) Date and date_api, for the example below (it uses a Date field: "birthday")
- Add one 'profile type' in '
http://example.com/admin/structure/profiles
':
Label ='Memberlist' - Add some fields in '
http://example.com/admin/structure/profiles/manage/memberlist/fields
':- 'Text'-field: Label ='Member-ID', Field name = 'field_mnr', length = 6
- 'Text'-field: Label ='Username', Field name = 'field_username'
- 'Text'-field: Label ='Complete Name', Field name = 'field_name'
- 'Date'-field[5]: Label ='Birthday', Field name = 'field_birthday', Date attributes to collect: Year = yes, Month = yes, Day = yes, Hour = no, Minute = no, Second = no
- 'Text'-field: Label ='Tel. 1', Field name = 'field_tel_1'
- Check the permissions for 'Memberlist' on '
http://example.com/admin/people/permissions#module-profile2
'
Step 2: Build the import-module
Containing directory
Create a containing directory for your module. For example, all later steps in this section assume that you're in the directory sites/all/modules/a_wusel_migration
.
.info file
In your module directory, create the file a_wusel_migration.info
containing the following:
name = "A Wusel Migration"
description = "Importing a CSV file as new users, each with Profile2 fields"
package = Migration
dependencies[] = migrate (>=2.8)
dependencies[] = migrate_extras
dependencies[] = profile2
; only for the 'Date'-field 'Birthday':
dependencies[] = date (>=2.8)
dependencies[] = date_api
; include the Classes file
files[] = a_wusel_migration.migrate.inc
; TIP: to show the migrate user interface delete the ";" at the beginning of the next line (or use drush)
;dependencies[] = migrate_ui
version = "7.x-1.2"
core = 7.x
Hints:
The line "dependencies[] = migrate (>=2.8)
" enforces to have a module-version of the migrate module like 7.x-2.8
or newer (e.g. like 7.x-2.8+xx-dev
).
Currently being tested with "7.x-2.8
".
The line "dependencies[] = date (>=2.8)
" enforces to have a module-version of the date module like 7.x-2.8
or newer.
Currently being tested with "7.x-2.8
".
.module file
In your module directory, create the file a_wusel_migration.module
[1].
/**
* @file
* THIS FILE INTENTIONALLY LEFT BLANK.
*
* Yes, there is no code in the .module file. Migrate operates almost entirely
* through classes, and by adding any files containing class definitions to the
* .info file, those files are automatically included only when the classes they
* contain are referenced. The one non-class piece you need to implement is
* hook_migrate_api(), but because .migrate.inc is registered using
* hook_hook_info, by defining your implementation of that hook in
* a_wusel_migration.migrate.inc, it is automatically invoked only when needed.
*/
This is all you require in your .module file. And you need this file!
Classes file
In your module directory, create the file a_wusel_migration.migrate.inc
, starting with the following code[1]:
/**
* @file
* Our own hook implementation.
*/
/**
* Implements hook_migrate_api()
*
* Returns 'api' => 2 for the 7.x-2.x branch of Migrate.
* Registers the migration classes for the 7.x-2.6 branch of Migrate (including
* 7.x-2.6+xx-dev).
*/
function a_wusel_migration_migrate_api() {
$api = array(
'api' => 2,
// Migrations can be organized into groups. The key used here will be the
// machine name of the group, which can be used in Drush:
// drush migrate-import --group=WuselMigrate
// The title is a required argument which is displayed for the group in the
// UI. You may also have additional arguments for any other data which is
// common to all migrations in the group.
'groups' => array(
'WuselMigrate' => array(
'title' => t('WuselMigrate Imports'),
),
),
// Here we register the individual migrations. The keys (Wusel_Step1_User,
// etc.) are the machine names of the migrations, and the class_name
// argument is required. The group_name is optional (defaulting to 'default')
// but specifying it is a best practice.
'migrations' => array(
'Wusel_Step1_User' => array(
'class_name' => 'Wusel_Step1_UserMigration',
'group_name' => 'WuselMigrate',
),
'Wusel_Step2_Memberlist' => array(
'class_name' => 'Wusel_Step2_MemberlistMigration',
'group_name' => 'WuselMigrate',
),
),
);
return $api;
}
Continuing with this same file, we then add an abstract class[1]:
/**
* Migration classes for migrating users and profiles
*
* based on: drupal.org/node/1269066#comment-4988994
* and: drupal.org/node/1190958#comment-4616032
*/
/**
* Abstract class as a base for all our migration classes
*/
abstract class Wusel_Basic_Migration extends Migration {
public function __construct($arguments) {
// Always call the parent constructor first for basic setup
parent::__construct($arguments);
// Avoid known line ending issue: "Invalid data value" at drupal.org/node/1152158#InvalidDataValue
ini_set('auto_detect_line_endings', TRUE);
}
}
You might find you don't need this abstract class, but it helps avoid a problem with line ending encodings that some people have reported. Basically, rather than using Migration
as our base class below, we always use Wusel_Basic_Migration
.
Continuing with this same file, we then add a class to migrate our users[1]:
/**
* User-only migration - not profile fields
*
* The data file is assumed to be in
* sites/all/modules/a_wusel_migration/data_sources/
*/
class Wusel_Step1_UserMigration extends Wusel_Basic_Migration {
public function __construct($arguments) {
parent::__construct($arguments);
$this->description = t('Import an CSV-file (only "Account"-fields)');
$columns = array(
// "Source": ('Fieldname', 'Description')
0 => array('MID', t('Member-ID (must be unique)')),
1 => array('mail', t('Email (Account)')),
2 => array('name', t('Name (Account)')),
3 => array('password', t('Password (Account)'))
);
// TIP: delete ", array('header_rows' => 1)" in the next line, if the CSV-file has NO header-line
$this->source = new MigrateSourceCSV(DRUPAL_ROOT . '/' . drupal_get_path('module', 'a_wusel_migration') . '/data_sources/drupaluser_import.csv', $columns, array('header_rows' => 1));
$this->destination = new MigrateDestinationUser();
$this->map = new MigrateSQLMap($this->machineName,
array('MID' => array( // this field is used to connect user und profile2
'type' => 'varchar',
'length' => 6,
'not null' => TRUE,
'description' => t('User\'s Member-ID') // description never used
)
),
MigrateDestinationUser::getKeySchema()
);
// Mapped fields
$this->addSimpleMappings(array('name'));
$this->addFieldMapping('mail', 'mail')
->defaultValue('')
->description(t('Email address'));
$this->addFieldMapping('init')
->defaultValue('')
->description(t('Email address used for initial account creation'));
$this->addFieldMapping('pass', 'password')
->defaultValue('asdf')
->description(t("User's password (plain text)"));
$this->addFieldMapping('is_new')
->defaultValue(TRUE)
->description(t('Build the new user (0|1)'));
$this->addFieldMapping('roles')
->defaultValue(DRUPAL_AUTHENTICATED_RID)
->description(DRUPAL_AUTHENTICATED_RID . t(' = "authenticated user"'));
$this->addFieldMapping('theme')
->defaultValue('')
->description(t("User's default theme"));
$this->addFieldMapping('signature')
->defaultValue('')
->description(t("User's signature"));
$this->addFieldMapping('signature_format')
->defaultValue('filtered_html')
->description(t('Which filter applies to this signature'));
$this->addFieldMapping('created')
->defaultValue(time())
->description(t('UNIX timestamp of user creation date'));
$this->addFieldMapping('access')
->defaultValue(0)
->description(t('UNIX timestamp for previous time user accessed the site'));
$this->addFieldMapping('login')
->defaultValue(0)
->description(t('UNIX timestamp for user\'s last login'));
$this->addFieldMapping('status')
->defaultValue(1)
->description(t('Whether the user is active(1) or blocked(0)'));
$this->addFieldMapping('timezone')
->defaultValue(t('Europe/London')) // 'America/Los_Angeles', 'Europe/Berlin', 'UTC', ... from drupal.org/node/714214
->description(t("User's time zone"));
$this->addFieldMapping('language')
->defaultValue(t('en')) // e.g.: 'en', 'fr', 'de', ...
->description(t("User's default language"));
$this->addFieldMapping('picture')
->defaultValue(0)
->description(t('Avatar of the user'));
// Other handlers
if (module_exists('path')) {
$this->addFieldMapping('path')
->defaultValue(NULL)
->description(t('Path alias'));
}
if (module_exists('pathauto')) {
$this->addFieldMapping('pathauto')
->defaultValue(1)
->description(t('Perform aliasing (set to 0 to prevent alias generation during migration)'));
}
// Unmapped destination fields
$this->addUnmigratedDestinations(array('role_names', 'data'));
}
}
Notice the use of addSimpleMappings: $this->addSimpleMappings(array('name'));
addSimpleMappings may be used when the source field and the destination field are assigned the same name/identifier. In cases where they have different names/identifiers you need to use addFieldMapping to map the source to the destination.
In the same file we also write a second class to migrate the Profile2 fields[1]:
/**
* Profile2 field migration
*
* The data file is assumed to be in
* sites/all/modules/a_wusel_migration/data_sources/
*/
class Wusel_Step2_MemberlistMigration extends Wusel_Basic_Migration {
public function __construct($arguments) {
parent::__construct($arguments);
global $user;
$this->description = t('Import an CSV-file with Profile2-fields ("memberlist"-fields)');
$columns = array(
// "Source": ('Fieldname', 'Description')
0 => array('MID', t('Member-ID (must be unique)')),
1 => array('mail', t('Email (Account)')),
2 => array('name', t('Name (Account)')),
3 => array('password', t('Password (Account)')),
4 => array('complete_name', t('Complete Name (for Memberlist)')),
5 => array('birthday', t('Birthday (for Memberlist)')),
6 => array('tel_1', t('Tel.#1 (for Memberlist)'))
);
// TIP: delete ", array('header_rows' => 1)" in the next line, if the CSV-file has NO header-line
$this->source = new MigrateSourceCSV(DRUPAL_ROOT . '/' . drupal_get_path('module', 'a_wusel_migration') . '/data_sources/drupaluser_import.csv', $columns, array('header_rows' => 1));
// Declare migration 'Wusel_Step1_User' a dependency in migration 'Wusel_Step2_Memberlist' to have them run in the right order, if needed:
$this->dependencies = array('Wusel_Step1_User');
$this->destination = new MigrateDestinationProfile2('memberlist'); // 'memberlist' is the "Machine name" of the profile2-"Profile type"
$this->map = new MigrateSQLMap($this->machineName,
array('MID' => array( // this field is used to connect user und profile2
'type' => 'varchar',
'length' => 6,
'not null' => TRUE,
'description' => t('User\'s Member-ID') // description never used
)
),
MigrateDestinationProfile2::getKeySchema()
);
$this->addFieldMapping('uid', 'MID') // Connecting the profile2 to the user using 'MID' - this row is "the trick"
->sourceMigration('Wusel_Step1_User') // If your user migration class was named 'MyUserMigration', the string is 'MyUser'
->description(t('The assignment of profile2-items to the respective user'));
// Mapped fields
$this->addFieldMapping('field_mnr', 'MID')
->defaultValue(0)
->description(t('The Member-ID (must be unique)'));
/* Delete this line, if you need the following:
$this->addFieldMapping('field_mnr:format')
->defaultValue('plain_text')
->description(t('The Text-Format of the Member-ID'));
/* */
$this->addFieldMapping('field_mnr:language')
->defaultValue('und')
->description(t('The language of the Member-ID<br />("und" = no language)'));
$this->addFieldMapping('field_username', 'name')
->defaultValue('')
->description(t('The login name'));
/* Delete this line, if you need the following:
$this->addFieldMapping('field_username:format')
->defaultValue('plain_text')
->description(t('The Text-Format of the login name'));
/* */
$this->addFieldMapping('field_username:language')
->defaultValue(t('en'))
->description(t('The language of the login name'));
$this->addFieldMapping('field_name', 'complete_name')
->defaultValue('')
->description(t('The complete name (for Memberlist)'));
/* Delete this line, if you need the following:
$this->addFieldMapping('field_name:format')
->defaultValue('plain_text')
->description(t('The Text-Format of the complete name'));
/* */
$this->addFieldMapping('field_name:language')
->defaultValue(t('en'))
->description(t('The language of the complete name'));
$this->addFieldMapping('field_birthday', 'birthday')
->defaultValue('') // empty = unknown
->callbacks(array($this, 'fixTimestamp'))
->description(t('The birthday (for Memberlist)') . '.<br />' . t('Format') . ': "YYYY-MM-DD" <br />' . t('or') . ' "MM/DD/YYYY" <br />' . t('or') . ' "DD.MM.YYYY"');
/* Delete this line, if you use version 7.x-2.6+24-dev of the module Date or newer
$this->addFieldMapping('field_birthday:timezone')
->defaultValue('UTC') // !!! NO time conversion !!!
->description(t('The timezone of the birthday field'));
$this->addFieldMapping('field_birthday:rrule')
->defaultValue(NULL)
->description(t('Rule string for a repeating date field [this field is not present]'));
$this->addFieldMapping('field_birthday:to')
->defaultValue(NULL)
->description(t('End date date'));
/* */
$this->addFieldMapping('field_tel_1', 'tel_1')
->defaultValue('')
->description(t('The main telephone-number (for Memberlist)'));
/* Delete this line, if you need the following:
$this->addFieldMapping('field_tel_1:format')
->defaultValue('plain_text')
->description(t('The Text-Format of the main telephone-number'));
/* */
$this->addFieldMapping('field_tel_1:language')
->defaultValue('und')
->description(t('The language of the main telephone-number<br />("und" = no language)'));
// Other handlers
/* Delete this line, if you need the following:
if (module_exists('path')) {
$this->addFieldMapping('path')
->defaultValue(NULL)
->description(t('Path alias'));
}
/* */
// some internal fields
$this->addFieldMapping('revision_uid')
->defaultValue($user->uid)
->description(t('The user ID of the user, who started the migration'));
$this->addFieldMapping('language')
->defaultValue(t('en'))
->description(t("The default language of the user (e.g. 'en', 'fr', 'de')"));
// Unmapped fields (this fields are in core and not needed as profile2-fields)
$this->addUnmigratedSources(array('mail', 'password'));
}
public function fixTimestamp($date) {
// enable empty (= unknown) birthday-string:
if (strlen($date) > 0) {
$date = substr($date, 0, 10) . 'T12:00:00'; // we add 'twelve o'clock in the daytime' for automatic compensation of a website time zone difference to UTC
}
return $date;
}
}
.install file
It is necessary to add a hook_disable() function to your module to deregister the Migration Classes when you uninstall your custom migration module using two Migration::deregisterMigration
lines.
So along with your other module files, create a_wusel_migration.install
, containing the following[1]:
/**
* @file
* Implements hook_disable().
*
* the migration module should deregister its migrations
*/
function a_wusel_migration_disable() {
// based on: drupal.org/node/1418350#comment-5557772
Migration::deregisterMigration('Wusel_Step1_User');
Migration::deregisterMigration('Wusel_Step2_Memberlist');
}
Create the directory for the data file
Create the directory "sites/all/modules/a_wusel_migration/data_sources/"[4].
Step 3: Enable your module and the Migrate modules
If not present, download and install to your modules-path like sites/all/modules
these modules:
Then enable your own module "A Wusel Migration"! Because of the dependencies[]
in the .info
file,
it should also automatically enable:
- Migrate
- Migrate UI (only, if you have activated the line '
dependencies[] = migrate_ui
' in the .info file) - Migrate Extras
- Date Migration (for the Date field "Birthday")
Your next steps are:
- Clear your cache!
- Perform the registration
- It is recommended that you use Drush to perform your Migration functions. There is a list of Drush commands for Migrate in the Community Documentation, or you can get the latest list with
drush help --filter=migrate
. After enabling the module, rundrush mreg
to register your Classes. Then check that they are displayed withdrush ms
- If you must use the web admin, visit '
http://example.com/admin/content/migrate
' and click the "Configure" option then the "Register statically-defined classes" button. - If you want to import big CSV files, visit '
Background operations
' on 'http://example.com/admin/content/migrate/configure
'. To enable running operations in the background with drush, (which is recommended), some configuration must be done on the server. See the documentation "Running imports and rollbacks from the UI via drush" on drupal.org.
Step 4: Prepare the CSV file
Prepare the CSV file[2] and name it drupaluser_import.csv, e.g.:
"member_nr","email","username","password","complete_name","birthday","tel_1"
"1001","new.tester@example.com","new tester","test","New Tester","07/13/1999","00000 12345-213"
"1002","old.tester@example.com","old tester","test","Old Tester","12.07.1945","00000/54321-0"
"1003","another.tester@example.com","another tester","test","Another Tester","1985-07-11","00000 678901"
"1004","","incognito user","test","Incognito User","",""
This file shows the possible forms of the birthday field (ten or none signs!) for testing the import.
The allowed date formats in the CSV-file are:
"YYYY-MM-DD" or "MM/DD/YYYY" or "DD.MM.YYYY".
The delimiters are different for each format and have to be used properly!
This is only for the import!
Within e.g. a view, you can chose the format of the output.
The columns of this CSV file have to follow the order of both
$columns = array(
...
);
-code-parts in the file 'a_wusel_migration.migrate.inc
'!
Ideally you may use the same names for the source fields in the class arrays as the column labels in the csv. The labels you use for the columns in the csv source file are ignored during the import via the array('header_rows' => 1)
attribute, they are purely for visual inspection. The classes assign names/identifiers to each field in the arrays - so in your arrays you can assign the source fields any arbitrary name, as long as your array follows the order of the columns in the csv from left to right.
Step 5: Import
Store the CSV-file drupaluser_import.csv from Step 4 in the place as specified (sites/all/modules/a_wusel_migration/data_sources/drupaluser_import.csv).
Clear your cache!
There are two alternatives for the rest of this step:
a) Import with Drush (recommended)
step 1 to create the user accounts:
drush mi Wusel_Step1_User
step 2 to create the profiles:
drush mi Wusel_Step2_Memberlist
hint: get a list of the registered Class Names with drush ms
If you named the Migration Class "Wusel_Step1_UserMigration" the class name will be "Wusel_Step1_User" (without "Migration").
Or:
b) Import with the Migrate UI via the web admin
If you have enabled the module migrate_ui:
Go to Administration > Content > Migrate and click on "WuselMigrate Imports" or go to http://example.com/admin/content/migrate/groups/WuselMigrate
.
Check the box for the first migration Wusel_Step1_User, select "Import" or "Import immediately" and click on "Execute".
Check the box for the second migration Wusel_Step2_Memberlist, select "Import" or "Import immediately" and click on "Execute".
Step 6: Show the import
Visit 'http://example.com/admin/people
'.
Notes:
[1]: Some sub-notes:
- Include the line "<?php" only once in the very first line of this file!
- Don't include the line "?>" in this file!
- Here they are several times visible to format this page.
[2]: Some sub-notes:
- The CSV-file must have the ","-separator. The ";"-separator is not allowed!
- The CSV-file must be in "UTF8 with BOM"-format, then the import of special characters (letters like €, £, ß, ö ,ä, ü, Ö, Ä and Ü or other non-ASCII-signs) is without problems.
- If you don't include the line
ini_set('auto_detect_line_endings', TRUE);
in the classes:
The needed/correct line-ending-char(s) of a Feeds-imported CSV-file depends on the type of the operating system of the www-server:
If you are using a Linux-Server, use only LF at the line-end of the CSV-file.
If you are using a Windows-Server, use CR+LF at the line-end of the CSV-file.
If you are using a Mac-Server, use only CR at the line-end of the CSV-file.
The changing of the line-end of the CSV-file before importing is important, if the source of the CSV-file (e.g. your computer or the database of the CSV-file) has a different operating system!
For the meaning of LF and CR look at http://en.wikipedia.org/wiki/Newline#Representations - You can use a good editor like 'notepad++' on windows or 'LibreOffice Calc', both when indicated: '... Portable', (or use 'MS Excel') to change this.
Tip:
Use "Save as" and change the needed properties before and/or during saving the file ("before / during" is depending on the program used).
[3]: You can translate this enabled module using the Locale module (in core). Or use 'Add a translation' (http://drupal.org/node/1524254).
[4]: It's convenient to have the sample data files (drupaluser_import.csv) in the module directory for the example module, but bad practice with real data. Don't try this at your real live server! Your source data should be outside of the webroot, and should not be anywhere where it may get committed into a revision control system.
[5]: To be able to store dates before "1901-12-14" (12/14/1901), never use the field type "Date (Unix timestamp)
"!
Good luck!
Comments
Thank you
I have been struggling with this for a while now, and finally got it working this morning, just before this page went up. Thanks anyway!
edit: Oh, this is based on my forum post! *facepalm*
The other Andrew Morton
Module not working correctly?
Trying out this module approach but getting the following errors when I try to run the Wusel_Step_1 import:
Suggestions?
-Trevor
Processed message shows 0
I'm also getting this message when running the first import:
Processed 0 (0 created, 0 updated, 0 failed, 0 ignored) in 0 sec (0/min) - done with 'Wusel_Step1_User'
-Trevor
Thanks for your postings
@ambientdrup:
Your posting on December 1, 2011 at 10:53pm:
It was my error.
I add a new line 3 of "class Wusel_Step2_MemberlistMigration" as "
global $user;
", please look at the code above. Please add it in your code, too.Your other posting on December 1, 2011 at 11:06pm:
Is your CSV-file at the correct path? On the tab/page "Source" of "
http://example.com/admin/content/migrate/Wusel_Step1_User
", you can find your path to the CSV-file.And always clear your cache after putting this (e.g. new) CSV-file in the correct path.
If you are using a Linux-Server, please use only LF at the line-end of the CSV-file.
If you are using a Windows-Server, please use CR+LF at the line-end of the CSV-file.
The CSV-file must have the ","-separator. The ";"-separator is not allowed!
Wusel
How to check conditional statement
In case our csv file content have duplicate user (eg: by mistake to add same user details in mutiple times) how can i check to using conditional statement and where do write the code..
You can do it in a
You can do it in a prepareRow() public method:
Look to the general migrate documentation to find out more about this method.
checking conditional
when i add user name and other details it shows 'it already exist' and that user name does not saved in database but other field values stored. i wish when user name is same other field values do not have to store in database and also that user account do not have to create. how to write code for this condition?
Finally got it working!
Thank you soooooo much!
Any chance you could put together an "Update Users and Profile2 Info" cookbook?
-Brian Lewis-
@ModsUnraveled
ModulesUnraveled.com
This has been a LIFE SAVER
This has been a LIFE SAVER for a client job - thank you!!
How to display results
I tried this example and it worked. At least I can see the Import worked like it should and imported 3 things as in the csv. But where are the results? How do I display those results after the import?
2+ Types of Profile2 Profile Types
This code looks like it only works for 1 profile type, can you please mod it so that it works with 2 or more? Thanks
freedom isn't free
Your Cookbook (main steps)
1.) Copy the whole "class Wusel_Step2_MemberlistMigration" as "class Wusel_Step3_SecondProfile2Migration" to the bottom of the file "sites/all/modules/a_wusel_migration/a_wusel_migration.migrate.inc".
2.) Edit this new class to fit your second Profile2.
Here you have to change "memberlist" in the line "MigrateDestinationProfile2('memberlist')" to your second Profile2-Name.
The "$columns" and the "AddFieldMapping"s depend on the fields of your second Profile2.
3.) Add the new fields to your CSV-file.
Good luck!
Wusel
Also:
Also:
4) Add the new class to 'migrations' => array() in function a_wusel_migration_migrate_api()
Unfortunately the extra column I import in my 3rd class gets imported as blank. Can anyone link to how to debug the migrate module? Alas print_r($this->source) within the functions in the original tutorial doesn't show the CSV contents even within the 2nd class, which imports succesfully.
Migrate menu missing Wusel_Step1_User and Wusel_Step2_Memberlist
Thanks for the tutorial and code.
For some reason my http://mysite.com/admin/content/migrate does not display the Wusel_Step1_User and Wusel_Step2_Memberlist options.
Only the MigrateExampleProfile2 and Date migration options are displayed.
I have the a_wusel_migration directory/file structure as follows:
modules (folder)
--a_wusel_migration (folder)
----a_wusel_migration.inc
----a_wusel_migration.info
----a_wusel_migration.module
----data_source (folder)
------drupaluser_import.csv
I removed the end ?> from .inc and .module files
Modules installed: migrate, migrate_extras, migrate ui, migrate extra profile2 example, a_wusel_migration, profiles2, entity api, date, date migration, date api, etc...
Added profiles to the user data. The profile fields mirror the ones on the Drupal 6 site.
Clear cache several times.
Basically, I am trying to convert a Drupal 6 to 7 website that has a membership list with profiles. The Drupal 7 site is fresh and I want to do the membership migration before adding content to the site. This new site is being developed on a local wampserver.
Is there a reason that the Wusel_Step1_User and Wusel_Step2_Memberlist options do not display?
My tip
@TJM:
First implement this Cookbook without any changes on a test-drupal-system (Drupal core 7 without any other module and without any other theme).
Only when there are no errors, then change part for part, but only one part in every step. "Part" may be a value of the description above or an additional module or another theme. If this runs you may change the next part.
Good luck!
Wusel
Hi Wusel, Thanks for your
Hi Wusel,
Thanks for your continued support.
I did a fresh install of Drupal 7 and when through the files again. Guess what? The problem was a file name. As seen above in my post, the module file name is not correct.
Wrong: ----a_wusel_migration.module
Correct: ----a_wusel_migration.migrate.module
After the correction, Step 1 and 2 show up correctly. Now I can proceed with your module.
Sorry about my mistake,
Ted
Integrity constraint violation: 1048 Column _ cannot be null
I'm getting this error:
SQLSTATE[23000]: Integrity constraint violation: 1048 Column 'field_company_value' cannot be null: etc, etc, etc
I've triple checked everything and looked at migrate issues (where this error only seems to come up for some filefield... in my case it's a plain text field).
- Wondering if anyone has seen this error in relation to this cookbook and has any advice
- And Wusel, I'd be happy to talk about compensation for troubleshooting...
Whoops, I realize the field
Whoops, I realize the field throwing the error is in fact a computed field.
So now I have to figure how to migrate to a destination computed field... any advice, anyone?
Add profiles(CSV file) for users who are already in database?
My users are already in my database by upgrading from Drupal 6 to Drupal 7. However, this upgrading did not transfer the Profiles to the new Profile2. So, currently, I have a database with all the users and their passwords in my database. How should your code be modified to add the user's profiles?
Thanks,
Ted
_
I think your problem is explained on this section and this section provides information about importing user profiles from a CVS file to profile2 to use them in Drupal 7. Please read this section again and if it didn't work do you well, ask your question again with more details that members can help you better. However i am not sure if it helps you, you can enable profile module in Drupal 7 too by going to
yoursite.com/admin/modules
page.Please create an issue
The trick of this import is to create two separate migrations from one source. The first creates the users, the second creates the profiles.
I think you have to do both steps of this migration and not only step two.
Or you have to use another field to connect ("map") user und profile2. But I don't know the trick for you.
Please feel free to create an issue at http://drupal.org/node/add/project-issue/migrate or (look) at http://drupal.org/forum/37
This may help you!
Please report your solution with a new cookbook-page! Thanks from the other users, who have similar problems.
Wusel
I have moved on to other
I have moved on to other projects. I might let the users enter their own profiles again. Possibly, someone will come up with a simple solution for just transferring the profiles of Drupal 6 to profiles2 of Drupal 7.
I appreciate all of your contributions and assistance.
Thanks
Ted
Importing for existing users
I was able to import profile data for existing users by modifying these files only slightly:
Wusel_Step1_User' => array('class_name' => 'Wusel_Step1_UserMigration')
class Wusel_Step1_UserMigration extends Wusel_Basic_Migration
$this->dependencies = array('Wusel_Step1_User')
sourceMigration('Wusel_Step1_User')
$this->addFieldMapping('uid', 'MID')
, but change the field names to match what's in your destination and source fields.$this->addFieldMapping
to link the columns in your source table to the appropriate profile2 fields, and update the $this->addUnmigratedSources array to include all of the source columns you're not migrating.To migrate data from the same CSV into multiple profile types, as mentioned elsewhere in the comments, just create new classes like
Wusel_Step2_MemberlistMigration
and add them to the .inc file, and to the migrations array in the .module file. SinceWusel_Step1_User
andWusel_Step1_UserMigration
no longer exist, you might also want to renumber the remaining step(s), although it's probably not required.I don't understand the
I don't understand the following, can you give an example of what field names to give and what the import CSV should have for them?:
how about a taxonomy field?
Thx a lot!
i have a problem about importing taxonomy term.
CSV:
name,sex,phone
cobenash,F,0000
My problem is the data of sex in CSV is "F". However,the taxonomy tid in my drupal site is 7.
How can i get the value of the CSV and then use if~else if to put the correct tid in the field?
Adding roles
Thanks for the cookbook, its great
Can you help me with giving a role to the migrated user.
thanks
nitish
My tip
In the file a_wusel_migration.module change the line
to
if you want e.g. to add the role with RID = 4 to all users. This RID must exist!
You also can import the roles from the CSV file. Add a new column and assign it to the field 'roles'.
Note:
"
DRUPAL_AUTHENTICATED_RID
" has the RID = 2.You can view this at '
http://example.com/admin/content/migrate/Wusel_Step1_User
', click at the tab 'Mapping: Done'.Good luck!
Wusel
Thanks wusel
You really helped,
Now i have got into another problem.
I have added the birthday field in the registration form only. I have added your timefix function there also, But i am getting:
DateTime::__construct(): Failed to parse time string (1T12:00:00) at position 0 (1): Unexpected character (/var/www/fodler1/sites/all/modules/migrate/includes/base.inc:1019)
error could you help with this.
Solved it..
I solved it by keeping the mapped field in the same order in CSV as in code.
Field Collection
Thanks, this is great :)
I was wondering if I can map Profile2 fields that are "Field Collection" type.
Step 3 - Add field collection items
Guess what? I've just added a Step3 with successfully field collection migration.
This is what i've done:
1. Add this patch to field_collection module
http://drupal.org/node/1175082#comment-6763310
2. Add new class to the .migrate.inc developed on the current post
3. Import one or more childs to profile2. Cool or what? :)
thanks Wusel!
Showing CSV content
I did all the things said above. Data in csv file doesn't store in database but table is created, and I can't view the data in my site. How can i show data stored in csv file on drupal site?
I done this. I forget to
I done this. I forget to execute the import.
Thanks for your post.
Date format
i did to create user profile from import data from csv file. The date format is storied in MM.DD.YYYY but i want to store this format DD.MM.YYYY. how can i write the condition code in this format
Hint
The allowed date formats in the CSV-file are:
"YYYY-MM-DD" or "MM/DD/YYYY" or "DD.MM.YYYY".
The delimiters are different for each format and have to be used properly!
You have to change "MM.DD.YYYY" to "MM/DD/YYYY".
This is only for the import!
Within e.g. a view, you can chose the format of the output.
You can read this in the "class Wusel_Step2_MemberlistMigration" in the third part of the file "a_wusel_migration.migrate.inc" for the field 'field_birthday'
or after enabling the module "A Wusel Migration" in the UI.
Good luck!
Wusel
Thank u
thank your for your replay
Field Collections Import Error
I keep getting an error returned when I add a new class for field collections as per this post:
"Missing bundle property on entity of type profile2. (../includes/common.inc:7693)"
This is the relevant line in common.inc
I can only assume this is related to the function 'MigrateDestinationFieldCollection' as the error is not returned if I remove this section of code.
Any suggestions welcome...?
Thanks
UPDATE
Managed to resolve the issue.
I was calling the wrong class for the sourceMigration.
Error 500
I seem to be getting an error when clicking on the "Execute" button. I've checked every line, twice! I've even stripped out my code to just have Step 1, with no solution in sight.
I just cannot see what is wrong. No errors are in the error logs or on screen. I'm at the end of my tether, having spent all day, and would really appreciate a pointer as to where to start!
Using Migrate 7.x-2.x-dev - But the same on 7.x-2.5
how to import using postal address uing csv file
i imported user details from csv file using migrate module. in my site i'm using address field module, and i wish to import postal address for each user from cvs file to address field. how can i do this?
My tipp
Please create an issue in the address field module issue queue.
Good luck.
Wusel
import using postal address field
i do like this format to importing address field and its work. This is the csv source to mapping used in array
i used to following this code to map csv file to postal address field using addFieldMapping
generate password randomly
i'm using migrate module. when i import a cvs file user account created successfully. i want to generate password randomly for each user automatically when import cvs file. how can i do this?
generate password with prepareRow
Hi.
The solution is to use the prepareRow function.
Let's assume that you have an CSV column with the title 'password', where you can have pre-generated passwords OR you can leave empty cells and let you migration code create random ones.
In you mapping fields, you have something like this inside __construct():
After you close __construct(), and still inside the class, you can add something like this:
This function will analyze every row.
It's very handy! :)
NULL value couldnt to store the database
i'm using migrate module. when i import a cvs file profile2 fields created successfully. In my csv file have some filed to import to create profile field. if any one column is NULL the database as store blank space and its display on only field name. In this situation i want to omit that data storing in database and also do not display on my profile view page how can i do this
Great Work....
One question however.
I have gotten this to work as expected with text fields. Having a slight issue getting the date field to work but am more interested in how to go about incorporating 'List (text)' fields?
Any help appreciated
My tip
For your 'date' field look at https://drupal.org/node/1477602 Step 1 No. 3 and 4 ("Time zone handling" to "No time zone conversion") and then at https://drupal.org/node/1477602#note1 and my hints at "Step 4: Prepare the CSV file" on this page.
If you want to store "date and time" information e.g. in the field 'Birthday', delete the line
->callbacks(array($this, 'fixTimestamp'))
and delete the line
/* Delete this line, if you use version 7.x-2.6+24-dev of the module Date or newer
and use the newest dev-version of the date module. Now you should be able (not tested by me!) to import the timezone of that field using an extra field of your CSV-file (you have to add some code!).
While building the 'List (text)' field, you have typed in the "Allowed values list", one value per line, in the format "
key|label
". Within the migration you have to import only the "key
"-value from the CSV-file. It may be that the keys must exist before the import.Good luck!
Wusel
Thanks for the response.
Thanks for the response. After much testing it turned out it was simply the date format on the .csv file that was causing a problem.
Managed to get the List(text) fields working though!
Thanks again.
Entity Reference Fields?!?
Is it possible to import these?
Thanks
Fields with multiple Values
Please can someone possible help?
I am trying to import data into a field that can contains more than one value.
i.e. A List(txt) field with a number of results in the drop down. Users can choose more than one item from this list.
A/ How do I write this into a class to map the fields
B/ What is the formatting of the .csv file to allow for multiple values?
Hopefully this makes some sort of sense?
Thanks
oop code issue
i found the basic routines worked with this change to them.
..and thanks for the example. a good base.
Existing users....
Hi,
Once again, this cookbook has proved invaluable for user migration including profile2 fields, so big thank you.
However, I would like to know how I can import profile 2 data to existing users via this method?
I'm guessingI need to remove the dependancy on the 'MID' but my code-fu is not good enough to quite suss this out.
Any advice please?
FYI, in case you want to use
FYI, in case you want to use Feeds to import your data, there's now a module called Feeds Profile2 which extends the built-in User processor so that Profile2 profiles can be created and updated as part of a user importer.
Migrating Address Fields
Hi all,
I am having a problem migrating address fields. Everything else in my migration works, but address field simply is blank. I have searched for many hours and havent found a definitive answer that works. I am using Drupal 7.37, Migrate 7.x-2.7, Address Field 7.x-1.1. I have set the default value for address to US and I am using the : subfield notation. Any ideas why this wouldn't work?
many thanks!
My migration class is as follows:
Sorry
this cookbook is about "Import new users and their Profile2 fields from one CSV file".
Your question is about importing from a database and using address field.
This a an entirely different issue that I've never used. Please repeat this post in another forum so that this problem is solved.
Here it does not belong.
Good luck!
Added:
Add a link here to your new post!
Wusel
ok. thank you. Not sure
ok. thank you. Not sure where it belongs... address field or migrate.