I just watched your presentation on the Migrate module at Drupalcon-09 and couldn't quite follow the bit about importing data to create users and profile content fields at the same time (at the 44 min, 25 sec mark). I know very little about SQL, so bear with me...

Migrate video: http://dc2009.drupalcon.org/session/migration-not-just-birds

Am I correct in assuming the Migrate module will create users, populating the standard profile fields, AND simultaneously populating content-profile fields, but only if you setup the corresponding table relationships first, in TW, to join in the content-profile fields? I took a look at the relationship area but couldn't figure it out. I understand the basic principle behind relationships with respect to Views, but don't fully understand the SQL behind that, like the left join VS right join concept.

The presentation also touched on indexing. I've read about indexing and understand it, but is this necessary for importing users and filling profile fields, or does it just speed the process up?

I have a table of people I want to import (already in MySQL). It has four columns; ID, Name (first last), email, phone. I want the Drupal user name to be the whole name, first then last (just as in the DB table), and the email to go into the usual profile field. The phone needs to go into a content-profile type field I created. The ID field is not relevant to anything and doesn't need to be imported.

I have already imported the basic user data (264 records) but not the phone field, as it's a node not a user type. Can you very briefly outline just exactly how I can create the content-profile records? I'm assuming I have to undo the user data load I did and start over with the correct relationships.

What do I select for "left join" VS "right join", for example? All of the relationship selection choices appear to be ID fields. I have one table with data going to different destination types, user/profile and content-profile.

It appears also that FK (foreign key) is part of this, but AFAIK, that feature is only available if the tables use the InnoDB engine. Do I have to convert to InnoDB before I can do this? That doesn't seem right since Views and relationships work just fine using MyISAM.

Can you also explain a little about how creating table indexing is relevant to this, and/or how it speeds up the migration process in general?

I couldn't find any documentation on using TW or Migrate.

This is a great piece of work, BTW, and I'm looking forward to future versions with expanded capabilities.

Comments

dejamuse’s picture

I thought about this some more and it occurred to me that I should split my single user's table into two tables, one with ID, name, email, and the other with the same ID field and phone. Then establish the relationship (left join with users on the left) between the two tables using the common ID field and import the first table as users, and the second as the corresponding content-profile. Then the Table Wizard relationship I created properly correlates users with their content profile.

Is that right?

drupalize’s picture

did u try something like this

1. create a profile in drupal with all the fields that you want to import (in a single table)

2. look at the corresponding table

3. convert your data to match that table

do u need help to convert your data in this way ?

if u get to this point let me know..

dejamuse’s picture

This is the answer I got back from Mike, today by email:

Migrate supports the content_profile module out-of-the-box, but not the core profile module. If you have content_profile installed, and have added CCK fields to the profile content type, those CCK fields show up as destinations on the mapping page for a user destination. You don't need to do anything special with the Table Wizard. You should be able to map the phone in your content set view directly to your CCK phone field.

Indexing is not necessary, but queries joining large tables will perform much faster if the fields used in the joins are indexed.

I don't see in what you've described a need to define any relationships. But, should you want to,... The relationships page does not define whether a join will be a left join or right join, but the direction of the join supported. I.e., if you define a relationship with a left side of table1.ID, and a right side of table2.ID, then when you use table1 as the base of a view you can add a relationship to table2 (and optionally make it a left join) but to start with table2 and add a Views relationship to table1, you first need to have a Table Wizard relationship with table2.ID on the left side and table1.ID on the right side.

I have to think about this some more and try importing users again, but I was able to import the user name and email into the standard profile, but don't recall seeing any content-profile fields in the fields list, when importing into the 'user' type. The fields did show up when selecting 'node' type, I think it was. I'll report back soon.

mikeryan’s picture

Two things:

  1. The destination content type of a user migration must be a role. I have not yet implemented using hierarchical select or some such to dynamically update the selection list to show only relevant types (there's an issue for this somewhere).
  2. Right now the migrate module assumes that the name of the content_profile content type is 'profile'. This is why the CCK fields aren't showing up for you.
dejamuse’s picture

I first setup content-profile fields in CCK.

Then I tried to import the user table I described before.

If I select the 'kind of destination' as 'user' and then select 'destination content type' as 'personal profile' (the name of my content-profile type), on the next page I get the selection box for 'destination content type' again, but only roles are shown (mislabled?).

At this point the mapping set shows only:

User: Username
User: Password
User: Email address
User: Account created
User: Account last accessed
User: Last logged in

Using this mapping set I was able to populate the username and email in the regular profile fields using the 2nd choice of role. Only if I select the 'kind of destination' as 'node' am I shown all the CCK fields in my content-profile.

I cannot set the 'kind of destination' to 'user' and get a mapping set with CCK fields from my content-profile type, only the six standard profile fields shown above. The choice of 'Profile' refers to the generic content-profile type installed by the content-profile module that isn't being used and has no added CCK fields. My custom content-profile types were created from scratch and told to become content-profile types (checkbox at bottom of form). I did not use the stock Profile (profile-content) form. There is no other choice for 'profile'. I have to set the 'kind of destination' to 'node' to see the CCK fields in my custom content-profile mapping set.

It seems I cannot get a single destination set that includes Username, Email, and CCK fields.

That's why I thought I had to use relationships to tie the content-profile migrate set to the standard profile set using the ID key as the common element so the two end up synched. I was thinking of splitting the user table into two parts, one with ID, Username, Email, and the other with ID, Phone, because they both use different 'kind of destination' types. Common source, different destination.

What am I doing wrong?

With the left join/right join thing, I actually meant left join and inner join, but I've since figured out the jargon and I'm pretty sure I get it now.

Tnx, Jeff

mikeryan’s picture

The only valid "destination content types" for a "Kind of destination" of user are the roles. Because I haven't yet implemented some AHAH stuff to automatically filter the destination content type (#392454: Dynamically update type selections), all kinds of destinations are displayed, but only the roles are valid - the mapping set page is correct in only showing rows.

I believe you said your content profile type is names "personal profile". Right now the migrate module only recognizes content profiles named "profile". If you recreate your content profile type with the name "profile", then your CCK fields will be visible.

drupalize’s picture

Related issue or question: It would be nice to be able to select the specific name that one gave to one's content profile node type, should one have used a name other than 'profile'.

I may have had a similar problem when I installed content profile, and I already had named one of my cck node's to be 'profile'. This mussed things up a bit with my installation, or maybe not, I am still not sure about this. thanks.

mikeryan’s picture

Title: How to import users AND populate content-profile fields simultaneously? » content_profile integration: use correct content type name
Component: User interface » Code
Assigned: Unassigned » mikeryan
Category: support » bug

The base problem in this issue is a restriction in the content_profile migration support, in that it assumes the content_profile content type name is "profile". It should figure out what content types are functioning as content profiles, and make all CCK fields for those types available to user migrations.

mikeryan’s picture

Status: Active » Fixed

Done - it will now search for all actual content_profile content types and make them available for user migration.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.