After encountering some issues with the migrate module, I decided to rewrite it all from scratch with this module. So far everything goes smooth, however there are some customizations that I am not sure how to accomplish:

I want to do some additional joins within the User migration class, in order to only import users with a certain role. How would I go from there? Is it possible to directly modify the $query object?

The joins I want to accomplish are (this snippet originates from the original migrate code):

    $query->join(SOURCE_DATABASE .'.users_roles', 'ur', 'u.uid = ur.uid');
    $query->join(SOURCE_DATABASE .'.role', 'r', 'ur.rid = r.rid');
    $query->orderBy('u.uid', 'ASC');

Comments

mixhael’s picture

I tried the following without luck:

Creating a new Class that extends from Drupal6UserMigration:

class MyUserMigration extends DrupalUser6Migration {
  public function __construct(array $arguments) {
    parent::__construct($arguments);

    $query = Database::getConnection('default', $this->sourceConnection)
             ->select('users', 'u')
             ->fields('u')
             ->condition('u.uid', 1, '>');
    $query->join('users_roles', 'ur', 'u.uid = ur.uid');
    $query->join('role', 'r', 'ur.rid = r.rid');
  }
}

and reflect that in the .module file:

// We just use the migrate_d2d D6 migration class as-is.
  Migration::registerMigration('MyUserMigration', $arguments['machine_name'],
                               $arguments);

Nothing changes, drush ms still says all of my users are awaiting migration

mixhael’s picture

Also tried the following without luck (meaning `drush ms` still states that all my users are awaiting migration):

class AweneUserMigration extends DrupalUser6Migration {
  public function __construct(array $arguments) {
    parent::__construct($arguments);

    $query = Database::getConnection('default', $this->sourceConnection)
             ->select('users', 'u')
             ->fields('u')
             ->condition('u.uid', 1, '>');
    $query->join('users_roles', 'ur', 'u.uid = ur.uid');
    $query->join('role', 'r', 'ur.rid = r.rid');

    $this->source = new MigrateSourceSQL($query, $this->sourceFields,
      NULL, $this->sourceOptions);
  }
}
mixhael’s picture

Status: Active » Closed (fixed)

excuse me, forgot to clear cache. The above works for me :)

mikeryan’s picture

You're moving too fast for me;)... Actually, since all the migrate_d2d classes use a query() method to define their queries, the clean way to modify the query is:

protected function query() {
  // Get the default query and modify it
  $query = parent::query();
  $query->join('users_roles', 'ur', 'u.uid = ur.uid');
  $query->join('role', 'r', 'ur.rid = r.rid');
  return $query;
mixhael’s picture

Well, you are still responding quite fast :)

Nevertheless, in both cases it doesn't work. When running `drush mi User`, it gives me the following:

Processed 215 (0 created, 215 updated, 0 failed, 0 ignored) in 170.3 sec (76/min) - done with 'User'                                                                [completed]

215 is the total amount of users that is in the database.

With the code I wrote myself `drush ms` tells me with the modified query that there are 20 waiting to be migrated....

 Group: default  Total  Imported  Unimported  ڕەوش  Last imported       
 User            20    0         20         Idle  2012-10-16 02:20:02 
 NewsCategory    11     0         11          Idle                      
 PDF             53     0         53          Idle                      
 Poll            15     0         15          Idle                      
 Profile         215    0         215         Idle                      
 Tag             919    0         919         Idle                      
 Article         5531   0         5531        Idle                      
 Blog            581    0         581         Idle                      
 Dossier         225    0         225         Idle                      
 Page            20     0         20          Idle                      

With the modifications I made according to your recommendation it tells me there are still 215:

 Group: default  Total  Imported  Unimported  ڕەوش  Last imported       
 User            215    0         215         Idle  2012-10-16 02:20:02 
 NewsCategory    11     0         11          Idle                      
 PDF             53     0         53          Idle                      
 Poll            15     0         15          Idle                      
 Profile         215    0         215         Idle                      
 Tag             919    0         919         Idle                      
 Article         5531   0         5531        Idle                      
 Blog            581    0         581         Idle                      
 Dossier         225    0         225         Idle                      
 Page            20     0         20          Idle                      

Still in both cases `drush mi User` tells me:

Processed 215 (0 created, 215 updated, 0 failed, 0 ignored) in 170.3 sec (76/min) - done with 'User'                                                                [completed]
mixhael’s picture

Status: Closed (fixed) » Postponed (maintainer needs more info)

This is what my class looks like btw:

class MyUserMigration extends DrupalUser6Migration {
  public function __construct(array $arguments) {
    parent::__construct($arguments);
  }

  protected function query() {
    // Get the default query and modify it
    $query = parent::query();
    $query->join('users_roles', 'ur', 'u.uid = ur.uid');
    $query->join('role', 'r', 'ur.rid = r.rid');
    return $query;
  }
}
mixhael’s picture

I think the basic problem must be somewhere else. Even when trying to do a basic mapping `drush mi User` gives me the following:

Processed 215 (0 created, 215 updated, 0 failed, 0 ignored) in 153.2 [completed]
sec (84/min) - done with 'User'

Any clue why this might be the case?

Still, the question arises that when I add this:

protected function query() {
  // Get the default query and modify it
  $query = parent::query();
  $query->join('users_roles', 'ur', 'u.uid = ur.uid');
  $query->join('role', 'r', 'ur.rid = r.rid');
  return $query;

to my class that `drush ms` keeps on saying that 215 users are awaiting migration, instead of 20... It seems the query has still not been modified

mixhael’s picture

Status: Postponed (maintainer needs more info) » Active

forgot to set it to active

mixhael’s picture

Getting towards the root-problem, but still not see what is exactly going wrong. When I click on the User migration in Migrate UI, it tells me the destination of the User Migration is the 'Profile node'

On the 'Destination' tab:

TYPE
node (profile)

The relevant code in my .module file looks as follows:

// The description and the migration machine name are also required arguments,
  // which will be unique for each migration you register.
  $user_arguments = $common_arguments + array(
    'description' => t('Migration of users from Drupal 6'),
    'machine_name' => 'User',
  );

  // We just use the migrate_d2d D6 migration class as-is.
  Migration::registerMigration('MyUserMigration', $user_arguments['machine_name'],
                               $user_arguments);

  // The description and the migration machine name are also required arguments,
  // which will be unique for each migration you register.
  $profile_arguments = $common_arguments + array(
    'description' => t('Migration of profiles from Drupal 6'),
    'machine_name' => 'Profile',
  );

  // Migrate profile nodes from the User data
  Migration::registerMigration('ProfileMigration', $profile_arguments['machine_name'],
                               $profile_arguments);

And the relevant code in node.inc:

class MyUserMigration extends DrupalUser6Migration {
  public function __construct(array $arguments) {
    parent::__construct($arguments);

    $this->addUnmigratedSources(array('profile_naam', 'profile_description'));
  }

  protected function query() {
    // Get the default query and modify it
    $query = parent::query();
    $query->join('users_roles', 'ur', 'u.uid = ur.uid');
    $query->join('role', 'r', 'ur.rid = r.rid');
    return $query;
  }
}
class ProfileMigration extends DrupalUser6Migration {
  public function __construct(array $arguments) {
    parent::__construct($arguments);

    $this->destination = new MigrateDestinationNode('profile');
  }
}
mikeryan’s picture

Status: Active » Postponed (maintainer needs more info)

OK, one thing that's going to cause trouble is joining to users_roles. It's essential for a migration source query to return one row per item (user, node, etc.) to import, and by doing an inner join to users_roles you'll get X rows per user, where X is the number of rows in users_roles for each user. You're not adding any fields from those tables, or conditions on those tables, so I don't see what you're trying to accomplish here.

Another issue is that you're deriving ProfileMigration from DrupalUser6Migration... but it looks like you have a separate issue open for that.

mixhael’s picture

I want to make sure only users are imported that have are not regular authenticated), but have one or more custom defined roles. Good point, will look into improving this query.

But how come the User migration says it's migrating into the destination of a Profile node?

mixhael’s picture

Status: Postponed (maintainer needs more info) » Closed (fixed)

Nevermind, all fixed now by adding a $query->distinct();

The migration into a profile node was a conflict with another issue.