I wrote this up in a blog post, and just remembered I hadn't created an issue for it - I'm actively working on this now:

When the concept of migration groups was introduced, I made them a class, MigrateGroup. All we've done with groups up to now is give them names, so having the class has been overkill. I think now is the time to start leveraging groups as objects in themselves. First off, group names up to now have been treated both as machine names (unique identifiers) and display names (shown in drush output, and now in the UI). We should have distinct machine and display names - and the wizard API (which currently prompts for a group name) should set them automatically. For example, in the WordPress case, the group display name could be derived from the blog title and URL in the source file - e.g., "My WordPress Blog (myblog.wordpress.com)", while the machine name (not displayed in the UI, mainly of use for power users issuing drush commands) would be a scrubbed version of the blog title as it is now - mywordpressblog.

More DX than UX, I also want to start using the MigrateGroup class to store information common to the migrations in the group. Currently, wordpress_migrate stores the input filename in the arguments of all migrations, while migrate_d2d stores connection info. Ideally we should normalize this information by storing it in the group, which implies we introduce a group table. Since apart from the machine name (which would be the primary key) and display name, the group data would vary according to the implementing module, so would be stored as a serialized array. Perhaps we could also store state information (when the last import for the group was run, what was the total running time, etc.).

Comments

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Pittsfield, MA, USA
commented

Going well so far, I pretty much have the wizard version of wordpress_migrate working with this. Now I'm getting more ambitious - it's pretty ugly to prepend group names to migration machine names to make sure they're unique, when we will now have a group column - I may change the PK of migrate_status to include both machine_name and group_name, so we can use the group name only when necessary for disambiguation. E.g., in the wordpress_migrate scenario, currently we may have MyBlog as a group name and MyBlogAuthor, MyBlogBlogPost, etc. as migration machine names - with this change you would have Author and BlogPost as machine names, and could run migrations off those as-is if they're unique. If you import a second blog, MyOtherBlog, then in drush you would have to run MyBlog.Author vs. MyOtherBlog.Author.

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Pittsfield, MA, USA
commented
Status: Active » Fixed

OK, this is committed (minus that last little bit, which is harder than it appeared at first glance - I'll open a separate issue for that). The highlights:

  • A migrate_group table has been added, with name, title, and arguments columns.
  • The migrate_status table has a new column, group_name, which joins with migrate_group.name.
  • Migration groups can be programmatically created via MigrateGroup::register($name, $title, $arguments). Referencing a group name via a migration constructor as has been done in the past will implicitly register the group, with the title defaulting to the name and an empty arguments list.
  • Groups may also be registered via hook_migrate_api() (see migrate_example_migrate_api()).
  • An option fourth argument, $group_name, has been added to MigrationBase::registerMigration().
  • A --group option has been added to the drush migrate-deregister command - it will deregister the group itself as well as all migrations tied to that group.
mikeryan’s picture

mikeryan
he/him
Primary language English
Location Pittsfield, MA, USA
commented

See also #1841894: Disambiguate migrations by group name.

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Pittsfield, MA, USA
commented

Just a note that I've committed some fixes to the update function here. I'll point out the note I just added to the project page:

The work for Migrate 2.6 means the -dev releases will be less stable than usual. In particular, the database update path between -dev releases should not be expected to be reliable - the goal is to produce a stable upgrade path from 2.5 to 2.6, intermediate steps will not be supported.

Automatically closed -- issue fixed for 2 weeks with no activity.