Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.I'm currently wondering about using migration for contrib module upgrade paths, but this would require a preparation step to create fields and the like.
I see on http://drupal.org/node/1343112 there's a preImport() method in the migration class, which sounds perfect. But the docs go on to say this:
> NOTE: If you are running your migrations through the web UI, which chunks the work into batches, if a given migration takes long enough to be broken into separate batches, these functions will be called for each batch.
What would be needed to make this method be reliably only called once at the start of the migration? Failing that, could a true pre-import set-up method be added?
Comments
Comment #1
mikeryanpreImport() is meant to be be called on each migration invocation - it is doing exactly what it's meant to do. The mechanism for doing one-time setup is to create a migration class derived from MigrationBase, and have your "regular" migration classes list that as a dependency. See WinePrepMigration in migration_example for an example.
Comment #2
joachim CreditAttribution: joachim commentedAh ok -- I hadn't been looking in the examples for this.
I see it now -- WinePrepMigration. But that means you have to use a dependent, fake migration?
That seems rather silly, and it clogs up the UI too presumably. Why can't a migration class have methods that really do just run the one time per migration run?
Comment #3
mikeryanThe issue here is what "per migration run" means. To you, that means a complete end-to-end run, and even if interrupted and resumed (as the Batch API does) to you it's all a single migration run. The migration class itself, however, doesn't see that larger picture - each time it's invoked it doesn't know the context.
At any rate, the means to achieve what you need exists - this is exactly why MigrationBase was split out of Migration, to add steps into the migration process that need to be performed a single time within the larger process, participating in the dependency chain to make sure any migrations that need that work done are run at the right time.
Comment #4
joachim CreditAttribution: joachim commentedI understand what you mean. But can we not make the migration class aware that it's in the middle of a process? It appears to know things such as whether it's previously run.
I understand how this is currently doable, but it really seems to me that having to create a dummy migration class to handle initial setup is very weird DX. If Migrate module becomes the new way to perform module updates, then this is going to be quite a common problem.
Comment #5
mikeryanI don't consider it a "fake" or "dummy" migration - it's a step in the migration process that doesn't actually move data. In complex migration projects, there are things that need to happen other than moving data across row-by-row, and the MigrationBase (yes, poor name - maybe it should have been MigrationStep?) class lets you insert other kinds of steps such that they occur at the right time during the process, using the same dependency system as "regular" migrations. Actually, the D8 Import API proposal I'm working on has Importer (equivalent to MigrationBase) and DataImporter (Migration) classes - with this discussion in mind, perhaps they would better be named ImportStep and DataImportStep?
Comment #6
joachim CreditAttribution: joachim commentedAh, and it seems they don't appear in the UI, which was the other thing I was concerned about.I spoke too soon -- they do appear in the UI. Which I think is problematic, when the preparation step is something that purely serves the other, main, migration.
The terminology is a bit peculiar, so ImportStep (or MigrateStep, either) and DataImport step would be clearer to understand, yes.
I'll try and find time to write this up in the docs :)
Comment #7
mikeryanClosing, the necessary functionality exists.
Comment #8
joachim CreditAttribution: joachim commentedIs there a way to hide a preparatory migration step from the UI?
Comment #9
mikeryanNo.
Comment #10
joachim CreditAttribution: joachim commentedI think that's going to be necessary for using Migrate for Drupal upgrades. You don't want someone running just the migration step that creates a field, say.
Comment #11
pifagor