Currently the installer just gets a list of all available migrations and then does $migration->processImport($options);
However, processImport() returns a value, and if that value is MigrationBase::RESULT_INCOMPLETE, then the migration should be run again.
Our code currently doesn't do that, which means that when the system is low on memory, the migration completes halfway through and the installation is completed without the complete set of content.

We need to rewrite our batch function to rerun the migration in that case. We also need special code for drush (since the batch doesn't help there), similar to the one in migrate.drush.inc.

Comments

Issue tags:-RC blocker+RC2 blocker

This can wait until RC2

Status:Active» Needs review

there's commit for batch function for kickstart migration to re-run for incomplete migration
https://code.drupalcommerce.org/#/c/463/

based on migrate_ui_batch() function code.

to do : special code for drush

Did this get committed and not updated? It's the only 2.0 release blocker that wasn't "complete (fixed)" ?

Curious because I'm going through all the drush related issues in the que and this one seemed like it needed and update (10 weeks old).

Given this is a processing-heavy operation, there may be a role here for drupal_set_time_limit(). I've opened a related issue on migrate: #1892296: Allocate sufficient time for migration processes.

Re #5. As Moshe pointed out in #1892296, batch API takes care of setting the time limit.

Issue summary:View changes

@joshmiller @bojanz Has this been fixed? It is tagged 2.0 release blocker but still needs review :)

No, this was never committed :/ Needs to be tested in a low memory situation (get the system to swap) to confirm that it retries / continues where the previous code doesn't.

Related bug report which might be useful in the future: #1851806: Install slow on import content