Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.
Problem/Motivation
Migration involves a processing-heavy set of operations that may trigger page timeouts. Typically in Drupal core such operations call drupal_set_time_limit() to allocate additional processing time, with 240 being the usual value used.
Proposed resolution
Add one or more calls to drupal_set_time_limit(240) before migrate processing calls.
Comment | File | Size | Author |
---|---|---|---|
#5 | migrate-time-limit-1892296-5.patch | 1.49 KB | nedjo |
#3 | migrate-time-limit-1892296-3.patch | 472 bytes | nedjo |
Comments
Comment #1
moshe weitzman CreditAttribution: moshe weitzman commentedThats the responsibility of the batch API. Also, long running SQL queries do not count against PHP time, AFAIK.
Comment #2
nedjoMakes sense. It's hard to see a use case for calling a migration outside of a batch, and if someone did they could take care of page execution time themselves. And, yes, sound point about most migration processing being outside of what counts as PHP time (except on Windows).
Comment #3
nedjoLooking closer, while the cron system does allocate processing time, batch API does not (presumably on the assumption that most batches will chunk operations into batches that don't require extra resources).
I opened this issue coming from #1763508: Reimplement Migrate integration in the installer, where bojanz noted "when the system is low on memory, the migration completes halfway through and the installation is completed without the complete set of content."
Yes, batches should include the sort of code that's in migrate_ui_batch(), examining the return value of the processImport() method and responding accordingly, with the result that incomplete imports will still be completed in subsequent rounds. Still, since migration is a resource-intensive and very infrequent operation, it seems like a good case for setting a high time limit to minimize RESULT_INCOMPLETE migration results, which can result from approaching the PHP time limit (see MigrationBase::timeExceeded()).
Two places we could put this. 1., before calling processImport(). 2., in processImport(), before initiating the import. Attached patch takes approach 2.
Comment #4
mikeryanThe issue is specific only to running through the UI so it should be done in the batch processing.
Also, I wouldn't hard-code the value - please make it a MigrationBase member that can be set in the constructor so migration implementors can adjust to fit their needs:
Comment #5
nedjoHere's a revised patch based on the suggestions in #4:
migrate_ui_batch()
.Comment #6
mikeryanCommitted (to wizard_api branch), thanks!
Comment #8
mikeryanTagging as a 2.6 feature.