Here is some in progress code to deal with memory exhaustion when running a batch. I really need for #211182: Updates run in unpredictable order to land before I can make much progress. That patch there is going to require changes in our updatedb process. Namely, we have to call update_resolve_dependencies() or call its wrapper, update_batch().

CommentFileSizeAuthor
#1 drush_batch_api.diff6.7 KBadrian
#1 batch_7.inc_.txt6.93 KBadrian
#1 batch_6.inc_.txt5.58 KBadrian
updatedb.patch6.63 KBmoshe weitzman

Comments

adrian’s picture

adrian commented
Title: updatedb for d7 » 'multithreaded' updatedb for better memory consumption
Status: Active » Needs work
StatusFileSize
new batch_6.inc_.txt5.58 KB
new batch_7.inc_.txt6.93 KB
new drush_batch_api.diff6.7 KB

here's an updated patch

well. rewritten.

it is for d6 and d7.

apply the drush patch and then drop the batch_X.inc files into commands/core/drupal

moshe weitzman’s picture

moshe weitzman commented

Thanks a lot. adrian explained via IM that this patch implements our own batch processing that honors the same batch definitions as expected by core. This turned out to be the sanest way to implement updatedb, for example.

I'll give this a test as soon as core gets its upgrade path sorted. Should be this week.

We need a @file declaration at top of batch.inc files explaining the deficiencies in core batch api and how our batch processing is different.

adrian’s picture

adrian commented
Status: Needs work » Fixed

i committed a cleaned up and documented version of this.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.