White Screen Of Death on bulk update (Memory)
gerd riesselmann - February 27, 2007 - 16:14
| Project: | Pathauto |
| Version: | 5.x-1.x-dev |
| Component: | Code |
| Category: | bug report |
| Priority: | critical |
| Assigned: | Unassigned |
| Status: | closed |
Description
When you have a large number of nodes (talk a few thousands) and you do a bulk update, Drupal dies whith the White Screen of Death, that is; it runs out of memory.
Reason is that on a bulk update every node is loaded, and loaded nodes are cached. Therefore, the system will run out of memory sooner or later.
My patch resets the cache before loading the next node. So this will prevent the system to run out of memory. However, it may make the process of bulk updating slower, depending on if modules implementing pathauto hooks do a node_load or not. Bulk updating may therefore run into a timeout on systems it didn't before (See here for problem of timeouts: http://drupal.org/node/103403).
| Attachment | Size |
|---|---|
| out_of_memory_fix.5.0.patch | 331 bytes |
#1
gerd - thanks for the patch. This is an interesting idea.
I think the best long term solution is doing the update in configurable sized chunks: http://drupal.org/node/67665
The patch is in the wrong format (use -up options to create it - see http://drupal.org/diffandpatch ).
Last question - did you run out of memory or is this just theoretical? This is the first I've heard of this problem.
#2
Sorry for the wrong format, attached a new patch.
This bug is not theoretical, but occurs at one of my customers who is porting most of his rather large sites to Drupal. The process of porting includes restoring old URL using pathauto's bulk update.
The chunked solution sure is a good way to go. I think however, that every module processing large numbers of nodes in a loop should always reset the cache. There is no need in keeping nodes around that were already processed.
#3
Thank you for the patch, it helped me bulk updating my aliases :)
("Just" 412 aliases, but without the patch I always got a white page and updating didn't finish)
#4
applied to both 4.7 and 5.x--2
Thanks!
#5
#6
I also applied this to the 5.x-1 branch due to this report: http://drupal.org/node/139380
I should have done that from the beginning...