use D6 batch API ?

my understanding is that mysql time doesn't count toward the timeout so perhaps this is not needed? did you try it? anyway, this is a probably a good idea.

The attached patch adds the batch API for the admin/content/generate form. I've marked the patch as "needs work" only because similar (but easier) functions need to be created for the user and taxonomy generate forms. I've put a @TODO in the comment to indicate where this needs to be done. Also, there's really no reason that most of the work needs to stay in the new devel_generate_batch.inc file. I did most of the work here, mostly so that I could have a clean patch and change as little of the existing code as possible.

i'd like to hear from chad or dww about any project profile impact.

I don't understand what you mean by "project profile impact." How would this patch to devel module impact the project module?

we need to keep devel_generate_content() consistent IMO. there's no need to break any existing programatic implementations of the function when we can make it a wrapper.

also, it might be best to change over taxo and user creation in new issues, instead of creating a mega patch.

I've reimplemented devel_generate_content for backward compatibility. I've also removed the TODO's by restoring the original generate taxonomies and users functions as-is... It makes less sense to change these because you're less likely to want large numbers of those tables.

i committed some pending changes i had to devel_generate_content. this likely needs a reroll. sorry about that. if you can reroll, i will commit before anything else sneaks in. thx.

LIVE FROM THE MINNESOTA SEARCH SPRINT, I'm reviving this old issue in preparation for search performance testing... I'm working off of a 6.x version for now, but will also post a 7.x version (which I suspect will be almost identical, if not exactly identical). This patch isn't quite right, but it's close.

... and a working version. However, it's not deleting content properly. I'm not sure if the not deleting content properly is a problem with devel or this patch, but because it's not working right, I'm still leaving as CNR.

Hi Doug. If you are already in devel_generate and are motivated to do so, I want to switch the node creation so that taxonomy and comment work on the pre-save operation of nodeapi(). Other modules will plug themselves in there too (I want bulk creation of og nodes and og posts). Just a thought. Thanks for working on this.

Note that project module's install profile uses node creation programmatically so they won't be able to use the batch creation but I think thats fine and we are not likely to break anything for them.

I fixed the delete problem. The attached patch now works. I used it last night to create 100,000 nodes.

And one more roll, this time with the new devel_generate_batch.inc.

This fixes num_comments.

And here's the 7.x patch

No time to actually test, but shouldn't this line

Thanks yched, Yes it should! Attached patch is for 7.x.

And here's the 6.x patch.

I'd love to see this committed before things get out-of-date again... Would you like me to commit it, or would you like some additional testing/comments? BTW, the batch api only kicks in if you try to create more than 50 nodes, so if you're testing, try creating 500 nodes.

@Doug - you may commit this to DRUPAL-6--1 and HEAD. I hope you will stay around to fix any bugs that may come up. Also note that I committed to devel_generate yesterday so please be careful to merge before committing.

Committed to both DRUPAL-6--1 and HEAD. Thanks!

Automatically closed -- issue fixed for two weeks with no activity.