Improve efficiency of options in select.inc

Thanks Damien for taking initiative on this. I marked the other issues you found as duplicates.

#273885: blank page when downloading results (memory exhausted)
#392290: Submissions: Modifying CSV download function, or SQL statement
#323493: Cannot download results of webform
#407460: "Download" results link leads to blank screen

FWIW, my issue at #407460: "Download" results link leads to blank screen is NOT a timeout issue. My webform had 5 fields, and only 4 submissions -- I'm pretty certain that if the server can handle views and aggregator and all of the other parts of that site, it's not a memory error that leaves this screen blank.

Oooh, turns out I had some other problem. I tried to apply the patch and got a big FAIL message. So I re-extracted the webform module and now everything works perfectly (sans patch). Thanks for your help!

(I always think it is so weird when that happens -- I expect that if some file gets corrupted I will end up with big PHP errors and a huge mess, not some stealth unobtrusive error.)

D5; it was a patch error that said that Chunk 2 had failed during the webform.module patching. I figured that since I hadn't changed anything in the actual webform module, something was amiss! Hence the re-extraction of a re-downloaded zip and now everything works as expected.

I came up with a similar patch (to #4) before seeing this issue, and for me it alone speeds things up enough to avoid the PHP timeout problem.

I applied the patches in #4 and #6 and it fixed my timeout issue - Havent noticed any bad effects yet. Thanks for the patch

I definitely like the improvements to _webform_filter_values(). There's no good reason to load that multiple times. The changes to select.inc look extremely suspicious though. Right now there's accounting in the static variable for "$flat = TRUE" and "$flat = FALSE", but it doesn't account for multiple select lists that are both flat or both not flat. It looks like this would erroneously return options for the first select list in the form for all the remaining select lists. Perhaps the $cid needs to be passed into the select list options.

A small code-style issue also: comments should be formatted as sentences, starting with capital letters and ending with periods.

Just a note that caching to _webform_filter_values() have been included in the 2.7 version. The select list options would still be good to cache, but they still require work per my comments in #21.

Unless there's further interest in completing these changes to the select list options, this will probably go unchanged. The caching in _webform_filter_values() already has corrected the bulk of the problem this issue was trying to solve.

I would like to take a look at this issue some more. I am currently running webform-6.x-3.0-beta2 and routinely have 30k+ submissions to my monthly sweepstakes webform. If I disable the output of my two option selects for country and state/province I can download all results with just a few seconds total processing time. However, if I enable both of those selects the process times out even with 512M max script size and 600 seconds max execution time which just seems totally ridiculous. This is definitely something that needs more work.

I just modified select.inc based somewhat on the earlier patch in #10 taking into account the comments in #21 and decided to just not cache non-flat select lists. I can now successfully generate and download a webform with 40k results resulting in a 9.5MB csv with just a few seconds parsing time. I am going to go ahead and make a patch and post it. It would be great to get this included in 3.0 final as I would almost consider it a bug.

Here is the patch I am currently using...it is working fine for me both on webform display and when downloading results. Please comment on your thoughts.

Thanksk leprechau! I don't understand why $flat determines when an item is cached or not. For select lists, $flat = FALSE, but for both radios and checkboxes this is TRUE. So we're only caching for select lists? This patch also deviates pretty far from coding standards, if you could clean it up a bit that would be great.

Yes, I was only interested in caching select lists. I can't see a situation where someone would have dozens of check/radio boxes listed down a page. I think most people when listing several dozen options would use a select list. I may be incorrect but it seemed rational to me.

I will re-read the drupal coding standards page and make changes where appropriate.

Okay after some input from quicksketch on irc I think this should be committable. The code is cleaner and less involved than previously suggested patches.

Thanks leprechau, I shortened up the variable name for readability (just $options until the end of the loop, then assign it to $cached_options[$flat][$md5]). Committed to both 3.x branches.

Looks great, thank you.

Would it be possible to apply those changes to 6.x-2.x as well?

I'm running a site with a webform that currently has just over 500 submissions. It's a large form, and I get a fatal error when I try to download a CSV file.

I would also be very interested in a port of #32 to the 2.x branch. Thanks for your work!

This patch is a backport of #32 and apply to Webform 6.x-2.9 (select.inc, v 1.22.2.30) and, at least, works for us.

Please disregard this attached patch, small typo at the end

This should work.

Subscribe

Patch at 38 is working fine for me on a 6.2.9 install.

hi, could anyone pls post the file itself (select.ini patched) for version 6.2.9, i dont have the tools to patch it here

thanks

This will not be backported, I suggest upgrading to 3.x now that it is the recommended/stable release.

Here is the patch from #38 against 6.x-2.x that will apply from drush make, for those that need it.

I'm having the same problem with 6.x-3.18 version. Blank page for table view and download results view. Only 800 submissions, the form has several select.

memory_limit = 128M
max_execution_time = 30 (180 same results)