Adding new conditions becomes slow after a large number of conditions (50+)

Having the same issue.

I'm using 7.x-4.0-alpha9 and have approx. 270 fields and 37 conditions set but need to add some more but it's not letting me add anymore. Nothing happening when I click +. Any chance of an update on this please, so I know if I need to tackle this in another way ?

Update: I downloaded the site be developed to work on it locally and try out entityforms. Tried one last time adding more fields and conditions and for some reason it's started to work ok.

Hi guys, thanks for opening this issue. I haven't tried adding so many conditions to a single form before, it wouldn't surprise me if things started becoming expensive with that number of conditions in place. I believe this is a problem, but considering 80+ conditions is a little on the unusual side, I'm downgrading this issue to normal.

I'm not sure what the status of each of your situations is. @associate: It sounds like things worked for you in the end with Webform? @Qandeel: Are you still using Webform at all? AFAIK, Entity Forms doesn't handle multiple pages or conditions, so I'd be surprised if switching would solve your requirements.

Still having problems. Can't add anymore fields now and trying to change the order of the fieldsets, doesn't work either.

I can confirm that editing conditions page does slow down. I've noticed it on forms with 40+ conditions. We have several with 70+ conditions and things really do start to get slow. I've not yet had time to profile the js, but that is almost certainly the culprit.

Entityforms isn't really an alternative as, in my brief experience using it, it does not deal well with complex forms (100+ fields), nor as @quicksketch said does it handle multiple pages or conditions. Handling of forms with many fields is the core reason we use webform.

As a temporary work around here try using a faster machine.

I have having the same problem with webform.
I have webform--7.x-3.19
I have drupal--7.24
The form I am working on is long--it has over 300 components. I was editing a webform form and then something stopped working and now I cannot add components or move them---I can delete components, but cannot add or edit components. Any ideas?

Here is some new information... I can edit the name of a component and clone components, but am unable to move them and am unable to create new components. When I create a new webform, all functions work properly....

@b_trebor This issue is about webform conditions which are only part of 7.x-4.x, you are on the 7.x-3.x branch which doesn't have conditions.

The form I am working on is long--it has over 300 components. I was editing a webform form and then something stopped working and now I cannot add components or move them---I can delete components, but cannot add or edit components. Any ideas?

You're describing #2088035: Cannot add components or move them around with large forms (400+ components), which is a different problem.

I re-read this issue today to see if I could figure out any problems by reviewing the code. I think this is due to the approach used by Webform to add a new row to the table of conditionals. When a new row is added for a new conditional, it's not just the new row that is replaced. Webform replaces the *entire table* when adding a new row, but then it fades in the new row so it looks like just one row was added.

This means that all the JavaScript attachments for all the elements on the page have to be remove and re-added every time a new component is added, and that's likely where the slowness is (rather than re-rendering the table, which comparatively is inexpensive).

I think we can solve this problem in two ways, or both for maximum improvement: 1) Only replace the last row in the table when adding a new row. This will prevent rebinding and save a lot of bandwidth/processing. 2) Bind the JS events to the wrapper or form instead of to individual elements. This will speed up binding/unbinding time on normal page loads as well as on AJAX actions.

Here's a stab at approach #2, though it's buggy (values are all reset to the first option in select list conditionals). Unfortunately I found that it did not yield a very noticeable improvement.

After building out a large number of conditionals, I checked the network time when adding a new conditional. Much to my surprise, the slowness seems to be *server-side*, rather than client-side. On a new form, it takes about 100ms for my localhost to return a response for a new conditional. After adding ~50 conditionals, it takes almost 2s. My computer is pretty fast, so I imagine this time could be significantly worse on a less powerful machine.

The approach #1 might help with both server-side and client-side problems, but it has some other challenges as well. Right now we use theme_table() to output the entire table at once. We'd need to make a new theme function for rendering out individual rows one at a time, since we need to return just a single row.

Here's a working version of approach #2. It helps... a little. It also helps smooth over some workflow oddities when dealing with conditionals (i.e. switching from "is" to "is not" operators no longer reverts to the original value). Considering this is generally best-practice when dealing with a lot of elements anyway, I've committed this patch. I'll work on the (more challenging) approach #1 next, since this alone isn't sufficient.

Great debugging Nate, most interesting to hear the bigger slow down is server side. Just tested #15 and it does help, but approach #1 seems to be where the big gains are to be had.

Hi @fenstrat, great to see you here. I've been plugging away at approach #1 and have it working as well, but sadly it *also* doesn't have the full impact I'd expect, because server-side slowness still makes responses take a few seconds to return. I think this patch really helps with a lot of conditionals, but we'll still need to find some kind of optimization to speed up the form submitting/rebuilding.

Here's another interesting patch (sorry, these are all over the place, there are lots of scaling problems here). This just removes an expensive bit of processing from _webform_conditional_expand(), which is intended to add JS settings to the page. This had been called once per conditional to increase the robustness of the "webform_conditional" FAPI type, but since it's extremely unlikely that other modules would use this FAPI element anyway, we can safely remove this check. Webform already adds the JS settings to the page at the form-level, so there's no need to duplicate this for every element. Just removing these two lines yielded a very consistent 25% improvement in response times.

No worries Nate, been away for a while (getting towards the end of a 12 month working holiday which has played havoc with contrib work!) but I'm finally starting to get the D8 branch caught up and have plans to move it forward.

As for this issue I've just tested #18 and #19 with a many conditions (70+, with 300+ components) and they both help. Adding a new condition, or even just using tabledrag to reorder them is still pretty painful. So no silver bullet as yet, scaling this is tricky stuff.

I've committed this patch which is the combination of #18 and 19.

Other ideas to speed this up:

- Don't use #ajax on the add/remove condition rule buttons, this would prevent the need for a lot of Drupal.settings being transferred back and forth and save more on JS processing. Similar to the changes we made in this patch, this would reduce the number of bindings on the page.

- Speed up the form generation more(?) I don't know how we can speed this up further. From looking at XHProf, 10% of the page load time is spent in check_plain(), inside of form_select_options().

I'm not sure what areas are worth pursuing at this point. Each of these optimizations make the code more convoluted or specialized, which could affect the maintenance of the project. I tried switching to custom AJAX handling, but it started to get pretty messy (new custom menu callbacks, etc).

As a thought, for forms with many components the select lists for those components are rendered many many times (both for If <select components> then hide/show <select components>). Would it help if they were turned into autocompletes?

Would it help if they were turned into autocompletes?

I think the page-weight problem is likely an issue, but switching to autocomplete seems like it'd be a pretty serious reduction in functionality. Interestingly, Wufoo uses a trick to prevent repetition of the same form elements by using JavaScript to replace all the elements on the page with placeholders. Then when you click/focus on the element, it swaps in a select list.

The idea of swapping in select list options when a user clicks into a field is interesting. Each component <select> is exactly the same, and with a form with hundreds of components page weight is certainly an issue.

#15 and #22 need porting to 8.x-4.x.

Committed and pushed 3ec813d and 36406b3 to 8.x-4.x. Thanks!

Moving back to Needs work as per discussions from #22 onwards.

I also ran into this issue when creating a very large webform (200+ components, 8+ pages). Once I had 18 conditionals in the form (a total of 25 conditionals in the database, 7 conditionals belonged to a different form altogether), I was no longer able to add any more conditionals through the user interface for the webform.

I tried to find a solution, read through this thread, and still don't have a solid handle on why it's happening or how to fix it. As I am building a production form-building site, this is a serious problem for me.

I was able to manually enter the conditions into the database after I had determined the proper ID for the components, and once the conditionals were in the database, I was able to manage them on the conditionals page. This is not a valid solution for me as I need the site to be totally functioning and workable for someone who has no direct access to the database.

Has anyone else encountered this issue of hitting a ceiling with adding webform conditionals, and hopefully a solution? I increased my php memory from 128M to 256M and it made no difference.

Drupal 7, Webform 4

@sxxxydan - I assume you are using the latest 7.x-4.x-dev so that you have quicksketch's patches. It doesn't sound like quicksketch is going to push this further. A solution that I used in another module is to create the new rows all in jQuery (no server calls). You then have to trick the server into accepting these new form elements. An additional complication is that you form changes based upon the field selected. You would have to do this all client-side too. I'm sure this would solve the performance issue, but it would be a lot of development work.

Today after add some components and rules, I also have this issue with a webform.
So I can't add or erase any webform conditional rules: 218 components, 10+ pages, 68 conditional rules.
For several days it was difficult to add more conditions rules. I had to save after each addition of a condition to add a new.
What can I do ? may be restore the database from yesterday ?

Drupal 7.31 / Webform 4 rc6

@RadiATIon - You don't say what your error is, but if you might try increasing your PHP memory or time-out in order to be able to make this work. Or try a faster server?

It would be helpful if you could investigate more what exactly is consuming the time or memory.

Changing this to a feature request since there is no concrete bug and no standard for how many components / conditionals are required.

If the delay is server-side and it could be that performance is O(N*M) where N = nr components and M = nr conditionals. Caching a dependency list would be quite a savings so that each component only executes the conditionals that affect it.

@DanChadwik. I have no error message. It's just that I can't add or remove components rules unless I do it with phpmyadmin. it's not a very comfortable solution

I saw on this page : https://www.drupal.org/documentation/modules/webform/faq#webform-not-saving that can come from variable max_input_vars if only to 1000.
I contacted the hoster of my website. He confirmed that I have a hosting with PHP FPM included by default, which means that the variable is passed to 16000.
Concern met so can not be due to the limitation of this variable.

Currently my website is hosted on a shared server.
By the end of the week, my site will be housed in isolation with clean and guaranteed resources to ensure availability of consistent and continuous power: 1vCore & 2GB RAM. Private SQL database 128MB
Do you think this will help? I hope....

Webform 4-0 / Webform custom buttons 7.x-1.0-alpha3 / Webform Phone Number 7.x-1.18/ Webform Postal Code 7.x-1.0-beta3 / Drupal 7.31

Hi,
My site is now in isolation with clean and guaranteed resources ; 1vcore & 2GB RAM, PrivateSQL database 128MB. Now the use of this latter is 64/128MB.
The Database size is 65Mo.

I still can't add new rules: when I click on the add button nothing happens.

Do you want more information ? Do not hesitate to ask me !

Thanks in advance for your help.

RadiATIon.
Webform 4-0 / Webform custom buttons 7.x-1.0-alpha3 / Webform Phone Number 7.x-1.18/ Webform Postal Code 7.x-1.0-beta3 / Drupal 7.31

@RadiATion -- I think you are going to have to push this forward yourself. You have the setup and environment to do some performance testing. If you don't have the development skills personally, you might hire someone to help you. I am happy to review patches, but I don't personally plan to invest days of work optimizing this use case.

Marking this Active since there is no patch that needs work.

I have only 23 conditions and can't add more, nothing happens on saving. It's not problem with max_input_vars.

I watched $_REQUEST and it's correct, it contains new condition. Then I watched $form_state (inside webform_conditionals_form_submit function) and it doesn't contain new rule.

But I have a lot of fields (about 200) and if I delete several fields then I can add one more condition...

I suppose that there is a separate bug, it's not perfomance issue.

...In webform_conditionals_form_submit() handler there is no new rule in $form_state['values']['conditionals'], but it is present in $form_state['input']['conditionals']. I tried $form_state['values']['conditionals'] = $form_state['input']['conditionals'] for testing purpose and seems that rule was added to database, but conditionals' page can't be rendered after that (white screen).

After experiments with webform_conditionals_form() I changed structure of targets' list. This line:

$target_list = webform_component_list($node, TRUE, 'path', TRUE);

to:

$target_list = webform_component_list($node, TRUE, TRUE, TRUE);

and then I can add more ~10 conditions. Some strange problems with long lists of components in conditions...

Hi, everybody!
I don't add more than 70 conditions.
I didn't understand as to solve this problem.
There is a decision?

Can do it because I don't speak English. I use translators :(

In the end I add rules to database manually. To hide some element if empty another one:

insert into webform_conditional (nid,rgid,andor,action,target_type,target,weight) values($NID,$NEXT_RULE_ID,NULL,'hide','component',$WHAT_HIDE_CID,$NEXT_RULE_ID - 1);
insert into webform_conditional_rules (nid,rgid,rid,source_type,source,operator,value) values($NID,$NEXT_RULE_ID,0,'component',$SELECT_FIELD_CID,'empty','');

I think the page-weight problem is likely an issue, but switching to autocomplete seems like it'd be a pretty serious reduction in functionality. Interestingly, Wufoo uses a trick to prevent repetition of the same form elements by using JavaScript to replace all the elements on the page with placeholders. Then when you click/focus on the element, it swaps in a select list.

I have another project which has a table with about 2000 rows and about 20 columns. Instead of having 40,000 select items, I have 1 which I create and place dynamically over the value to be changed. The change is made, the cell updated, and the select element removed.

I think this might be a tool. The server does NOT have to regenerate the whole page because the data is return to the brower and the jQuery makes the change.

I'm not sure if quicksketch is still following this issue, but I'd welcome his input.

In setting up some tests with just over 200 components (one text and 200+ markup components) and just over 200 conditionals (each markup component being shown based upon the text component), I discovered that the conditional page stops saving due to hitting the PHP limit of max_input_vars.

I raised it from the default of 1000 to 10000 in php.ini, and I could once again save changes. I suspect some of the issues reported with not being able to create / save conditionals is related to this. This setting was introduced in PHP 5.3.9.

The OP is reporting a slowness, which I'm not really seeing (about 5 seconds on a medium-speed windows development machine).

Since each conditional uses 5 input elements, we need to do something if we are to support > 200-ish conditionals.

@edhel re #42, that is very, very interesting. The only effect of the change you made is that the option elements will have longer names. The prefix for the parent fieldset will be the fieldset's name, rather than a dash.

1. I wonder if you are hitting a server post limit size. Do you have very long field set names?
2. You say that $_REQUEST contains the new conditional, but does it also contain all the rest of the required fields, including the submit button, the form_id, and the form_build_id? If this data is missing, your changes would not be saved.
3. Are you absolutely sure you aren't hitting the max_input_vars limit? Have you tried increasing it to, say, 10000 and then restarting apache2? If your conditionals have a lot of conditions within them, you could exceed 1000 input variables with just 23 conditions.
4. Did you notice the conditionals page getting slow (like 30 seconds slow) before it stopped working, or do you think execution time is unrelated.
5. And last, if you can export your webform using node_export, I'd like to look at it. If it is not confidential, please upload it to this issue so that others can help too. Otherwise, let me know and I'll send you my e-mail address.

For those whose conditionals page stops working, rather than just gets slow, I added a check for exceeding max_input_vars in #2417757: Conditionals page or webform client form may exceed PHP max_input_vars and stop functioning.

I would like an example of a webform that has terrible performance with a reasonable number of components and conditionals for further testing.

Closed for lack of activity and detailed instructions to reproduce slowness unrelated to max_input_vars issues. Please reopen if you can provide detailed STR.

I am also not able to add another field after create 100 field in the webform at the live server. but working in the local machine
please tell me solution as soon as possible.