OK So I have been having problems with this for a while.

I have 4 field collections, inside of each other.

first field collection has 3 fields, one of which is another field collection.
That field collection has 3 fields, one of which is another field collection.
That field collection has 5 fields, one of which is another field collection.
That field collection has 2 fields.

When there is a small amount of data within the node in these field collections, there is no problem. However each of these field collections has an unlimited number of values.

If I call the structure the following:
first field collection [1]
-second field collection [2]
--third field collection [3]
---fourth field collection [4]

A node which this is breaking has roughly the following:
2 collections of [1]
16 collections of [2] in first [1], and 1 inside of second [1]
an average of about 7 collections of [3] in each [2]
almost all [4]s are just one collection.

That means that as a rough estimate, there are:
2*[1] = 6 fields
17 * [2] = 51 fields
Roughly 119 * [3] = 595 fields
roughly 119 * [4] = 238 fields

So in total, that makes roughly 890 fields within my multi level field collections.

This is a real use case, and I could easily have more data when I start filling up more nodes with data I want to store.

If I have any fields beneath these field collections on the node edit form, their values do not save. If they are above the field collection they do however. This doesn't sound like too much of a problem... however it really is. It also effects the stuff at the very bottom of the node edit form, such as whether the node is published, promoted to front page, ETC ETC.

Do you know what I can do to remedy this? I am guessing nobody has used field collections in this way before as I can't find any other issue like this. You can see my use case here: http://placesinswansea.co.uk/content/gershwins-coffee-house (the field collections are at the bottom, in the tabs and fieldsets). I am still editing this at the moment so it might change slightly over time but the general structure will still be the same.

Comments

neddstark’s picture

neddstark CreditAttribution: neddstark commented

Hello and thank you for your feedback. I never used FC at this level, but it looks to me like a huge amount of fields. what errors do you get when trying to add new data? FC is a resource hog when used at this extreme level, with huge quantity of data. The first thing that comes in my mind: is your mysql server robust enough to handle such number of transactions?

pingu2k4’s picture

pingu2k4 CreditAttribution: pingu2k4 commented

Hi Neddstark,

It takes a while to load the node/%/edit overlay... maybe 30 seconds or so in firefox... (and recently I tried in chrome and it took around ten minutes...). However editing via that form shouldn't happen that often...

When you use the links to edit stuff from the node view, where it gives add edit and delete link for the FC items that's fine...

I think this is only effecting checkboxes beneath the FC now. As I can change the author, and that saves... but the published, promoted etc do not save... and I did have 3 fields in a fieldgroup beneath the menu until recently... 2 contained checkboxes... and the third field was a text field, which saved still. Also, with this amount of data... If I edit a field in the field collection it doesn't save.

Until recently... I was only allowing 1 price per item... However I need multiple prices. When I added a FC for pricing... This happened. So I tried it other ways, but even just adding one more field to the item FC caused this problem, so I think the problem is the number of fields, not the fact there are 4 FC's within each other. I did try using alternatives to using another FC for pricing, but every other option resulted the same way.

I do not get any errors... Thouh I am only looking in the recent log section... Where else should I be looking?

neddstark’s picture

neddstark CreditAttribution: neddstark commented

on such amount of data, I would suggest breaking the form into multiple steps, using the Multistep module. Then, to maximize performance, you should unset $form['other_steps_fields_names'] to prevent that data being inserted in the database. At least this would be my approach on this number of fields.

pingu2k4’s picture

pingu2k4 CreditAttribution: pingu2k4 commented

Hi Nedd, thanks for that. Before I had broken in intp multipages using multipage module, which just hid them using CSS / JS... So it was loading the entire form every time anyway.

I have split them up now, using https://drupal.org/project/msnf. This works great. I have all fields (including additional settings on one page, then the field collections on the second page. If you only need to access the other stuff, they load in a decent time, and you can save right from step 1. If you do need to hit the field collection system... you cna go to the next page, which does take ages to then load... but everything is saving great.

Other than the top level field collection, I am using the add / edit / delete links within node view, which makes things easier than using the node edit form. With that many fields... adding 1 more field collection takes a really long time. So I will do as much as possible from node view, not node edit.

Loading the second page of the node edit form where the field collections reside takes ages in Chrome still. (and by ages I mean over ten minutes). It seems to load all the form items fairly quickly... as the page has a tiny scrollbar, and you can see the top stuff... but the page goes unresponsive for way over 10 minutes... where mousing over a link will not change the cursor style to hand, you can't scroll.... or anything like that. (or the fieldset doesn't change its label into a link you can click to collapse it). I prefer Firefox myself anyway... and I don't mind warning the users of my site they should use FF and not chrome... but I was posting this bit for info on the problem.

Above, you mentioned "you should unset $form['other_steps_fields_names'] to prevent that data being inserted in the database."... I don't know how I would go about doing this... and I don't know what this would do. Should I be happy and stick with what is currently working in its current form... or would doing this provide something extra for me?

neddstark’s picture

neddstark CreditAttribution: neddstark commented

I am glead it works for you. I have never used msnf, I used instead https://drupal.org/project/multistep so I have no experience with msfn. I don't remember exactly why I choosed multistep instead msnf but I suspect the functionality is different. Anyway, the right approach is to split the huge form into as many steps as you can, to prevent overloading your browser. and the splitting should be a real page refresh next/previous, not using js+css technique, since that won't help you very much.

pingu2k4’s picture

pingu2k4 CreditAttribution: pingu2k4 commented

I tried that module first, but got a bunch of errors, so I used the other module instead. Seems to do pretty much the same thing...

I cannot split up the second page any more, because it only appears as one field on edit fields for content type, but its a field collection ofc, containing multiple others.

Does the "you should unset $form['other_steps_fields_names'] to prevent that data being inserted in the database." comment only apply to the multistep module, and not the one im using?

neddstark’s picture

neddstark CreditAttribution: neddstark commented

as far as I know, unsetting $form['other_steps_fields_names'] apply only to Multistep module.

pingu2k4’s picture

pingu2k4 CreditAttribution: pingu2k4 commented

Hi Nedd,

Thanks for your help on this. I have now fiddled about with it enough and got it to a point wher eI am completely happy.

Basicly, I have the add, edit and delete links for each field collection (other than the most embedded one, which rarely has more than 1 value, and I can't ever see a situation with more than 3... so having this inside the field collection above it is fine) visible to those that need it now.

Because I am displaying the top-level field collection as tabs... I couldn't have their edit delete and add links as normal... but I created a view to do this for me. The block is only shown to admin and content authors.

The other links were added the normal way.

Then I was looking at how to hide fields from the node edit form... without going down the route of CSS. I finally found the display format for the form of "hidden" for the field collections.

So, I have hidden the field completely, and it doesn't attempt to load anything form it for the node edit page... Meaning I no longer need multisteps, and there is no part to my node edit form which either takes ages to load, or breaks any of the data I put in. And all editing of the field collections is done now from the node view links. the ONLY thing which cannot be done without admin going in and changing a few things is re-ordering the menu's or Categories.... However this isn't that bad really... Attached an admin view of the node view page to show the links etc.

Thanks for your help with this! I no longer have a problem as I have this workaround. :) (though the problem still exists if you do keep it in node edit page with that much data in...)

jchatard’s picture

jchatard CreditAttribution: jchatard commented
Version: 7.x-1.0-beta5 » 7.x-1.x-dev

Hi all,

I have kind of the same problem as described here, but simpler, I guess.

1 content type, 1 field collection with 20 fields.

When I try to save a node with more than 40 fields, I have thousands of errors.

The problem I had was the PHP setting max_input_vars which on my server was by default to 1000. So as describe the values of the bottom of the form (status, url aliases, etc.) weren't saved because of a form validation error, because the form token wasn't reaching Drupal either.

So after increasing this value and also my Nginx client_max_body_size to 20M, I though I would be ok.

But still, on nodes with a huge amount of fields (actually huge here means 100 :-/), it breaks with this error reporting: https://gist.github.com/jchatard/bc19c2d3c7bfcbf95667

What I have found so far is that during the form processing, at some point, $form_state['node'] = $node; is not defined. And that is the only difference between a working node submission and a failing one. This leads to some issues with the field_collection_items validations and writing and they do not have track of the host entity, and for some reasons I still need to understand, the module (fc) tries to save form values but without being able to find the not the $field nor the $instance.

Sorry, this is really shitty explanation for now, I just need more time to understand what's going on.

So I'm digging all this right now. But if someone knows more on this I'd be more than happy to have this worked out :D

Thank you,
Jérémy

jchatard’s picture

jchatard CreditAttribution: jchatard commented
Category: Bug report » Support request

Ok, I fixed the problem.

What was happening was the call to cache_set('form_state_form*******', $data) failing silently because the Exception which should be caught has no code inside. So the cache of the $form_state wasn't in the database. This made the form process fail, well crash actually.

The reason it was failing was because of the cache_form table format, which was using Antelope as opposed to Barracuda and the field limitation was too small for my $form_state variable which was 5Mb.

So migrating to Barracuda and Dynamic table format did the trick.

Now, obviously this is too big anyway, so I'll have a look at https://www.drupal.org/project/msnf to see if I can make my $form and $form_state lighter.

Hope this helps some people around here :-)

Jérémy

PS: I'm changing the status of the issue, as this is not a bug related to the module, nor Drupal, but a MySQL setting, so it falls under support request.

jmuzz’s picture

jmuzz CreditAttribution: jmuzz commented
Status: Active » Closed (fixed)