This bug is triggered by the query ran by the components in the implementations of hook_features_export_options(), when the query returns a large number of objects (files, nodes, terms), in my case more than 100 000 nodes. The options array parsed for the feature create form becomes huge and Drupal quickly runs into allowed memory size exhausted error.

My suggestion is to create an administration screen for UUID Features and expose options/limits/filters on object selection queries from the implementations of hook_features_export_options(). Any thoughts?

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

jgraham’s picture

Status: Active » Needs review
FileSize
2.26 KB

Just ran into this on a site here. I would suggest a two-route way to resolve this. The below approach only focuses on nodes, but should be similarly and appropriately applied elsewhere.

  1. Limit the node types that are queried in uuid_node_features_export_options(), via a variable exposed on an admin config screen.
  2. Add a drupal_alter() at the end of uuid_node_features_export_options() to allow site-specific overrides for the exportable nodes.

Attached patch implements the above described approach for nodes.

jgraham’s picture

  1. Updated node patch to default to all node types so that this is consistent with the existing behavior.
  2. Provided similar implementation to limit the exportable taxonomy terms. This is limited by vocabulary and additionally can be limited by implementing hook_uuid_term_features_export_options_alter() similar to the available uuid_node.
jgraham’s picture

Same patch as 2, but with titles for admin settings page.

jgraham’s picture

Updated patch from 3 adjusted to use taxonomy machine names rather than vids so that this is features safe for exporting via strongarm.

jgraham’s picture

Oy, sorry for the issue spam. Missed a call to taxonomy_get_vocabularies() in uuid_features_admin().

happysnowmantech’s picture

I encountered this problem too. I re-rolled the patch in #5 with a few fixes (implement hook_permission() and also check if any node types or vocabularies are actually selected). As far as I can tell, though, it still doesn't solve the problem of being able to use UUID feature integration with node types that have a large number of nodes, my website still runs out of memory.

Jeffrey C.’s picture

Issue tags: +Need tests

Adding the tag.

steve.elkins’s picture

Category: bug » feature
Status: Needs review » Patch (to be ported)
FileSize
3.38 KB

Here's another patch borrowing from #6 for the latest 7.x-1.x-dev. Had to rearrange a bit to account for the changes introduced by #1904946: Nodes should be able to export file fields and #1893914: Taxonomy terms should be able to export file fields.

askibinski’s picture

Patch at #8 works nice with latest -dev, thanks.

However I agree with #6 that this patch doesn't really solve the problem. If you would have 1 content-type selected which has a lot of nodes, memory limits will occur.

The 'drupal-way' to fix this would be an autocomplete field instead of checkboxes for each node. (maybe showing an autocomplete if more than > 100 nodes or something). But I don't know if features allows for this kind of UI. And also, an autocomplete is not the most user friendly thing on earth.

askibinski’s picture

saltednut’s picture

Status: Patch (to be ported) » Needs review
saltednut’s picture

#8: uuid_features-1719414-8.patch queued for re-testing.

saltednut’s picture

However I agree with #6 that this patch doesn't really solve the problem. If you would have 1 content-type selected which has a lot of nodes, memory limits will occur.

I think this is a problem with the Features UI. Features 2.0 should be better about this and also lazy-load the checkboxes but it doesn't.

This patch you guys have been working on is really great because it at least gives us control to prevent specific node bundles from destroying Features admin screens.

I think that anything further than this is a problem that Features UI should handle. They have a few options - maybe lazy loading checkboxes in JS or they could provide pagers. I'm not a UX expert but thats my advice for them. Features still needs to know what to do when a call for 3,000 checkboxes is made.

Because they don't do this for us, we cant assume the act of UUID Features Integration attempting to provide exportables for all the nodes on your site is always safe.

So, I think it should be turned off by default. This patch adjusts the new settings so that the particular bundle must first be turned on in at admin/config/content/uuid_features order to make uuid_features expose them to Features.

Some other additional changes added here:

@@ -96,6 +96,22 @@ function uuid_features_settings($form, &$form_state) {
+  $form['entity']['uuid_features_entity_node_types'] = array(
+    '#type' => 'checkboxes',
+    '#title' => t('Exportable node types'),
+    '#description' => t('Which content types should be exportable?'),
+    '#default_value' => variable_get('uuid_features_entity_node_types', array()),
+    '#options' => $bundles['node'],
+  );
+
+  $form['entity']['uuid_features_entity_taxonomy_term_vocabularies'] = array(
+    '#type' => 'checkboxes',
+    '#title' => t('Exportable term vocabularies'),
+    '#description' => t("Which taxonomy vocabulary's terms should be exportable?"),
+    '#default_value' => variable_get('uuid_features_entity_taxonomy_term_vocabularies', array()),
+    '#options' => $bundles['taxonomy_term'],
+  );

Adding this twice was unnecessary so I rewrote it in a way that could support our uuid_entity types.

I decided to introduce hook_entity_info_alter as well so that we can keep easier track of what entities uuid_features support - I then used that information to build the settings form.

The next thing I realized though is that beans didn't yet support file field exports (though they easily could) so I went ahead and added that too just so this would completely work.

saltednut’s picture

#13: uuid_features-1719414-13.patch queued for re-testing.

paolomainardi’s picture

This patch works as expected, but i have some strange warnings when i use drush cc:

➜  uuid_features git:(dev) ✗    drush cc all
Invalid argument supplied for foreach() field.info.inc:220                                                                                                                  [warning]
Invalid argument supplied for foreach() og.module:2343                                                                                                                      [warning]
Invalid argument supplied for foreach() og.module:2361                                                                                                                      [warning]
Invalid argument supplied for foreach() og.module:2361                                                                                                                      [warning]
class_implements(): object or string expected entity.module:84                                                                                                              [warning]
in_array() expects parameter 2 to be array, boolean given entity.module:84                                                                                                  [warning]
Invalid argument supplied for foreach() og.module:2383                                                                                                                      [warning]
Invalid argument supplied for foreach() og.module:2343                                                                                                                      [warning]
Invalid argument supplied for foreach() og.module:2383                                                                                                                      [warning]
Invalid argument supplied for foreach() og.module:2343                                                                                                                      [warning]
Invalid argument supplied for foreach() og.module:2361                                                                                                                      [warning]
Invalid argument supplied for foreach() og.module:2406                                                                                                                      [warning]
Invalid argument supplied for foreach() og.module:2361                                                                                                                      [warning]
Invalid argument supplied for foreach() og.module:2343                                                                                                                      [warning]
Invalid argument supplied for foreach() og.module:2361                                                                                                                      [warning]
Invalid argument supplied for foreach() og.module:2361                                                                                                                      [warning]
Invalid argument supplied for foreach() og.module:2361                                                                                                                      [warning]
Invalid argument supplied for foreach() og.module:2361                                                                                                                      [warning]
Invalid argument supplied for foreach() og.module:2361                                                                                                                      [warning]
Invalid argument supplied for foreach() og.module:2361                                                                                                                      [warning]
Invalid argument supplied for foreach() og.module:2361                                                                                                                      [warning]
Invalid argument supplied for foreach() og.module:2361                                                                                                                      [warning]
Invalid argument supplied for foreach() field.info.inc:220                                                                                                                  [warning]
in_array() expects parameter 2 to be array, null given field.crud.inc:92                                                                                                    [warning]
in_array() expects parameter 2 to be array, null given field.crud.inc:92                                                                                                    [warning]
in_array() expects parameter 2 to be array, null given field.crud.inc:92                                                                                                    [warning]
in_array() expects parameter 2 to be array, null given field.crud.inc:92                                                                                                    [warning]
Invalid argument supplied for foreach() field.info.inc:220                                                                                                                  [warning]
Invalid argument supplied for foreach() field.info.inc:220                                                                                                                  [warning]
Invalid argument supplied for foreach() field.info.inc:220                                                                                                                  [warning]
Invalid argument supplied for foreach() field.info.inc:220                                                                                                                  [warning]
Invalid argument supplied for foreach() field.info.inc:220                                                                                                                  [warning]
Invalid argument supplied for foreach() field.info.inc:220                                                                                                                  [warning]
Invalid argument supplied for foreach() field.info.inc:220                                                                                                                  [warning]
'all' cache was cleared in self                                                                                                                                             [success]
The Administrator role has been reset for all permissions.   
joseph.olstad’s picture

No problems with 'drush cc' in my environment using a rollup patch on the latest 7-1.x dev build as of dec 4, 2013.

The report of the "drush cc" issue is unconfirmed. I applied the patches and ran "drush cc all"
'all' cache was cleared in /path/vhost#default [success]

see: https://drupal.org/node/2149949

joseph.olstad’s picture

Issue summary: View changes
Status: Needs review » Closed (duplicate)

see: solve uuid hangs on too many nodes - multiple patch rollup on dec 4 dev build
#2149949: Multiple Patches need to be committed to UUID Features and have been rolled into one.

saltednut’s picture

Thanks for working on this. BTW, you can use brackets to link to issues to save yourself some time.

#2149949: Multiple Patches need to be committed to UUID Features and have been rolled into one.

joseph.olstad’s picture

Assigned: Unassigned » joseph.olstad
Status: Closed (duplicate) » Needs work
joseph.olstad’s picture

Special thanks to the great work of the people HERE (jgraham, happysnowmantech, steve.elkins, brantwynn )as well as Spotzero and others that designed and contributed to this patch.

Summary howto

for others who want to test this #2149949: Multiple Patches need to be committed to UUID Features and have been rolled into one.

test scenario:

  • 138000 nodes on 'content type C'
  • 60 nodes on 'content type G'

This patch adds the following functionality

  • allows you to select /unselect node bundles before hitting the features screen.
  • effectively allows you to select what you want to include in your feature without overloading the features admin screen.

Howto

  1. Selection form is found in: "Configuration" -> "Content Authoring" -> "UUID Features Integration" aka admin/config/content/uuid_features.
  2. After selections go to the Features "create" form.
  3. Create feature selecting nodes of 'content type G' with uuid's

Expected results:

  • before patch: crash, WSOD , CANNOT exclude 'content type C'
  • after patch: exclude node bundles except desired 'content type G', everything works
saltednut’s picture

Status: Needs work » Closed (duplicate)

This was broken into two issues. Let's just close this one and continue on the other one since that is where the latest patch is.

#2149949: Multiple Patches need to be committed to UUID Features and have been rolled into one.