In the same vein as #1893914: Taxonomy terms should be able to export file fields - this is something I'd like to see.

There's a crossroad here where either this module steps aside to node_export or it becomes a more general solution.

If it is to do the latter, it must be able to version and deploy nodes along with their file attachments.

Comments

Jeffrey C.’s picture

I go with the latter. It's definitely something I would like to see in UUID Features.

saltednut’s picture

Status: Active » Needs review
StatusFileSize
new18 KB

Moved the functions introduced in #1893914: Taxonomy terms should be able to export file fields into uuid_features.module and gave them a more 'anonymous' treatment - this allows for nodes to also use them. Updated the settings page to reflect nodes. Changed the system variables a bit to make this work and modified both uuid_node and uuid_term to use the re-appropriated uuid_features_file_field_export and uuid_features_file_field_import.

sheldon rampton’s picture

Hi brantwynn. This looks like an interesting direction to go in. I'll test it when I get a chance.

FYI, I recently submitted a patch recently to the node_export module to make it do a better job of handling node export/imports involving entity references:

http://drupal.org/node/1896384

The issue is that entity references tend to break if the export contains multiple nodes whose node IDs are different upon import. For example, suppose you export the following nodes, which pertain to organizations and software applications and use entity references to specify which applications are being used by each organization, and vice versa.

Title Node ID Content type Entity reference field Referenced node ID
Drupal 1 application field_related_org 3
Wordpress 2 application field_related_org 4
Acquia 3 organization field_related_app 1
Humane Society 3 organization field_related_app 2

The entity references indicate that Drupal is being used by Acquia, and Wordpress is being used by the Human Society. Now let's supposed that after exporting these nodes, we import them into a Drupal website where there is already an "application" node with nid 1 for Joomla, and an "organization" with nid 2 for the Red Cross. Since nids 1 and 2 already exist, the exported nodes will be renumbered upon import, beginning with nid 3, so you will get the following:

Title Node ID Content type Entity reference field Referenced node ID
Drupal 3 application field_related_org 3
Wordpress 4 application field_related_org 4
Acquia 5 organization field_related_app 1
Humane Society 6 organization field_related_app 2

After the import, therefore, the "Drupal" and "Wordpress" nodes will each have entity references back to themselves, while the "Acquia" and "Humane Society" nodes will now have entity references back to "Joomla" and the "Red Cross," respectively, which is wrong. In order to correct this, the node import needs to correct the entity references so they look like this instead:

Title Node ID Content type Entity reference field Referenced node ID
Drupal 3 application field_related_org 5
Wordpress 4 application field_related_org 6
Acquia 5 organization field_related_app 3
Humane Society 6 organization field_related_app 4

My patch to the node_export module gets this working correctly for entity references, but it doesn't handle references created by the references module, which is different:

http://drupal.org/project/entityreference
http://drupal.org/project/references

If the UUID Features module is going to serve as a full alternative to node_export, it is going to have to handle entity references like these, and currently I don't think that functionality exists. UUID Features currently includes soe code that references the "nodereference" module, but no such module exists in Drupal 7. (There was a noderereference module included with CCK in Drupal 6, but in Drupal 7 the References and Entity Reference modules have replaced it.)

Anyway, these considerations don't bear directly on the patch you just made to add file handling, but they do highlight some of the work that nees to be done to make UUID Features function as a real practical alternative to the Node Export module.

saltednut’s picture

@sheldonramption -this is indeed valid and a good writeup, but needs to be the issue summary for a new issue and not buried in this simple issue about file exports for nodes. Can you create a new issue based around entity reference support?

sheldon rampton’s picture

Yes, I'll do that, thanks.

saltednut’s picture

Issue tags: +demo_framework

This still needs review. Bumping and adding tags.

sheldon rampton’s picture

@brantwynn: I'd like to do some testing on this, but I'm really slammed with client work right now. I'll see if I can find some time for testing it this weekend.

sheldon rampton’s picture

@brantwynn: Also, I'd just like to comment on your statement in the original ticket that "either this module steps aside to node_export or it becomes a more general solution. If it is to do the latter, it must be able to version and deploy nodes along with their file attachments must be able to version and deploy nodes along with their file attachments."

I agree with Jeffrey C. that it should aspire to do the latter. However, I think the true ambition of this module should be to "version and deploy fieldable entities along with their relationships to other Drupal content including files, taxonomy terms and relationships to other entities." That's a more ambitious goal than merely exporting nodes, which would have been adequate for Drupal 6 but not for Drupal 7 and beyond. I don't think that affects the scope of this particular ticket, but it may affect some decisions about how to architect the code for this module.

saltednut’s picture

@Sheldon Rampton - thanks for taking a look at this. I tried to write the patch in a way where it works more toward the "version and deploy fieldable entities along with their relationships to other Drupal content including files, taxonomy terms and relationships to other entities." goal for architecture.

Jeffrey C.’s picture

Hello guys,

Does the patch in #2 work?

saltednut’s picture

@Jeffrey C. - it needs further review. Works for me but I authored it so I can't RTBC it. Do you have time to test it?

jduhls’s picture

Got this when enabling a feature of about 60 nodes with text and image file fields. After reverting, if the node had an image field none of the date was imported.

Warning: Invalid argument supplied for foreach() in uuid_features_file_field_import() (line 301 of /var/www/vhosts/staging/drupal7/sites/all/modules/contrib/uuid_features/uuid_features.module).

php 5.3.x, postgresql 9.x, drupal 7.20

UPDATE: Disregard my post here - I've gone through many trial & error's with dev and non-dev modules and ended up using "Node Export" via drush to migrate my content (deadlines!). I'm not sure I can reproduce my error again nor do I remember the config that produced this error.

saltednut’s picture

Thanks for testing this @jduhls:

  1. Are you also versioning the content type for the 60 nodes in the Feature?
  2. Did you create the Feature w/ 60 nodes in it after applying the patch?
  3. When you say "none of the date was imported" did you mean the data was not imported or were there issues with the actual date fields on the nodes?

I don't think # of nodes should be an issue. What I am wonder is that if this doesn't return anything:
$fields = field_info_instances($entity_type, $import_bundle);
Then I would assume there would be an error similar to the one you saw.

It seems reasonable that Features may have just built the content type, but caches had not been cleared before the import would begin. So when we attempt to return the field information, its not there. I'll test this out.

jduhls’s picture

Added update to my previous post. It may just be a wild goose chase or not relevant - I was going through many different configs in a hurry. Thanks for your attention, however.

saltednut’s picture

@jduhls - Thanks for the quick response. I'd recommend use node_export if you're under a deadline. It is much more mature in this problem-space.

For example, though, to anyone looking to do further testing on this... I created a Feature that stores 75 nodes generated with devel. Each has a small image attached. I'm able to import them successfully using the patch in #2, however, for some reason the images are not inserted unless I revert after enabling the Feature.

saltednut’s picture

StatusFileSize
new907.5 KB

Here is the Feature mentioned in #15 for testing.

sheldon rampton’s picture

StatusFileSize
new4.17 KB

I tried creating my own feature (attached) and am getting errors when I try to enable it. The error messages are as follows:

Notice: Trying to get property of non-object in file_field_presave() (line 220 of /Users/sheldonrampton/Sites/drupal7/modules/file/file.field.inc).
Notice: Undefined property: stdClass::$uri in file_save() (line 570 of /Users/sheldonrampton/Sites/drupal7/includes/file.inc).
Notice: Trying to get property of non-object in file_field_presave() (line 220 of /Users/sheldonrampton/Sites/drupal7/modules/file/file.field.inc).
Notice: Undefined property: stdClass::$uri in file_save() (line 570 of /Users/sheldonrampton/Sites/drupal7/includes/file.inc).
PDOException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '' for key 'uri': INSERT INTO {file_managed} (filesize, status, timestamp, uuid) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3); Array ( [:db_insert_placeholder_0] => 0 [:db_insert_placeholder_1] => 1 [:db_insert_placeholder_2] => 1362349647 [:db_insert_placeholder_3] => fdc1f00b-4c0a-3c24-6158-9fbcca3e098c ) in drupal_write_record() (line 7120 of /Users/sheldonrampton/Sites/drupal7/includes/common.inc).
sheldon rampton’s picture

Oops, disregard my previous comment. I forgot to apply the patch before doing my testing. Duh.

sheldon rampton’s picture

StatusFileSize
new279.79 KB

Hm, I've reviewed this further, and it is working, sort of. I'm attaching an updated copy of the feature I created, which is now working, as is the feature that brantwynn uploaded previously. However, there's a catch. I still was getting getting error messages like the ones I listed above in #17 until I went to admin/config/content/uuid_features and saved the configuration to specify that files should be exported for nodes of my Article content type and taxonomy terms for my Navigation vocabulary.

If I uncheck all of the the CONTENT types at admin/config/content/uuid_features, I get the error messages above. If I uncheck all of the taxonomies, I don't get the error message.

That's as far as I've gotten with tracing the cause of this bug. It appears to me that the module is still trying to export some information about file attachments when nodes are exported, even if the content type of those nodes has been marked to omit file uploads for those content types, and this is what is causing the error message.

sheldon rampton’s picture

Here's a new patch that fixes the bug I noticed. I also removed the lines of code that were exporting configuration settings as part of the feature using variable_set(). This was creating a buggy situation in which an enabled feature would override and change those configuration settings even after I tried changing them via the configuration page. Strongarm does a better job of exporting and reimporting Drupal variables, so people should use that instead.

This raises somewhat the question of whether we should have configuration settings that give people the option of _not_ exporting file fields with certain taxonomies or content types. I think I could argue that those configuration settings are unnecessary and should be removed entirely, but it is also possible that someone will want to have the option of not exporting file fields with their features.

saltednut’s picture

This raises somewhat the question of whether we should have configuration settings that give people the option of _not_ exporting file fields with certain taxonomies or content types. I think I could argue that those configuration settings are unnecessary and should be removed entirely, but it is also possible that someone will want to have the option of not exporting file fields with their features.

I think this was a concept directly lifted from node_export. I do find it a bit cumbersome to have to turn them on or version them with the Feature as strongarm variables since they are not TRUE by default. Perhaps the best answer is to have these file export variables remain, but change them so that they are TRUE by default, and if people want to turn them off, they can do so.

saltednut’s picture

+    // Check if this field should implement file import/export system.
+    if (in_array($info['type'], $supported_fields)) {

Actually remembering now how this is stored and its, I suppose, not so easy to set bundles as part of the array by default. We'd have to introduce a fairly complex system of bundle management to make these turned on by default. So I guess I'm okay with the current system used here. We could perhaps chase the usability issue in a followup.

saltednut’s picture

Status: Needs review » Reviewed & tested by the community
StatusFileSize
new55 KB

Patch applies. Tested using 7.20 - I'd say this is ready to go. Example attached.

sheldon rampton’s picture

We could just change the wording and logic slightly so that instead of people checking boxes to INCLUDE file field support for specific content types and taxonomies, administrators check boxes to EXCLUDE certain content types and taxonomies.

saltednut’s picture

We could just change the wording and logic slightly so that instead of people checking boxes to INCLUDE file field support for specific content types and taxonomies, administrators check boxes to EXCLUDE certain content types and taxonomies.

That seems more confusing to me, honestly.

sheldon rampton’s picture

I tried changing the wording and logic as I suggested, but you're right that it seems more confusing that way. I think you're right that this is ready to be committed, and we can chase the usability issue in a followup.

Jeffrey C.’s picture

Alrighty. I'll commit it when I get to use my computer.

Jeffrey C.’s picture

Status: Reviewed & tested by the community » Fixed

Committed! Thanks! :).

saltednut’s picture

Status: Fixed » Closed (fixed)

Thanks for committing our patch Jeffrey!

Note: When it comes to patches worked on my multiple people, no need to attribute authorship to anyone else - though I'm sure Sheldon appreciates the credit. :D