Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.
When exporting a node, field_collection data is not exported.
This happens because the data for each field_collection is a separate entity. Node export references that entity using UUID, but there's no way to actually export those entities...
If node export could be made to work with entities instead of nodes... now that would be a major improvement. :) Failing that, we really could use support for field_collection export.
Comment | File | Size | Author |
---|---|---|---|
#51 | Screen Shot 2016-05-27 at 11.25.27.png | 110.63 KB | mibstar |
#50 | Screen Shot 2016-05-27 at 11.27.08.png | 136.64 KB | mibstar |
#39 | node_export-field_collection-1670740-39.patch | 10.55 KB | Ronino |
#35 | node_export-1670740-35-field-collection.patch | 6.93 KB | jamsilver |
#34 | node_export-field_collection-1670740-34.patch | 6.74 KB | jrb |
Comments
Comment #1
danielb CreditAttribution: danielb commentedNode export dependency will maintain the relationship to the correct field collection entity (supposedly - I haven't actually tried it - but the code that plugs in to support field collections is there). The only thing is; you'd have to find your own way to copy the field collection entity to the new site.
See, this is node export, and that is the only entity this module actually exports. Except for files, which it supported before they became entities. It would be good if it was an all-round entity export though, but it would require a considerable redesign of the module. And once we cross into that territory it will become a nightmare to maintain.
The problem so far with Node export is that it's not simply a matter of coming up with a way to convert a node object into some code and then converting it back upon import. There are all kinds of hacks and shit to support all the little fiddly functionalities that are common in nodes. These are things like menus, publishing options, and dependencies. Then there are considerations of where this module provides it's functionality, like on the node page, content admin page, VBO, etc... How to identify a node or collection of nodes, e..g through features or drush. You have to understand everything about a node, specifically, to be able to handle it. There is no standard list of these functionalities or any automatic way to get this information. The only way you find out about this stuff is people making issues when they encounter a limitation. Because of this, handling just one entity type, the node, has become a monstrous task.
Preparing an entity for export (and handling the import) is really something that should be part of Drupal core so everyone that makes an entity type is aware that it may be exported/imported and that there are standard things to consider in that regard. I'm not saying Drupal core needs to provide the actual export mechanism or any interfaces - just some standard way of saying "get me the exportable version of this" and then to be able to plug that object back in and have it saved properly.
The entity API has got us part way there, but it seems when designing the entity system nobody really thought about the full scope of how entities would be used. This includes export/import of the entities, and even obvious things like a consistent way to search for, represent, reference, and link to entities - these are challenges I face in another of my modules.
If I support field collection, a contrib module, I may as well support exporting users and taxonomy - which are core entities and probably more relevant to a larger audience. Don't get me wrong, I'd love to do that - I just don't have a plan for it right now. I guess the obvious thing would be to take everything 'node' related out of the module, and add support for nodes as a plugin - to open the door for other entities to be handled. The problem is, a bulk of this module is in fact node specific.
Right now I'm not far off a full release of Node Export in Drupal 7, and I'd like to just fix up the few remaining issues related to that, and then consider what a future version of this module should look like. It would probably be a tighter relationship with modules that deal with entities in a broader sense.
Comment #2
danielb CreditAttribution: danielb commentedOh I could be overestimating the challenge multiple entities will cause, for all I know Nodes could be the only entity with such complexity. I suppose I could play around with it and see what happens.
Comment #3
danielb CreditAttribution: danielb commentedI can't even get Field Collection to work with UUID. You mentioned you had it working - weird...
Comment #4
danielb CreditAttribution: danielb commentedlong story short: Field collection is not supported for now.
Comment #5
bcweaver CreditAttribution: bcweaver commentedFWIW, the project page for Node Export contains this text which seems a bit confusing
I couldn't find any modules called "node export dependency".
Comment #6
danielb CreditAttribution: danielb commentedEnsure you have the latest version.
Comment #7
AnybodyIt seems that there is still a problem with the order of uuid and field collection (views) installation. In one installation where I installed field collection first, everything is OK. In another where I first installed uuid for example the ID field inside views is missing for field collection views!
Any Ideas?
Comment #8
shenzhuxi CreditAttribution: shenzhuxi commentedThe dev version of uuid module already supported Field collection.
I patch node_export_dependency to add field_collection export/import. File field, reference fields, field_collection fields and etc. inside field_collection_item are not supported yet.
I think File export mode "Inline Base64" should be separated (maybe put into node_export_dependency) so it can be re-used not only for field_collection_item but also other entities.
Comment #9
Ronino CreditAttribution: Ronino commentedshenzhuxi, I tried your patch, thank you. Export works fine, but when trying to import my nodes with field collections with file fields, I get the following errors:
It obviously works for you with field collections without file fields?
Comment #10
shenzhuxi CreditAttribution: shenzhuxi commentedRonino, do you have file field in your field collection? It's not supported yet.
Comment #11
Ronino CreditAttribution: Ronino commentedYes, I do. But even if it's not supported, it should at least ignore them. Otherwise I can't use the export for any content type that has a file field in a field collection even if it's okay if the file data is left out.
Comment #12
shenzhuxi CreditAttribution: shenzhuxi commented@Ronino In this patch $field_collection->save() was used and field_collection module doesn't handle existing file entity during saving. I think we should patch field_collection module to fix that.
Comment #13
Ronino CreditAttribution: Ronino commentedshenzhuxi, I took your patch as a basis to create a new one which works for me.
Changes:
For me, it works for multi-value field collection items. It should work for multi-value fields inside the field collection itself, I doubt that it works with field collections containing field collections.
I first tried for quite a long time with FieldCollectionItemEntity::save() and EntityAPIController::save() which led to duplicated field collection item data. Then I found out that it's only necessary to add the data to the $node object. I don't think that patching field_collection is necessary.
Comment #14
brettbirschbach CreditAttribution: brettbirschbach commentedThe patch in #13 worked great for me, except for the file fields (which Ronino mentions as unsupported). In fact, there is also a bug in the workaround code that unsets the file field values in that it fails to unset file fields in data instances other than the first one when there are multiple data instances (i.e. multiple instances of the field collection with 1 file, not 1 instance of the field collection with multiple files).
At any rate, I wrote a quick patch that is meant to be IN ADDITION TO #13. #13 must be applied first.
This patch is meant to support the import of file fields. I basically stole the code from the node_export module and duplicated it here. Note that my patch also includes some lines of code from another file field patch @ http://drupal.org/node/1868450
Comment #15
jason.fisher CreditAttribution: jason.fisher commentedI am getting "corrupt patch" while using git apply on #14.
Comment #16
brettbirschbach CreditAttribution: brettbirschbach commentedSorry for that...might have had to do with the fact that I added some extra comments and such into my code after applying #13 and before creating #14.
I reverted my node_export module, and the cleanly applied patch #13 and my changes from #14, and re-rolled into a single patch. Please give this one a try.
Comment #17
Rhodungeon CreditAttribution: Rhodungeon commentedPatch #16 works perfectly.
To be committed!
Comment #18
kytom CreditAttribution: kytom commentedThis might be a newbie question, but applying #16 gives me this:
Am I doing something wrong?
Other than that, importing a single node with field_collection threw an error:
PDOException: in field_sql_storage_field_storage_write() (line 451 of /.../modules/field/modules/field_sql_storage/field_sql_storage.module).
I should note that the target site has some nodes added, containing field_collections of different type.
Comment #19
ekes CreditAttribution: ekes commentedRe-roll of #16 with
git diff
against head of 7.x-3.x Working nicely here, with one type of field collection. Anyone repeated #18?Comment #20
Ronino CreditAttribution: Ronino commentedHmm, I applied the patch from #19 and I don't get file data exported. node_export_dependency_handle_dependency() seems to import files via _node_export_file_field_import_file(), but I don't see any code in field_collection_node_export_dependency_field() to export file data.
Am I missing something?
Comment #21
Ronino CreditAttribution: Ronino commentedHitmanInWis wrote in #14:
Oookay. But how can we import files if they aren't exported?
I think we need to rework node_export_file_field_export() to handle field_collection items.
Comment #22
brettbirschbach CreditAttribution: brettbirschbach commentedMaybe check if the patch applied correctly. I haven't played with this in awhile, but I know that field collection export/import was working for me as I used it for a production deploy.
Also, I havent really compared patch #19 to #16 - might be worth checking if something got missed.
Lastly, its possible that something changed in the node_export module that makes this patch no longer work. Perhaps something you can look into?
Comment #23
Ronino CreditAttribution: Ronino commentedHitmanInWis wrote:
Did you really ever export files in field collections from one system and import in another one?
I mean, field_collection_node_export_dependency_field() in any of the patches does not export the real file data, only meta data like uri and filename, but not the file data itself. There is no e.g. base64 encoded binary data in the export files, so there is nothing to import.
Comment #24
brettbirschbach CreditAttribution: brettbirschbach commentedAhh...sorry...you mean the actual files are not being exported. I manually put the files into a folder, and imported them that way... Not ideal, but it was sufficient for my situation. So ya, I think there is probably more work to be done in that area.
Comment #25
Ronino CreditAttribution: Ronino commentedHitmanInWis wrote:
Yes ;-).
But sounds like a useful workaround. Thanks for the tip.
Definitely. Any volunteers?
Comment #26
vzblk CreditAttribution: vzblk commentedI'm trying to use patch#19 to export collection field with entity relation field (node) in this collection and failed. It is not enough information in exported data and i don't see code that can handle this information. So it is necessary to export UUID for referenced node and handle this during importing. I think the same is for other relation (user and taxonomy). I'm trying to make it work. And potentially we should look at field_collection_deploy module and try to use it.
Comment #27
ekes CreditAttribution: ekes commentedJust for clarity. We're using #19 with references to entities that have been exported & are then imported additionally. The problem was that the field collection itself wasn't exported, the patch does that, with UUID in. It doesn't seem to matter if the referenced entity is imported after the field collection either, the UUID still match up once it's there.
Comment #28
mpv CreditAttribution: mpv commented#19 worked for me, thank you!
Comment #29
Honza Pobořil CreditAttribution: Honza Pobořil commentedIt seems there is module Field Collection Deploy to export collections values.
Comment #30
loziju CreditAttribution: loziju commented#19 doesn't work for me when I have term reference field in field collection. UUID of the term is not showing anywhere (contrary to what ekes mentioned in #27).
Comment #31
caspervoogt CreditAttribution: caspervoogt commentedTried #19 but no field collection data was included in the export..
Comment #32
lil123 CreditAttribution: lil123 commentedHi,
Image fields in field collections where not imported. With this it seems to work :
Comment #33
onigoetz CreditAttribution: onigoetz commentedHi @lil123,
Your code seems to work, but with a little modification.
the "fid" has to be reset to be able to be imported or Drupal will use the provided fid, even if it doesn't exist.
this would give us :
Comment #34
jrbAfter applying the patch in #19, it still wasn't correctly creating the field collections for me.
In my case, I had a field collection called "Location" (field_location) that included fields for an address (field_address) and a phone number (field_phone). After import, I was seeing new data in field_data_field_address, field_data_field_phone, and field_data_field_location; but the row in field_collection_item wasn't being created.
The problem was that the SQL in the patch was only checking if the collection existed in the field that was the field collection (field_location, in my case). SQL in patch:
Since it found data in field_location, it never created the full field_collection entity. I've attached an updated patch that just changes the SQL to add a join to the field_collection_item table as well. This makes sure that the field collection entity is fully created. New SQL:
Now, I had also patched the Node Export module to make it work correctly with Organic Groups #2140075: Receiving EntityMalformedException: Missing bundle property on entity of type node when importing nodes with Organic Group dependencies, so it's possible that my problem may be related to how the patch in #19 works with that change. But, I think my new patch should work regardless of whether or not that patch is applied and shouldn't cause any additional issues.
Comment #35
jamsilver CreditAttribution: jamsilver commentedThat worked great except that it didn't support nested field collection fields.
I've just added a single extra line to add additional additional dependencies.
Comment #36
jrb#35 made it export the nested fields, but the import didn't work correctly.
I have a field collection with an entity reference field. It wanted to add the entity reference field to the node rather than to the field collection.
Comment #37
Ronino CreditAttribution: Ronino commentedI took the patch from #35 and added the functionality to really export and import file data. That means in exports you have the file contents of field_collection fields hashed in the field 'node_export_file_data' like in non-field_collection fields.
I did this by adapting node_export_file_field_export() and node_export_file_field_import() to work with nodes and field_collection items. The changes suggested in #32 and #33 are not necessary anymore as that works automatically with this patch (apart from the fact that those changes only added the file URI, not the real data to the export).
This is the first time for me that exporting nodes with files in field_collection fields on one system and importing them on a different system fully works.
Comment #38
Ronino CreditAttribution: Ronino commentedComment #39
Ronino CreditAttribution: Ronino commentedMy patch from #37 contained the code of another patch ;-). Here is a working version...
Comment #40
zrhoffman CreditAttribution: zrhoffman commentedAre there any updates on this? I would find this bugfix extremely useful.
Comment #41
applegamuza CreditAttribution: applegamuza commentedhopefully this module can be updated with this bug..
Comment #42
john.oltman CreditAttribution: john.oltman commentedMaybe this is what you need?
https://www.drupal.org/project/field_collection_deploy
Comment #43
rcross CreditAttribution: rcross at CrossFunctional commentedI see from the project page that this module supposedly supports both entity reference and field collection. Is this issue still relevant? Does it support including the referenced data in the export, or just the actual reference id?
Comment #44
dragon658 CreditAttribution: dragon658 commentedI have applyed #39 patch.
Field collection data was successfully exported to file.
But when I tried to import it to another site, images and body field of field collection were not exported.
Also I suspect that this solution does not support nested field collections.
I am going to write my own module for sync sites because of it. (field collection deploy module also does not work)
Did not find any good drupal module for syncing content.
Comment #45
rcross CreditAttribution: rcross at CrossFunctional commented@dragon658 - I think you misunderstand the point of this module and this issue.
You won't be able to import content across sites if the node types (and fields on that node) are the same. Look into the features module to managing changes of content types (i.e. node types) across sites.
This issue is about exporting of field collections. As the underlying structure of field collections are actually entities, and thus the node just holds a reference to that entity, then a basic node export would simply include the reference rather than the values. This issue is about how getting the values exported, as that is the expected/desired behaviour in most cases.
As for a more general solution to content synching, you would need to dig around more. Most cases, people are actually talking about content staging. (i.e. dev>stage>prod) rather than an arbitrary content synch. There are ways of achieving both, though from experience they all are a bit fragile and depend alot on the specific implementation. I would start looking here https://www.drupal.org/node/980186 but also check out https://groups.drupal.org/large-scale-drupal-lsd-projects-and-plans/cont... http://www.youtube.com/watch?v=akcK5DVaqA8 http://london2011.drupal.org/conference/sessions/content-staging-and-dep... (there is probably a video floating around online, but check out the slides) https://www.drupal.org/project/deploy https://www.drupal.org/project/content_staging
Comment #46
dragon658 CreditAttribution: dragon658 commentedHello, rcross!
Thank you for your answer!
I have read articles you provide above. (long time ago)
I have tried Deploy module and lots of other drupal modules for content syncing.
Existing modules are very laggy and does not allow to properly sync difficult content types. (unfortunatelly!)
My two sites are syncing by remote script every day.
So uuid's of content on both sites are the same.
I tried to use node export to sinc single nodes between sites on demand.
It is working good.
But it does not work good with field collections.
It does not support nested field collections and does not import field collection content to new site.
Comment #47
smndey CreditAttribution: smndey commentedhttps://www.drupal.org/node/1670740#comment-7061168 #8 Works fine for field_collection_item but doesn't provide support for files. #8 breaks the file export.
Comment #49
danielb CreditAttribution: danielb commentedComment #50
mibstar CreditAttribution: mibstar commentedHi danielb,
I've reviewed your commit and while it does make reference to nested item collections I've not been able to import an example I've been working on either via features nor node-import.
The node gets created but none of the item are present. I've attached a screenshot of the source and destination. Does this take into account multiple levels of nesting?
Comment #51
mibstar CreditAttribution: mibstar commentedIt seems like I can only attach one file at the time so here is the screenshot of the resulting import.
Comment #52
inversed CreditAttribution: inversed commentedTo any that may still be having trouble, make sure to enable the "Node export dependency (experimental)" module.