Early Bird Registration for DrupalCon Portland 2024 is open! Register by 23:59 PST on 31 March 2024, to get $100 off your ticket.
Hi there,
It would be nice to have multilingual support for the migrate module.
The attached patch adds a destination handler for migrate which takes care about translations.
It's not a really elegant nor a fast solution - but it works. (At least in my test cases ;) )
The patch adds support for node and content based translation.
Comments
Comment #1
plachHi Peter, thanks for working on this.
I'm afraid I'll have to won't fix this issue (and any other similar one): the reason is Content Translation aims to replace a core module, hence adding dependencies to contrib modules might render core inclusion more difficult. An exception is the Title project which will have to make its path into core too, somehow. Any other module with strong possibility to get into core might be taken in consideration for integration, though.
I'd like sun's feedback about this before actually closing this issue, but IMO the right way to go is either moving this feature request to the Migrate queue or creating a separate project.
Comment #2
das-peter CreditAttribution: das-peter commentedI fully agree with you. This is nothing that has to go into core, I'll ask mikeryan (migrate maintainer) how we can handle this - because an own project would be just overhead.
I'd like to see better multilingual support in migrate itself anyway :)
Comment #3
sunBut why is this node-centric? (Sorry, if this is natural for Migrate module; didn't work with it yet)
Powered by Dreditor.
Comment #4
das-peter CreditAttribution: das-peter commentedThanks sun, you're right!
Looks like I mixed some stuff - beside the fact that it's ugly like hell ;)
I hope all this can be cleaned by changing some stuff in migrate itself.
Comment #5
sunAlright, I hope you don't mind if we mark this issue postponed for now then. I think we have loads of more important, "basic" todos for this project currently, and contrary to those, this issue sounds fairly advanced to me. I'd be happy to revisit this at some point, because I've heard that Migrate becomes a very important solution, so we logically should make a best effort to support it. But yeah, "later". :)
Comment #6
das-peter CreditAttribution: das-peter commentedNo problem - I need it anyway and it doesn't really matter where it resides. Thus I'll continue this in my project specific code. As soon as it's "later", I'll bring it back ;)
Btw: Let me know if you have some workpackages of the basic todos. I use the module already and I'm waiting for more feedback on #924968: Initial work - what's somehow related to this module. Means I'm interested in working on multilingual stuff ;)
Comment #7
das-peter CreditAttribution: das-peter commentedI'll try to keep my patch here up to date.
Comment #8
plachMoving to the Entity Translation queue. Read #902760: Change module name so it can work with Core translation for details.
Comment #9
bastnic CreditAttribution: bastnic commentedA quick update on this patch, seems to works with my use case.
Comment #10
bastnic CreditAttribution: bastnic commentedComment #12
bastnic CreditAttribution: bastnic commentedreroll without the path of my project
Comment #13
bastnic CreditAttribution: bastnic commentedComment #15
donquixote CreditAttribution: donquixote commentedYeah, I want this!
@das-peter,
how is this designed to work? Is one row = one language, or is one row = all languages at once?
If one row = one language, what is the primary key? As far as I remember, primary keys in migrate destinations have to be single-value integers, or not?
I guess, as you did not add any destination class (just a destination handler), it is one row = all languages at once. So the field handler needs to deal with arrays as field values.
Do rollback and --update work?
@plach, sun,
what about node_save() with entity_translation enabled, does this save all languages or just one translation?
Comment #16
plachAll field translations available in the
$entity
data structure are saved to the storage, It's the standard core behavior.Comment #17
donquixote CreditAttribution: donquixote commented@das-peter / bastnic:
One thing I still don't get...
This will only work, if the $entity->nid is already set.
So, either during an --update run, or if the migration (class) that imports the translations is a different one than the migration (class) that created the node itself.
How do your migration classes look like, that use this handler?
How do you get the translations from the source db into $entity in the first place?
I suppose, at the time that MigrateTranslationEntityHandler::prepare is called, this has already happened, and now the translations sit in $entity->translations->data[$lang] ?
Comment #18
donquixote CreditAttribution: donquixote commentedI'm getting a little confused, what the EntityTranslationDefaultHandler::setTranslation() method actually does.
In fact, it is only writing on the $entity object, which it keeps as a protected variable.
Is there any documentation about this stuff?
Comment #19
KarenS CreditAttribution: KarenS commentedJust a note that this could be a really useful feature to create a system to bulk update translations. As noted in another issue, lots of translations are done offsite and the 'translation' work is mostly doing lots of copy/paste into the forms. How much nicer would it be to ask for translations to be provided in something Migrate can use (XML or a spreadsheet or a database), map the source to the site, then use Migrate to bulk update everything at once? So getting Migrate working properly could be very useful.
Comment #20
bastnic CreditAttribution: bastnic commented@donquixote indeed, i began to save the node only in english in a first import, and then I do what @KarenS suggest to import all translations in another migrate import class.
Comment #21
das-peter CreditAttribution: das-peter commentedI'll need some time to get into this again.
As far as I remember I changed again some stuff to keep it working.
The approach I've used was to have a row per language. Where only the translatable content differs from row to row.
Comment #22
plachThat method just sets the updated translation records in the translation handler. These are subsequently stored through
EntityTranslationHandlerInterface::saveTranslations()
.Comment #23
donquixote CreditAttribution: donquixote commentedOk.
But saveTranslations() is not called in the patch in #12. Is this triggered by something else?
Comment #24
plachIn
entity_translation_field_attach_update()
throughnode_save()
.Comment #25
bastnic CreditAttribution: bastnic commentedI'm coming back on this ticket with a beautiful WTF effect. I really really did see my patch working but not anymore.
First :
can't work, we should have
Second,
... cause entity_translation not to work with (I think) migrate 2.3. So I did that:
I sincerely have no idea why my previous patch had ever worked!
Comment #26
sinasalek CreditAttribution: sinasalek commentedI replaced name and description of a vocabulary and i can longer migrate to it.
This is the error i'm getting
Comment #27
radiobuzzer CreditAttribution: radiobuzzer commentedHi, until this is resolved, I have to mention how I did it, in case this helps anybody. I used the "prepare" method of the Migrate api, which is added in the end of any Migration class.
In this example I am migrating two fields, name_en and name_el into one translatable field. I also know that the default language is 'el' for all of the nodes, but this can be easily changed
Comment #28
primozsusa CreditAttribution: primozsusa commentedHi, did anybody solved or has an example of using migrate with entity translation field. Maybe field handler example... #27 works kind of. if you have one record with multiple languages for import. but for taxonomy term with #27 the problem is that the data is not saved the same as if it would be done manually.
any suggestions welcome.
thx Primoz
Comment #29
mvdve CreditAttribution: mvdve commentedYou probably want to look at Issue 1069774.
There is complete description of how to create the translation. This can be done in the complete function.
Comment #30
make77 CreditAttribution: make77 commentedHi,
I updated the patch for the latest version of the module (7.x-1.0-beta3).
Comment #32
steinmb CreditAttribution: steinmb commentedBroken patch. This patch apply cleanly on dev. Let's see if bot also is happy.
Comment #33
greenjuls CreditAttribution: greenjuls commentedHi, I just updated to version 7.x-1.0-beta3 of entity_translation module but it broke my migrations.
Problem is that the translations were not loaded in
MigrateTranslationEntityHandler::prepare()
.I replaced the following piece of code
with:
Comment #34
plopescHello
I'm trying to use migrate and enityt_translation with no luck, and I think this patch can be helpful.
Could you include an example of how are you implementing it?
Thank you in advance
Comment #35
primozsusa CreditAttribution: primozsusa commentedWhat finally worked for me:
- make sure your filed is set translatable in UI
- mapping
- prepareRow
- prepare
- complete: to update entity_translation table
Comment #36
brockfanning CreditAttribution: brockfanning commentedI may be missing something, but the latest patch in #33 didn't seem sufficient for my needs. It's limited to nodes, and also limited to existing nodes, which I don't think was intended. It also has a section about supporting "node-based translations" which I'm not clear on, as I thought that entity_translation was the alternative to node-based translations. Also I don't believe it is registering the destination handler correctly.
Given all that I wrote a new patch that is working for me.
The way I'm using it is to map an array of language codes to each translatable destination field's ":language" subfield, and a corresponding array of values to the field itself. For example, something like this in the migration constructor:
Where $row->legacy_subtitle might be...
And $row->legacy_subtitle_languages would be...
Also in combination with migrate_d2d and this patch: https://www.drupal.org/node/2389783 the mapping can be as simple as (assuming a D7 multilingual to D7 multilingual migration):
Comment #37
heldercor CreditAttribution: heldercor commented@brockfanning, doesn't this break fields with multiple values?
Comment #38
brockfanning CreditAttribution: brockfanning commented@heldercor, not that I know of, though in my testing all translatable fields have been single-value. In theory it would be the same, but the $row value would be an array of arrays, like:
But as said, I haven't tested that.
Comment #39
nedjoThanks, this is working well on my testing.
For anyone who like me is wondering why it works to set the value and a language subfield as arrays, the answer is in migrate's field handling--see MigrateFieldsEntityHandler.
A couple of minor suggestions:
Maybe default this to
$entity->status
if set?Could use
REQUEST_TIME
rather than callingtime()
.Comment #40
kristiaanvandeneyndeWorking great on my tests!
@nedjo in #39:
I also cleaned up some whitespaces and moved the hook to entity_translation.migrate.inc as suggested by the Migrate docs.
Seeing as everyone likes the patch and the change from #39 is small, I'm going to RTBC it immediately after testbot goes green.
Comment #41
kristiaanvandeneyndeComment #42
.bert CreditAttribution: .bert commented@brockfanning & @kristiaanvandeneynde the patch in #40 (essentially the same as #36) doesn't seem to work for multiple values on a single field.
It could be my setup, but it works beautifully on single values. Any thoughts on where to look?
@heldercor, why did you think it might break on multiple values?
If anyone can point me in the right direction of where to start to troubleshoot this, that would be greatly appreciated.
Thanks
Comment #43
kristiaanvandeneyndeHow did you provide the values? Did you map them as specified in #38?
Edit: To clarify, this patch only makes sure $entity->translations is properly set. The whole value populating mechanism is part of Migrate's "subfield" system.
Comment #44
.bert CreditAttribution: .bert commentedYes, the values are mapped out as described in #38.
Here's the basics of what we're doing.
The set_descriptions() method grabs the data and applies it to the translation fields
Adding multiple values in this fashion works fine on it's own without using entity translation, as seen in $row->supplemental_description. I also tried using an array to add the language data, like this:
So far, it will only add single values and will not add multiple values. I will attempt to troubleshoot, but if anyone has some guidance that would be greatly appreciated.
Thanks.
Comment #45
.bert CreditAttribution: .bert commentedI can confirm that the method described in #38 does not work as described. Instead, the following works for that example when importing multiple values:
Basically, you're setting the language on the subfield array using the same key as the value array. I'm not sure exactly "why", but appears to run without issue for our setup.
From my previous comment, the following works instead:
The other thing to note is that $row->language does not need to be set in prepareRow().
I hope this helps someone down the line.
Comment #46
kristiaanvandeneyndeVery nice write-up, I'll try to confirm this later on and see if this is indeed intended to work that way for subfields.
Comment #47
gambryI implemented my own handler to deal with this, but I tested the patch anyway hoping to have pathauto/alias support. However even with the #40 patch, migrated translations still don't have aliases.
Is translations aliasing on migrating working for any of you?
I can see different issues about entity_translation and pathauto, but not of them refer to migrate.
I'll start having a look to find out why it doesn't create aliases, but it would be good if there is already a solution somewhere.
Thanks and great work!
Comment #48
gambrySo far the only solution I found is this, but to properly work with the #40 it needs to be inside the
complete()
handler.Waiting for any better/official solution, I'll leave my comment here for those poor souls seeking help. :)
(If a better/official solution does NOT exist, I'm happy to update the patch).
Comment #49
kristiaanvandeneyndeThat should probably go in some sort of PathautoMigrateDestinationHandler and be committed in that module.
Comment #50
gambry@kristiaanvandeneynde I don't think that should be part of a Pathauto handler, but definitelly is NOT part of this Entity Translation handler.
My comment is a hint to whoever lands to this issue finding a solution on migrating/programmatically-creating an entity translation + URL aliases, and it can be ignored for the real purpose of the issue.
Comment #51
estoyausenteThe patch in #40 work correctly.
I apply it and change my migration code and the entities (nodes in this case) are imported correctly.
In the construct:
PrepareRow method:
I hope that this example helps somebody :)
Comment #52
kristiaanvandeneyndeI can confirm the method .bert describes in #45 works for multi-value fields. I normally double-check this by analyzing the flow of the code, but time is short right now so you'll have to take .bert's and my word for it :)
Comment #53
mikeryanDoing my first migration involving entity_translation, and I'm very happy to see this patch here. On a source code scan, all that jumped out at me is the one nit below (not worth moving out of RTBC for that). It's late in the day here, but tomorrow I'll try the patch for reals and report back on my experience.
Thanks!
Nit: entities.
Comment #54
mikeryanWell, I made the patch, reran my migration (which had previously properly migrated field translations, but not the entity_translation table stuff so the UI showed that it was translated), and it worked perfectly.
+1 RTBC from me!
Comment #55
rymoanother +1 RTBC here for #40
Comment #57
plachCommitted and pushed #40, awesome work, thank you all!