Hi there,

It would be nice to have multilingual support for the migrate module.
The attached patch adds a destination handler for migrate which takes care about translations.
It's not a really elegant nor a fast solution - but it works. (At least in my test cases ;) )

The patch adds support for node and content based translation.

Files: 
CommentFileSizeAuthor
#33 translation-migrate-support-929402-33.patch4.99 KBgreenjuls
PASSED: [[SimpleTest]]: [MySQL] 721 pass(es).
[ View ]
#32 translation-migrate-support-929402-32.patch5 KBsteinmb
PASSED: [[SimpleTest]]: [MySQL] 721 pass(es).
[ View ]
#30 translation-migrate-support-929402-30.patch5.67 KBmake77
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch translation-migrate-support-929402-30.patch. Unable to apply patch. See the log in the details link for more information.
[ View ]
#12 translation-migrate-support-929402-9.patch5.15 KBbastnic
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch translation-migrate-support-929402-9_1.patch. This may be a -p0 (old style) patch, which is no longer supported by the testbots.
[ View ]
#9 translation-migrate-support-929402-9.patch5.71 KBbastnic
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch translation-migrate-support-929402-9_0.patch. This may be a -p0 (old style) patch, which is no longer supported by the testbots.
[ View ]
#7 translation-migrate-support-929402-7.patch4.46 KBdas-peter
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch translation-migrate-support-929402-7.patch. This may be a -p0 (old style) patch, which is no longer supported by the testbots.
[ View ]
translation-migrate-support.patch4.45 KBdas-peter
PASSED: [[SimpleTest]]: [MySQL] 0 pass(es).
[ View ]

Comments

Hi Peter, thanks for working on this.

I'm afraid I'll have to won't fix this issue (and any other similar one): the reason is Content Translation aims to replace a core module, hence adding dependencies to contrib modules might render core inclusion more difficult. An exception is the Title project which will have to make its path into core too, somehow. Any other module with strong possibility to get into core might be taken in consideration for integration, though.

I'd like sun's feedback about this before actually closing this issue, but IMO the right way to go is either moving this feature request to the Migrate queue or creating a separate project.

I fully agree with you. This is nothing that has to go into core, I'll ask mikeryan (migrate maintainer) how we can handle this - because an own project would be just overhead.
I'd like to see better multilingual support in migrate itself anyway :)

+++ includes/translation.migrate.inc 1 Jan 1970 00:00:00 -0000
@@ -0,0 +1,106 @@
+        // Load old node to make sure it's the right id - mapping of migrate is not language sensitive
+        $current_node = node_load($entity->nid);

But why is this node-centric? (Sorry, if this is natural for Migrate module; didn't work with it yet)

Powered by Dreditor.

Thanks sun, you're right!
Looks like I mixed some stuff - beside the fact that it's ugly like hell ;)
I hope all this can be cleaned by changing some stuff in migrate itself.

Status:Needs review» Postponed

Alright, I hope you don't mind if we mark this issue postponed for now then. I think we have loads of more important, "basic" todos for this project currently, and contrary to those, this issue sounds fairly advanced to me. I'd be happy to revisit this at some point, because I've heard that Migrate becomes a very important solution, so we logically should make a best effort to support it. But yeah, "later". :)

No problem - I need it anyway and it doesn't really matter where it resides. Thus I'll continue this in my project specific code. As soon as it's "later", I'll bring it back ;)
Btw: Let me know if you have some workpackages of the basic todos. I use the module already and I'm waiting for more feedback on #924968: Initial work - what's somehow related to this module. Means I'm interested in working on multilingual stuff ;)

StatusFileSize
new4.46 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch translation-migrate-support-929402-7.patch. This may be a -p0 (old style) patch, which is no longer supported by the testbots.
[ View ]

I'll try to keep my patch here up to date.

Project:Content translation» Entity Translation
Version:7.x-2.x-dev» 7.x-1.x-dev

StatusFileSize
new5.71 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch translation-migrate-support-929402-9_0.patch. This may be a -p0 (old style) patch, which is no longer supported by the testbots.
[ View ]

A quick update on this patch, seems to works with my use case.

Status:Postponed» Needs review

Status:Needs review» Needs work

The last submitted patch, translation-migrate-support-929402-9.patch, failed testing.

StatusFileSize
new5.15 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch translation-migrate-support-929402-9_1.patch. This may be a -p0 (old style) patch, which is no longer supported by the testbots.
[ View ]

reroll without the path of my project

Status:Needs work» Needs review

Status:Needs review» Needs work

The last submitted patch, translation-migrate-support-929402-9.patch, failed testing.

Yeah, I want this!

@das-peter,
how is this designed to work? Is one row = one language, or is one row = all languages at once?
If one row = one language, what is the primary key? As far as I remember, primary keys in migrate destinations have to be single-value integers, or not?
I guess, as you did not add any destination class (just a destination handler), it is one row = all languages at once. So the field handler needs to deal with arrays as field values.
Do rollback and --update work?

@plach, sun,
what about node_save() with entity_translation enabled, does this save all languages or just one translation?

what about node_save() with entity_translation enabled, does this save all languages or just one translation?

All field translations available in the $entity data structure are saved to the storage, It's the standard core behavior.

@das-peter / bastnic:
One thing I still don't get...

<?php
   
if (entity_translation_enabled($entity_type) && property_exists($entity, 'nid') && $entity->nid) {
?>

This will only work, if the $entity->nid is already set.
So, either during an --update run, or if the migration (class) that imports the translations is a different one than the migration (class) that created the node itself.

How do your migration classes look like, that use this handler?
How do you get the translations from the source db into $entity in the first place?

I suppose, at the time that MigrateTranslationEntityHandler::prepare is called, this has already happened, and now the translations sit in $entity->translations->data[$lang] ?

I'm getting a little confused, what the EntityTranslationDefaultHandler::setTranslation() method actually does.
In fact, it is only writing on the $entity object, which it keeps as a protected variable.
Is there any documentation about this stuff?

Just a note that this could be a really useful feature to create a system to bulk update translations. As noted in another issue, lots of translations are done offsite and the 'translation' work is mostly doing lots of copy/paste into the forms. How much nicer would it be to ask for translations to be provided in something Migrate can use (XML or a spreadsheet or a database), map the source to the site, then use Migrate to bulk update everything at once? So getting Migrate working properly could be very useful.

@donquixote indeed, i began to save the node only in english in a first import, and then I do what @KarenS suggest to import all translations in another migrate import class.

I'll need some time to get into this again.
As far as I remember I changed again some stuff to keep it working.

The approach I've used was to have a row per language. Where only the translatable content differs from row to row.

I'm getting a little confused, what the EntityTranslationDefaultHandler::setTranslation() method actually does.
In fact, it is only writing on the $entity object, which it keeps as a protected variable.
Is there any documentation about this stuff?

That method just sets the updated translation records in the translation handler. These are subsequently stored through EntityTranslationHandlerInterface::saveTranslations().

Ok.
But saveTranslations() is not called in the patch in #12. Is this triggered by something else?

In entity_translation_field_attach_update() through node_save().

I'm coming back on this ticket with a beautiful WTF effect. I really really did see my patch working but not anymore.

First :

      // Content based translation
      if (in_array($entity->type, variable_get('entity_translation_entity_types', array()))) {

can't work, we should have

      // Content based translation
      if (entity_translation_node_supported_type($entity->type)) {

Second,

// Preserve original language setting
$entity->language = $entity->translations->original;

... cause entity_translation not to work with (I think) migrate 2.3. So I did that:

--- a/entity_translation/includes/translation.migrate.inc
+++ b/entity_translation/includes/translation.migrate.inc
@@ -61,6 +61,7 @@ class MigrateTranslationEntityHandler extends MigrateDestinationHandler {
           );
         }
         // Preserve original language setting
+        $entity->field_language = $entity->language;
         $entity->language = $entity->translations->original;
       }
       // Node based translation
--- a/migrate/plugins/destinations/fields.inc
+++ b/migrate/plugins/destinations/fields.inc
@@ -91,6 +91,8 @@ abstract class MigrateFieldHandler extends MigrateHandler {
         return LANGUAGE_NONE;
       case isset($arguments['language']):
         return $arguments['language'];
+      case !empty($entity->field_language) && $entity->field_language != LANGUAGE_NONE:
+        return $entity->field_language;
       case !empty($entity->language) && $entity->language != LANGUAGE_NONE:
         return $entity->language;
         break;

I sincerely have no idea why my previous patch had ever worked!

Component:Code» Base system

I replaced name and description of a vocabulary and i can longer migrate to it.
This is the error i'm getting

SQLSTATE[21S01]: Insert value list does not match column list: 1136 Column count doesn't match value count at row 1: INSERT INTO {field_data_description_field} (entity_type, entity_id, revision_id, bundle, delta, language, description_field_value, description_field_summary, description_field_format) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5, :db_insert_placeholder_6_0, :db_insert_placeholder_6_arguments, :db_insert_placeholder_7, :db_insert_placeholder_8); Array ( [:db_insert_placeholder_0] => taxonomy_term [:db_insert_placeholder_1] => 9783 [:db_insert_placeholder_2] => 9783 [:db_insert_placeholder_3] => food [:db_insert_placeholder_4] => 0 [:db_insert_placeholder_5] => en [:db_insert_placeholder_7] => [:db_insert_placeholder_8] => plain_text [:db_insert_placeholder_6_0] => [:db_insert_placeholder_6_arguments] => Array ( [format] => filtered_html ) ) (modules/field/modules/field_sql_storage/field_sql_storage.module:448)

Hi, until this is resolved, I have to mention how I did it, in case this helps anybody. I used the "prepare" method of the Migrate api, which is added in the end of any Migration class.

In this example I am migrating two fields, name_en and name_el into one translatable field. I also know that the default language is 'el' for all of the nodes, but this can be easily changed

<?php
class RegionMigration extends Migration{
  public function
__construct() {
    ...
  }
  function
prepare($entity, stdClass $row){
   
$entity->language = 'el';
   
$entity->title_field['el'][0]['value'] = $row->name;
   
$entity->title_field['en'][0]['value'] = $row->name_en;
   
$entity->translations = (object) array(
           
'original' => 'el',
           
'data' => array(
                   
'el' => array(
                           
'entity_type' => 'node',
                           
'entity_id' => $entity->nid,
                           
'language' => 'el',
                           
'source' => '',
                           
'uid' => '0',
                           
'status' => '1',
                           
'translate' => '0',
                    ),
                   
'en' => array(
                           
'entity_type' => 'node',
                           
'entity_id' => $entity->nid,
                           
'language' => 'en',
                           
'source' => 'el',
                           
'uid' => '1',
                           
'status' => '1',
                           
'translate' => '0',
                    ),
            )
    );
    return
$entity;
  }
}
?>

Hi, did anybody solved or has an example of using migrate with entity translation field. Maybe field handler example... #27 works kind of. if you have one record with multiple languages for import. but for taxonomy term with #27 the problem is that the data is not saved the same as if it would be done manually.
any suggestions welcome.
thx Primoz

You probably want to look at Issue 1069774.
There is complete description of how to create the translation. This can be done in the complete function.

Issue summary:View changes
Status:Needs work» Needs review
StatusFileSize
new5.67 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch translation-migrate-support-929402-30.patch. Unable to apply patch. See the log in the details link for more information.
[ View ]

Hi,

I updated the patch for the latest version of the module (7.x-1.0-beta3).

Status:Needs review» Needs work

The last submitted patch, 30: translation-migrate-support-929402-30.patch, failed testing.

Status:Needs work» Needs review
StatusFileSize
new5 KB
PASSED: [[SimpleTest]]: [MySQL] 721 pass(es).
[ View ]

Broken patch. This patch apply cleanly on dev. Let's see if bot also is happy.

StatusFileSize
new4.99 KB
PASSED: [[SimpleTest]]: [MySQL] 721 pass(es).
[ View ]

Hi, I just updated to version 7.x-1.0-beta3 of entity_translation module but it broke my migrations.
Problem is that the translations were not loaded in MigrateTranslationEntityHandler::prepare().

I replaced the following piece of code

<?php
// Load translations if necessary
if (!property_exists($entity, 'translations')) {
 
$entity->translations = $translation_handler->getTranslations();
}
?>

with:

<?php
// Load translations
$translation_handler->loadTranslations();
?>