Sync not working with term references

Confirming and subscribing.

Here is my tentative patch, it fixes the issue for me.

Only tested against non i18n enabled vocabs and 'localized' vocabs.

Additionally, the function "i18n_sync_node_translation_taxonomy" should probably be removed as it is D6 related and dead code in the D7 version..

Taxonomy terms may have a translation so we cannot just copy them over.

sub

+1

Will #2 work properly with localized vocabularys?

Thinking this should be handled by i18n_taxonomy implementing some 'sync callback'.

So for now, removed that 'taxonomy_term' exception so fields are copied over. Patch committed, thanks.

Pending: implement that sync callback into i18n_taxonomy module.

Subscribing and confirming that patch (#2) is working for localized terms.

This should be working now for all kind of terms:

- Reworked field sync API to be more like fields API.
- Implemented hook_field_prepare_translation() for taxonomy fields (i18n_taxonomy)
- This function is reused to synchronize fields too.

YAY!
This is great news :-D Thanks Jose, for all your hard work.

Is it normal that 7.x-1.0-beta6 has the same bug? Some patch is working for that version too.

beta6 was generated May 24, all commits after that date is not included, and all the bugs fixed after that day is still present ;)

If your installation have problems related to issues fixed after that date, pls update to the dev. branch. Commits done is found at http://drupal.org/node/5917/commits. I'm sure Jose Reyero and the team will at some point roll a beta7, but if you are in hurry, or want to help out giving us feedback, use dev.

Thank you very much.

I'm using beta7, which includes this fix, but it's not working yet.

I have set the Tags vocabulary to 'Localize', and in Content type settings > Synchronize translations checked 'Tags'.
I have a term (which also has a translation), and when I add this to a node and save it, it says "One node translation has been synchronized." but it is not added to the translated node.

I'm having the same problem. A taxonomy reference field with a vocabulary set to 'Localize' will not sync.

The data for those fields is stored with LANGUAGE_NONE. But i18n_sync_field_translation_sync() tries to sync from "source langcode" to "destination langcode" and fails.

I don't know if it's the right approach, but the following patch fixes the issue for me.

It seems to work for me with both translated and localized vocabularies. Also my fields have a language code that matches the node language.

Are you using any other module that may change fields language, like entity translation?

Ok, this is a long story. So I will start with my conclusion:

A term reference field, created before i18n is enabled, will have 'translatable' = 0 and produce data with LANGUAGE_NONE and it will always remain like this even after enabling i18n.
Also, this will always be the case for 'field_tags' since it is preinstalled (at least with the standard profile)! At the current state, field_tags can never be synced unless it is modified in the database (didn't try it yet). Or with my patch which is probably not a good solution though.

For now the only solution (other than to modify the database) is to never use term reference fields created before i18n is enabled.

Im not sure what the best solution would be. Updating all field configs when enabling i18n? Being able to somehow update the 'translatable' attribute for fields?

I didn't check if other field types are affected as well.

End of conclusion.
_________________________________________________________

Now the long story (a protocol of my research):

Very strange. I did some tests and I'm not able to get a term field stored with a language code other then LANGUAGE_NONE (und) at all.

This is what I did:

• Fresh install of drupal-7.4
• (Install admin_menu)
• Add a term reference field to content type 'Basic page' ('field_tags', vocabulary 'Tags') .
• Add term 'Test' to vocabulary 'Tags'
• Create a 'Basic page' node with term 'Test'.

The term is stored as 'und' (field_data_field_tags).

• Install i18n (7.x-1.x cloned from git)
• Enable i18n
• Enable i18n_field
• Enable i18n_node
• Enable i18n_string
• Enable i18n_sync
• Enable i18n_translation
• Create a 'Basic page' node with term 'Test'.

The term is stored as 'und'.

• Enable Multilingual support with translation for 'Basic page'
• Create a 'Basic page' node (language English (from now on)) with term 'Test'

The term is stored as 'und' (the node as 'en').

• Enable i18n_taxonomy
• Create a 'Basic page' node with term 'Test'

The term is stored as 'und'

• Enable Tranlation mode 'Localize' for vocabulary 'Tags'
• Create a 'Basic page' node with term 'Test'

The term is stored as 'und'

• Enable Tranlation mode 'Translate' for vocabulary 'Tags'
• Create a 'Basic page' node with term 'Test'

The term is stored as 'und'

• Set language of term 'Test' to English.
• Create a 'Basic page' node with term 'Test'

The term is stored as 'und'

At this point everything is translatable (content type, vocabulary). Page and term have a language (English). Still it is stored as 'und' here...

Debugging shows that _field_invoke(...) (modules/field/field.attach.inc:172) sets the language to 'und'.

This is because field_available_languages(...) (modules/field/field.multilingual.inc:86) returns array(LANGUAGE_NONE) for field_tags.

There field_is_translatable(...) returns false.

And this is because $field['translatable'] is 0 (where $field['field_name'] = "field_tags").

Now I realize that 'translatable' in db table field_config for 'field_tags' is 0. Because when that field was created (at drupal installation since it is a preinstalled field) the vocabulary was not translatable.

Editing and saving the 'Tags' field and content type 'Basic Page' doesn't change anything.

Creating a new term reference field (field_tags2) with vocabulary 'Tags' does end up with 'translatable' = 1 in field_config.

Now I set Translation mode for vocabulary 'Tags' to 'Localize' and create a new term reference field. It ends up with 'translatable' = 1 in field_config.

Now I set Translation mode for vocabulary 'Tags' to 'No multilingual options' and create a new term reference field. It still ends up with 'translatable' = 1 in field_config.

Here I came to the conclusion at the beginning.

After manually setting field 'translatable' in table 'field_config' for 'field_name'='field_tags' to 1, everything works and synchronizes as expected. No matter what the translation settings for the vocabulary are, even with i18n_taxonomy disabled.

This makes my patch in #15 unnecessary.

@jox, thanks for such a detailed report (#17)

I think I am starting to understand what's going on here. This is also related to #1125290: Why if I disable multilingual support for a content type then the language is marked as the default instead of neutral?

Fields take the node language when you create a node. Nodes created before you enable locale take LANGUAGE_NONE as their language. But after you enable locale, nodes without multilingual support enabled take the default language (unless you enable that new i18n option in the other issue).

AFAIK no i18n module does anything to change available languages for fields so it must be either locale or translation module. Btw there's a ton of documentation about field languages on field.multilingual.inc

This creates some awful inconsistencies IMHO as fields languages seem to depend more on nodes/enabled modules history than on current status.

So how should we fix this? Should we update field definitions? Or should we check language for each field using field_language() before doing any update/sync operation ?

Maybe the second option is the best one as we don't touch anything on field storage, just do our business whatever it is.

Also I'm wondering whether we should file a bug for Drupal core about this.

@Jose, you're welcome, glad to help.

I think I am starting to understand what's going on here. This is also related to #1125290: Why if I disable multilingual support for a content type then the language is marked as the default instead of neutral?

Fields take the node language when you create a node. Nodes created before you enable locale take LANGUAGE_NONE as their language. But after you enable locale, nodes without multilingual support enabled take the default language (unless you enable that new i18n option in the other issue).

Interesting. I notized and wondered about that when I created the very first test node. It had LANGUAGE_NONE whereas all following nodes would have the default language 'en' (obviously after enabling locale). I was almost going to mention this.

I had already some concerns about the other issue also, which I'll better post there maybe.

AFAIK no i18n module does anything to change available languages for fields so it must be either locale or translation module. Btw there's a ton of documentation about field languages on field.multilingual.inc

Well, take a look at field_available_languages() (field.multilingual.inc:86). Available languages are depended on the field config's 'translatable' setting. And you're right, it's all documented there.

This creates some awful inconsistencies IMHO as fields languages seem to depend more on nodes/enabled modules history than on current status.

Unfortunately so true...

So how should we fix this? Should we update field definitions? Or should we check language for each field using field_language() before doing any update/sync operation ?

Maybe the second option is the best one as we don't touch anything on field storage, just do our business whatever it is.

I'm not sure if the second option will correctly handle the case where the field definition has 'translatable' = 0. Maybe field_language() finds the right language (possibly LANGUAGE_NONE?) but this still leaves potential for inconsistencies, because the data for different languages is still stored with the same key (LANGUAGE_NONE). Does that make sense? I'm confused.

After all I think the problem is that enabling i18n/locale produces inconsitencies with existing data (nodes, field data, field config, ...?). So updating field definitions looks like the cleaner solution. But then field data needs an update too.

More general: data storage follows different (incompatible?) conventions depending on the sites configuration.

Is there a clean way without modifying existing data when changing the configuration? Should all functionality consider all conventions?

This is tricky. I need some more time to think about that.

Two question came into my mind. It might be documented somewhere, but I'm asking it here quickly to save some time and keep the thoughts going.

1.) Is it valid, or by design, if a node has field data stored with a language other then the language of the node itself?

2.) Does the language of field data reflect the language of the field data or the the language of the node?

After some research, it seems latest Drupal core version has changed default field languages behaviour. I've reworked most of it to use field_language() and it seems to be working now. And added some tests.

Check out RC3

Thanx Jose, for all your hard work!

It's not working yet for me.

Following adjustments are made:

* Multi-language set sr (Serbian) and en (english)
* Created the vocabulary with the "Multilingual Options - Translate. Different terms will be allowed for each language and they can be translated".
* I added and translated terms. I paired them. In "Translation sets" they are paired, translated.
* In the "Content type" I added the "Manage Fields -> Field -> Term reference", adding the appropriate vocabulary
* In the "Manage display" I "Link (Localized)"
* In the "Publishing options -> Multilingual support -> Enabled, with translation"
* In the "Synchronize translations" in "Configurable fields" I have chosen the correct options for synchronization.

I confirm #26.
I have the same configuration as described by krug. The issue I'm experiencing is the following:
* I have Drupal 7.8 with Internalization 7.x-1.0, and 3 languages with default - English;
* I have vocabularies with Translation mode set to "Translate. Different terms will be allowed for each language and they can be translated. "
* All terms are translated and translation sets look just fine.
* When I edit a node with a term reference filed to the described vocabulary I can choose only English terms but when I save the node the synchronization is done correctly.
* When I display the node with a term reference filed to the described vocabulary the terms are showing properly in all different languages, BUT
* when I edit the same node I can see only the default (English) terms in the node's add/edit form when I'm editing the node's NON-English translation, and I can't choose any of the translated terms for that language.
* When I save this node form the term reference fields are reset (unset) and all other languages are synchronized with this new value of the term reference field including the original (english) resulting in loss of information.

I concur.. confirm #26,

I tried to hack together a solution for the time being by creating a hook_form_alter for 'page_node_form'. If the form had $form['#node']->translation_source->my_taxonomy_field['en'][0]['tid']. Then i tried to force the synchronization.

I was testing this by making an english page then translating to french.

The odd thing is when creating the french page if i tried to load an english taxonomy term (taxonomy_term_load($english_tid);) it results in null. But i can use the same function to load up any french term.

I need the original english term to call i18n_taxonomy_term_get_translation

This is not a solution to this problem. Just a way i wanted to manually force synchronization. But isnt it odd that i can't use taxonomy_term_load to get a term because the current language doesn't match?

For the last few posts on this thread, maybe try enabling "URL" on the following page: /admin/config/regional/language/configure. This solved a lot of my taxonomy translation issues.

i can confirm this issue and i have #29 setting

this issue take place for terms with "Translate. Different terms will be allowed for each language and they can be translated." setting

Actually the sync "works" but it saves the source reference terms, not the terms translations.
it also displays the right translated terms.

Maybe fixed by this one? #1312946: The first synchronized image is created with language='und'

more details about my case and a possible solution

- the issue takes place for the creation of node translation that has multiple term reference for a taxonomy with translation.
- i discovered that the issue in this case it was within i18n_taxonomy_field_prepare_translation function.
this function can't get the source terms translation the "taxonomy_term_load" for the source terms return nothing.
- what i did it was to switch the order of "functions"
first to get the translation id with "i18n_taxonomy_translation_term_tid($item['tid'], $langcode)"
and then
to get the term for the translation id, and it seems it works

present:

function i18n_taxonomy_field_prepare_translation($entity_type, $entity, $field, $instance, $langcode, &$items, $source_entity, $source_langcode) {
  foreach ($items as $index => $item) {
    $term = isset($item['taxonomy_term']) ? $item['taxonomy_term'] : taxonomy_term_load($item['tid']);
    if ($translation = i18n_taxonomy_term_get_translation($term, $langcode)) {
      $items[$index] = array(
        'taxonomy_term' => $translation,
        'tid' => $translation->tid
      );
    }
  }
}

changed to:

function i18n_taxonomy_field_prepare_translation($entity_type, $entity, $field, $instance, $langcode, &$items, $source_entity, $source_langcode) {
  foreach ($items as $index => $item) {
    if ($translation_tid = i18n_taxonomy_translation_term_tid($item['tid'], $langcode)) {
      $translation =  taxonomy_term_load($translation_tid);
      $items[$index] = array(
        'taxonomy_term' => $translation,
        'tid' => $translation_tid
      );
    }
  }
}

here it is the patch

change version, as the pacth is built against the dev

#33: Sync_not_working_with_term_reference-1159912-33.patch queued for re-testing.

I have juste appllied this patch in #35 on a bilingual site (en+fr), and as long as I am careful to switch the language in the admin area before editing my translated node, the translation synching is behaving as expected.

My content type is multilingual, referencing three vocabularies using the ' Translate' option, so that a term and its translations are on two different entities (TID).

This patch does the job for me.

Is it rational to expect that thie behaviour this patch provokes should be native for i18n?

Thanks a lot

> as long as I am careful to switch the language in the admin area before

I am afraid the patch is not good enough then, what happens if you don't switch the language? (Maybe it works also without the patch).

About the patch, it looks good though I think we should try to fix the underlying issue, why is why we are not getting the right terms in the first place? I guess this is related to query alter in i18n_select(), which maybe shouldn't be triggered for node loading.

Tip: Try excluding the path node/add/* from 'Multilingual settings/Selection'

What you are refering to in #38 looks a lot like what I describe here : http://drupal.org/node/1494584

It has to do with the "i18n select" using the main language of the page (from url), which in the admin is not always the same as the language of the node being edited.

In some cases, in can see why it does that ( a taxonomy menu with taxonomy menu module , needs to take the language from the page). In the case of editing a node in the admin, it seems like it should take the language from the node being edited.

One way to fix this is by adding:
node/*/edit
as an exception in admin/config/regional/i18n/select , but you then get the taxonomy terms from all languages clogging your selection list.

It seems to me this could be tackled in i18n_select.module , in the i18n_select_language() function.

with reference to #32 and corresponding patches , here is my take:

Here we have a problem because "i18n select" blocks access to taxonomy terms of other languages. Thus it returns nothing:

<?php
function i18n_taxonomy_field_prepare_translation($entity_type, $entity, $field, $instance, $langcode, &$items, $source_entity, $source_langcode) {
  .....
    $term = isset($item['taxonomy_term']) ? $item['taxonomy_term'] : taxonomy_term_load($item['tid']);
    if ($translation = i18n_taxonomy_term_get_translation($term, $langcode)) {
    .....
      );
    }
  }
}
?>

Here you avoid the problem ! wohoo.

<?php
function i18n_taxonomy_field_prepare_translation($entity_type, $entity, $field, $instance, $langcode, &$items, $source_entity, $source_langcode) {
  .......
    if ($translation_tid = i18n_taxonomy_translation_term_tid($item['tid'], $langcode)) {
      $translation =  taxonomy_term_load($translation_tid);
     .......
    }
  }
}
?>

#39 still valid tho.

This is pretty weird because both translation sets and synchronization are meant to disable i18n_select when loading translations. There may be some static caching issues with entity load though.

After doing some testing, what I've found it's not working is the field values list (taxonomy select drop down), when using interface with different languages. This is a simple patch that tries to fix that.

Still, from the reports I don't have a clear idea of when this is failing, some questions:
- What are we doing here: editing, creating a translation?
- Are you using other modules? Admin Language? Entity Translation?
- Which are node and interface language for each case?
- Field settings and widget used for the term reference field (As said above some edit pages are failing with some widgets).

No follow up, no one trying the patch...

Hi @Jose Reyero, I'm come here following this link: http://drupal.org/node/1494584#comment-5817618 .
I've reopened this issue because I'm testing your latest patch and it seems improve the situation :-)

I've done some testing using the following Vocabulary settings: Translate. Different terms will be allowed for each language and they can be translated. Moreover I'm not using translation sync on this tests.

First test without patch.

1. created a new EN node with the EN user interface and selected an EN term.
2. translated the EN node to IT (the UI is still EN): the node add/edit form displays EN terms.
3. switch to IT language end go back to IT node edit form: no terms are selected (and only IT terms are displayed).

Second test with patch.

1. created a new EN node with the EN user interface and selected an EN term.
2. translated the EN node to IT (the UI is still EN): the node add/edit form displays IT terms and the default one is the correct translated IT term.
3. switch to IT language end go back to IT node edit form: everything looks fine: IT terms are correctly displayed.

P.S: I've done this two tests even adding a second term reference inside a field collection, in this special case the field collection ignore the entity language and will always display the current user language instead of the node one, with or without the patch. I'm reporting results from field collection too in order to give more informations.

Test with i18n_sync enabled (and without the previous patches because are not applicable on latest i18n 7.x-1.7):

Unpatched i18n:

1. create a EN node (user interface EN) with a EN term referenced.
2. from the EN user interface create the IT node translation.
3. the term field on the add/edit form displays EN values and nothing is selected.
4. select the EN term
5. after saving the IT node switch the interface to IT and the node still displays the EN terms
6. from the IT interface edit the IT node: term reference fields are all emptied
7. change the term reference to the IT term and save
8. IT node term is correct (use the IT term) and EN node have the correct EN term too

Using the patch #41, when you translate the node for the first time, the correct IT term is selected so you don't have to edit and save it again.

The patch #41 is not bad but a big problem still remains: when you first create the node you should be using the same user interface of the node language.

Thanks for the attention.

Hello,

I reopen this post because I have the same problem.(https://www.drupal.org/node/2339897#comment-9154995)
I can't patch the module and must to use only stable versions of modules for my site. Is there a solution without the above patch ? This patch is it included in the last version of i18n ?

Thanks for your replies.