Node reference: results returned for interface language rather than node language

There are some use cases for the opposite behavior. So ok for an aditional option, no for always switching.

See #315638: where is "Interface language is independent" option

> f this is not done, the translated element is totally messy
About 'messy', please define and open specific issues

...it is only logical that the current language is the same than the one wish is selected for the content created...

This is not logical at all. When living in a multilingual country, you want the interface consistently in the same language (typically your native language). It should not change just because you're translating a piece of content.

I'm also having the same issue. Like jvieille, this is causing problems for me when editing content types which have link to other nodes, since I can only select nodes for the current language, which is not the same as the content language when I create a new translation.
This prevents our clients from editing some content types easily.
For one content type, I had to create a view that list nodes for all languages, so the user could select the correct node instead of only having the wrong language nodes show up.

An option that would make "add translation" and the links to switch translations also switch the current language would be ideal for me. In fact, for my needs the current language should always be the same as the content language, when the two are different it causes too much problems, and it is confusing to the user.

Changing the title to reflect the problem instead of a possible solution.

I agree that this is critical.

Steps to reproduce (in beta6):

1. Select current language 1
2. Create a node in language 1
3. The node reference selector will show nodes in language 1; select a node and hit Save
4. Click the Translate tab
5. Choose Add translation for language 2
6. The situation is: current language 1, node language 2 (greyed out), node reference selector shows nodes in language 2
7. Select a node in language 2 and hit Save
8. Error "This post can't be referenced"; and now suddenly the node reference selector shows nodes in language 1
9. Selecting a node in language 1 will not result in the error, but will be wrong nonetheless

Please note that it's also possible in steps 2 and 3 to create a node in language 2 and select a reference to a node in language 1. Which is equally wrong. I guess this is what jvieille was originally referring to.

The real problem is that the node reference selector is based on the current language, where it should be based on the node language.

I found this related issue, which has a patch for D5: #242601: translate node references with i18nsync

If you use language path prefixes a work-around is to add the path prefix by hand in the URL after step 5.

i can confirm this behaviour. I also get the error:

An illegal choice has been detected. Please contact the site administrator.

note that it has nothing to do with the sync module, as this error also occurs, when there is only the i18n module enabled.

This issue appears to result from the "content selection mode" implementation.

The query rewriting that i18n does to allow filtering of what nodes a user sees affects all node queries--not just ones that are for display purposes.

My general view is that all but the most simple sites are best off without "content selection mode" functionality. There are better, less problematic ways of filtering content by language.

One possible approach would be to limit the query rewriting by path--e.g., if we are at node/x/edit, don't apply the language filtering. But that could again produce unexpected results.

The quick fix is: set content selection mode to "All content. No language conditions apply." Then, when you need lists of content that are only in the current language, use Views with a language filter.

I agree with the jvieille. For me this is a bug. I understand that someone might want to keep the way things are being done (editing in one language while translating to another) but this only truly works for simple content.

The solution for this is either addressing this issue as a patch (while keeping the ones happy with the present situation) or making this possible as an option in administration (could be in the "Multilingual system" screen).

Right now the only issues I am having with i18n module is this one and another I started here (take a look and see if you agree with me)
http://drupal.org/node/345238

Besides these two very specific (but really important!) issues, everything is working great with this absolutely indispensable module for multilingual websites.

Re #13:

The query rewriting that i18n does to allow filtering of what nodes a user sees affects all node queries--not just ones that are for display purposes.

If so then I don't understand what I describe in step 6 in #11: first nodes in language 2 are shown, then in language 1. All the while the "current language" stays the same.

Just rewording and recategorizing.

Hello Jose,

I'm a bit puzzled. In #11 I reported an error, which I thought to be part of this issue. To me this is clearly a bug. Am I wrong? Or are we talking about two different issues and should I open a new one?

I just want to add that the only way to solve this issue is to change the language prefix manually at the URL to match the translation language.

If you change the language in the language selection block while editing the translation, the translation information will be lost and you will find yourself in a plain create node page (why?).

This limitation makes it virtually impossible for regular contributors to make translations. Therefore, I insist that for my case the only possible way to solve this issue is to automatically change the language to match the translation language.

I'm looking into this.

There is already code in place that is supposed to ensure that the language used for query rewriting is that of the node (when editing an existing node) or of the new translation (when creating a translation).

None of this affects nodereference, however, because the rewriting is happening via an AJAX request at a path that i18n doesn't recognize as related to node editing.

We could of course hack in something for node reference, but that's not a very satisfying approach.

Really, this is part of a general issue--how to recognize that an AJAX request relates to a particular editing context?

I'll review the nodereference code and see if there are openings there.

Clarifying title.

This is indeed a bug. When query filtering is in place, the expected behavior when nodes are being edited is that the node's language is considered in editing. This works on the original edit page, but not via AJAX.

Proposed solution: Add a js file that dynamically rewrites URLs in node forms to use the node's language prefix rather than the interface language one.

This turned out to be trickier than I thought.

I tried registering a #process callback for textfield elements with hook_elements() and changing the autocomplete path there (adding a prefix for the node's language). But that didn't work because theme_textfield() calls menu_valid_path(), and a manually prefixed path returns false. Meaning we don't have a way to alter the path rendered for the autocomplete.

The alternative is to do so via Javascript, which is what the attached patch does.

On hook_init() we test the path and send data to the browser on the base paths for both the current interface language and the node's language.

Then, in the JS, we substitute the node language path for the interface one. Result: autocomplete inputs pass a language prefix.

On my basic testing this seems to work.

Issues:

1. This doesn't work for new node creation, e.g., user creates node, sets language as 'fr'. Autocomplete won't fetch French content. Fixing this would be hard because the autocomplete path is read in on load, so dynamically changing the path won't make a difference. I'm inclined to leave out this use case.

2. Should we be testing the language negotiation setting and only doing this for certain values?

Sorry, could you explain better what your patch is supposed to do.

After applying it I went to a node that was originally in Portuguese and chose to add a translation in English but the problem persisted. The language chosen while creating the English translation was Portuguese and after saving the content the problem regarding node reference didn't go away (error message with - this post can't be referenced).

This patch works for me - created a node in one language, went to translate it, and nodereference autocomplete works as expected (and not without the patch). Since it only works with some kind of path negotiation, it might make sense to restrict it to those cases. But very nice work!

@slider_13: I got the same error. Tracked this down to this: in i18n_selection_mode(), the 'translation' selection mode wasn't set if data had been posted. This produced a mismatch between the data in the original form and that when the form had been submitted.

Removing this condition fixed the issue.

But I don't know why the empty($_POST) test was there in the first place, or whether removing it might produce errors elsewhere.

This patch version also includes a bit of minor cleanup from the prior one, fixing an error in setting the language for the content language url.

Last patch was missing the .js file.

The patch looks good and the proposed approach looks like a great solution for this overly complex issue.

Also I like having a js file that we may use later for some more features like switching stuff if changing the node language, etc..

Just one issue to RTBC. Can't we move the 'do-we-nedd-the-js-file' logic out of the init hook, maybe into the form_alter hook so it doesn't need to run always?

@nedjo, please feel free to commit this one (also if you think there's a reason to keep that logic into the init hook)

We need to be sure that this file loads prior to autocomplete.js, since its behavior needs to preceed autocomplete so that it can alter the autocomplete path before autocomplete inputs are processed.

hook_form_alter() IIRC is not called for cached forms, so we could get a form rendered without hook_form_alter() called.

Maybe a better approach then would be to implement hook_elements() for textfields and put the code in a #process callback instead. I'll give that a try.

New patch using hook_elements(). On my basic testing this seems to work. More testing would be great though. jvieille?

@jvieille: Take a look at http://drupal.org/patch/apply for seeing how to apply a patch.

@nedjo: I still can't manage to fix the problem. Same as before, when I hit translate, the language doesn't change and I get the same 'this post can't be referenced' error. I have checked both i18n.js and i18n.module and they have both been applied the patch.

(I was slider_13, but I changed my username)

I don't know whether this is important, but for the language negotiation I am using 'Path prefix with language fallback'.

@tiago.gmarques: thanks. I haven't been able to reproduce this error. Could you give precise steps to reproduce, from a clean install?

Rerolling the patch. Only difference is I've added a test for 'language_negotiation' setting (skip if LANGUAGE_NEGOTIATION_NONE).

Clean install and still nothing!

Here are the steps, one by one (using Drupal 6.8 and i18n 6.x-1.0-beta6):

1 - Clean install;
2 - Copy translation files for Portuguese and French;
3 - Copy i18n and install patch (last one);
4 - Activate Locale, Content Translation, Internationalization, Content type translation, String translation, Synchronize translations, Taxonomy translation;
5 - Site Configuration > Language: Configure - Language negotiation: Path prefix with language fallback;
6 - Add language Portuguese and French (imported correctly, tested the administration pages with different languages);
7 - English still as default language and set 'en' as the English prefix;
8 - Site Configuration > Multilingual system: Content selection mode: Current language and language neutral;
9 - Block, language switcher to right sidebar (just to help);
10 - Create new content type 'Test Content Type' with Multilingual support Enabled, with translation;
11 - Created content in the three different languages, every time I go to a content in the selected language, and choose translate and then a different language, the language does not switch automatically as it is supposed to with the patch. See the following example:
11.a) Created a content called Another test in English with Language set in English.
11.b) View the content at http://localhost/sandbox/en/node/2
11.c) Click on Translate (http://localhost/sandbox/en/node/2/translate) and then Portuguese add translation.
11.d) Goes to http://localhost/sandbox/en/node/add/test-content-type?translation=2&lan..., which means that the language did not switch (creating content in Portuguese while the set language in English as can be seen in the prefix)

Please, help me. What did I do wrong in these steps? If I understood correctly the patch is supposed to switch the language automatically while translating a content to a certain language.

Thanks!
Tiago

@tiago.gmarques:

Thanks for the detailed response.

The patch does *not* change the interface language when creating a node.

Instead, it changes the path used by autocomplete fields to include a prefix for the language the node is being translated to.

To see exactly what it does:

1. On a clean install, working in English, set up 'en' and 'fr' and make node types 'page' and 'story' translatable.

2. Add a nodereference field to 'page' that can reference 'story' content.

3. Select 'path prefix with language fallback' language determination.

4. Create a 'story' in 'en' and translate it to 'fr', making sure that the fr title is different than the en one (so you'll recognize it).

5. Create a 'page' in 'en'.

6. Translate the page to 'fr'. Note that the autocomplete only shows the 'en' story.

7. Apply the patch. What you should get is that the autocomplete shows the 'fr' story.

Ok! I have seen what the patch is for.

I hadn't notice any change before, since I was translating content with the nodereference field already filled, which was translated automatically (this in spite of the autocomplete not working). This way I never got the chance of filling the autocomplete field in the translation and did not realise it was not working properly.

However, this patch does not solve the error while submitting the translation. Ok, I get the widget to work but when I press Save in the bottom of the page the same error appears.

In the case of the clean installation, and following your steps, I get 'story: this post can't be referenced.'

Is it just me?

Oh... And sorry for the confusion I made with the purpose of the patch. I read rewriting the language prefix in the URL somewhere and I thought that the patch changed the language prefix in the URL of the page, thus changing the interface language to match the translation. As I have mentioned before, this is the only way I found to being able to submit the node without any error, since the language switcher doesn't also work... (see comment #21)

@nedjo: Just to clarify a thing in my previous post. Even before the patch, in spite of the autocomplete not fetching the nodes of the correct language, if the node you are translating has already the node-reference field filled, you get the node in the correct language.

This way your step 6, it is only partially correct. The autocomplete does show the 'fr' story if in the 'en' page you had referenced the 'en' story.

But the main problem persists. When I try to submit the translation I still get the error 'this post can't be referenced'.

Looks like we are making some progress but are not yet there, all this paths depending on the language negotiation mode may not work always.

So, I'm thinking, to fix this one, and maybe other related issues, once and forever, what if we add some query string parameters to force rewriting mode and language for queries? Then we can just add these parameters to the autocomplete path...

Something like: /anyautocompletepath?i18nmode=strict&i18nlang=en

@nedjo, if you think this may help, I'll be adding this feature so we can use it later on this patch, do you think it's a good idea?

There are other autocomplete paths, like the ones for taxonomy tags that may make good use of this too.

@Jose:

Thanks for looking at this.

From the reported error, it seems that the path we're passing to the autocomplete callback is working fine. We get the correct filtering, by the language of the node.

The error occurs on form submission (at the original path). When the form is submitted, the node that was autoreferenced is not available.

I too got this error with an earlier version of the patch. That's why I added this to the patch:

-        elseif (arg(1) == 'add' && !empty($_GET['translation']) && !empty($_GET['language']) && empty($_POST)) {
+        elseif (arg(1) == 'add' && !empty($_GET['translation']) && !empty($_GET['language'])) {

At one point an earlier version of this patch was accidentally included in a commit. It was later mostly rolled back, but this change was not rolled back, and so it isn't in the latest patch.

tiago.gmarques: can you confirm which version of the above line you have? If you have the one with empty($_POST), that would be thre reason for the error you're getting.

@nedjo

I did have first line with empty($_POST). I checked all five the patches that you submitted and only the second (#30) and third (#32) have those lines included.

I tried again with the patch on comment #32 and it works great (thank you!). Which is the best version of the patch?

@tiago.gmarques: Thanks. That makes this ready to commit.

Fixing version.

Ok, I'm sorry to bring in my confusion to this thread, I will try to tell what is happening:

(The listing of the images in the comment is the opposite of what they were before posting; but use the names to correspond with the numbers in the text.)

I am browsing for untranslated content in English (see image 1, address bar), and press "add translation" for the English language. I will then get a new page with the address bar url as presented in image 2. If I continue translating at this point and save, I will end up with corrupted data.

However, if I change the url in the address bar to what you see in image 3 (changing from language prefix 'en' to language prefix 'nb', everything will work fine from that point onwards. I've confirmed this modus operandi several times while translating today. I agree with #1, this is a very problematic issue.

The problem, as I could see from the database is that the i18n_node table isn't updated properly, and I'm not sure about what is supposed to go into the tnid in the node table. Personally I just set the trid and tnid to the same for all entries, and just had different nids in i18n_node together with the trid. It worked (somewhat) to get the nodes working together; but it looses the pointer to what is the original and what's the translation.

I can't look into the code myself at the moment, because I have a few hundred nodes to translate and a site which needs to get live by this weekend.

I hope the screenshots is of help for anyone, and that this is the same issue as I experience.

Never mind my #51, after I continued translating a few more nodes doing it the "right" way, it still creates some orphaned translated nodes which I have to go into the tables to manually connect.

Or rather, if I don't edit the URL I will always get it wrong. Perhaps I forget to reload the page after changing the url a few times?

@pkej: Please open a new issue, referring to this one, if the issues persist. This issue is now about nodereference. We can't solve everything in one place.

@jvieille, #49: These issues would require separate solutions. Please open new issues if you wish. I noted above why changing the autocompletes on the fly is difficult.

My personal view remains, I would never use the i18n content selection mode query filtering on a site. I think it's way too heavy handed and error prone. Instead, as I said in #13, I recommend Views filtered by language. Use that, turn off i18n's content filtering, and these problems will disappear.

OK @nedjo. That's were I couldn't quite relate to the discussion.

I can confirm that node references are working perfectly in the dev build from yesterday. I always get the right copyright node, regardless of interface language, as long as ?translation=349&language=en is present, which it is as long as I use the translation interface.

I applied the patch.

I am with jvieille. With this I don't mean to throw away all the work done by nedjo (you have solved one issue that was making impossible for content of my site being translated by anyone else than me). However, in my case (and I believe in many more) it is just more simple that when the translation is being created, the interface language is switched to match it. This way all nodes, taxonomies and so on, are kept within the same language and no filtering needs to be done. I know this solution does not apply to everyone and I think I know why.

For sites that a lot of people are editing content and without any kind of knowledge of the platform (wikis), the best solution is to keep things simple and in this case is matching the languages. Why? Because this way the user knows, for sure, in which language he/she is working. There is no ambiguity here. And also, a user might not understand perfectly the language of the source translation and when chooses to add a translation for his/hers own language is expecting the interface language to change as well (you might be asking why a person who does not understand a language is translating a node from that language, but don't forget that translating nodes is different that translating content. The content might be different but refer to the same subject).

However, for sites with less people people as editors, their level of the understanding of the platform might be greater and therefore the matching of the content language and the interface language does not need to be implemented. So in this case a simplification is obtained by separating interface language and node language so that if users want to translate content but keeping their language they can. However, for this to work correctly one issue needs to be solved. When editing a node and changing the interface language (at the language switcher block), the information already filled is lost; this has harmful consequences in node translations - if a translation is being created and the language is switched the user is presented with a plain 'create node' page.

So, what I suggest for solving this issues is:
1 - create an option in the administration screens, that allows to lock the interface language to the node editing language if set. This can be enabled for only certain users roles.
2 - solve the language switcher issue, so that if admins want the other way around they can allow their editors to choose the interface language while editing content independently of the content language.

I know that this is not the correct place but is just the continuation of the discussion gone here. I have created a new issue for this in http://drupal.org/node/361949.

This thread is departing from the original issue that was fixed with nedjo's patch. Also the option to switch language when translating has been addressed.

I'm closing it. Please post other remaining questions as different issues.

None of those issues are in relation to node reference, and quite a few of them look unrelated to the i18n module, please open a new issue at http://drupal.org/node/add/project-issue/i18n with detailed steps to reproduce.

Double post.

Sorry for re-opening this, but I'm pretty sure this bug has slipped in again.

I'm currently using i18n 6.x-1.7 & CCK 6.x-2.8, and am experiencing the exact issue mentioned above. I see that the proposed patch that fixes this issue is applied, but it doesn't seem to solve it (anymore?). Anyone else experiencing this?

Tried it on a clean install - same thing. Select/checkboxes work fine, autocomplete only returns content in the interface language, not the node language.

I'm using cck 6.x-2.8 and i18n 6.x-1.7, and this is not fixed.

My situation:
admin/settings/language/i18n
"Content selection mode": "Current language and language neutral". Why? This was not my decision, but I imagine it was for a good reason.
"Content translation links": I did not check "Switch interface for translating". Why? Because I don't speak Hebrew, and want to be able to visit the node edit form for Hebrew content with an English interface.

Content types "child" and "parent".
"parent" is multilanguage with translations, "child" has no language options.
"child" has a nodereference to "parent".

Visiting hebrew/node/123/edit (node type "child"), I see only Hebrew nodes in nodereference dropdown. I choose one and save. Fine.
Visiting english/node/123/edit, I see only English nodes in the nodereference dropdown. I only came here to change a bit of text, and save. The value for the nodereference is unset. Bad, bad, bad.

It is quite obvious why the above happens, but the behavior is definitely problematic, seeing that it can damage existing content.

I think this is rather a nodereference issue than an i18n issue. Nodereference should be more careful with db_rewrite_sql(). Should we continue the discussion in a cck issue? Which one?

EDIT: This one (nodereference issue):
#671406-7: NodeReference and multilanguage contents / db_rewrite_sql()

Cleaning up the issue tracker. Closing all issues that haven't got any follow up for the last year (and are not on RTBC state). Feel free to reopen if you're willing to (really) work on this.

@jvieille
Thanks for the update.

It seems cleaning up the issue tracker produces unexpected benefits :-)

Merry Xmas