Note: This is now part of the master plan on #2313159: [meta] Make multilingual views work

Problem/Motivation

Views with row plugin "Fields" are inconsistent about the language that is used for display.

Steps to reproduce

a) Install with Standard install profile.

b) Turn on Content Translation module and its dependencies, like Language.

c) Add Spanish language (or whatever language you want)

d) Go to admin/config/regional/content-language

e) Turn on "Content" as having custom settings, and enable translation for Article content type. Make sure both Title and Body are checked as translatable.

f) Add an Article node in English, with title/body set to "Food in Paris".

g) Translate this into Spanish, as "Comida en Paris".

h) Add a new View of Article content, without filtering beyond published and content type.

i) Make the view use the Fields row plugin, and display the Title, Body, View link, and Edit link.

What should happen

If instead of using the Field row plugin, you use the Entity View Mode row plugin, things are fine. For instance, if you choose Teasers and "Translation Language" for the language option, you will see the English and Spanish translations of the title and body; and the title, edit, and translate links will be going to appropriate language-specific URLs. This is good. Screen shot from March:
teaser view
(Note: This looks slightly different now because there are some contextual links included, but the translations are still fine and link URLs are correct, as of Oct 17 2014.)

You can also choose "original language", which will display both rows in English, or "current language" (which is really "the selected/detected Content language"), which will also display both rows in English.

What does happen

One output row will show the node title, links, and body in English, which is fine. The other row of output will have the node title in Spanish but linked to the English URL; the node body in English; the links for edit and view going to the English URLs. Here's a screen shot showing the title and body at least:
field view
Also, if you are viewing your site in Spanish, the same thing happens -- you will still get the English body.

The current reason for this is that there is a setting under "Advanced" in the Views UI, for each Display, called "Field language". It gives you the usual language choices, and says it governs the language used for all fields. The default is "Current users language", which actually means the language chosen by the language switcher and/or language negotiation settings. This is why my Body fields are being shown in English. But the Title is displaying in Spanish, so they do not obey this setting.

And there is no way to specify, as you can for Entity View Mode row plugins, "Display each row in the translation that that row represents", or "Display each row in the original language for the entity".

Proposed resolution

Currently, we have a setting for Entity display language that is on the Entity row plugin, and a setting for Field display language that is at the display level and applies to the Fields row plugin and possibly to the Table style plugin (which doesn't use a row plugin).

These settings should be unified, and so they need to be on the Display plugin, because some need to apply to Row plugins (Field, Entity) and some to Style plugins (like Table). And anywhere entities are displayed, they need to be translated into this language first, and the links and base fields and regular fields and complete entities and whatever else needs to be displayed using that language.

And the language settings should give you options for:
- The detected/selected content language for the page the view is displayed on
- The translation language (language of the row)
- The original language of each entity in the row
- Specific languages (English, Spanish, etc. - whatever is on your site)
- Site default language

Currently, Field settings give you specific and site default language options, and Entity settings give you the other 3. Both should allow both.

However, see comment #44 - given how entity base fields are currently set up, it may not be all that possible to do this. We may instead need to leave the EntityRow plugin as it currently is, and make it so that Field API fields always use the row language like base entity fields do.

Remaining tasks

- Set up the unified display setting for language, with unified list of options.
- Remove existing settings.
- Make existing row/style plugins obey this setting.
- Tests

User interface changes

- New unified setting at the Display level for dsiplay language.
- Remove existing Field Language Display setting in Advanced and Entity display language setting on the Entity row plugin.
- Field, link, and entity view mode languages will behave consistently in views rendering, no matter which style/row plugins are being used.

API changes

Some views settings will move.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

jhodgdon’s picture

This is interesting. I was going to look at the query for the field-based view to see what was being queried, but it seems that the Body field being in the fields list has no effect on the main Views query. I guess it's being done differently in D8 than it would have been in D7. So the main query is just:

SELECT node_field_data.title AS node_field_data_title, node.nid AS nid, node_field_data.created AS node_field_data_created
FROM 
{node} node
INNER JOIN {node_field_data} node_field_data ON node.nid = node_field_data.nid
WHERE (( (node_field_data.status = '1') AND (node_field_data.type IN  ('article')) AND (node_field_data.title LIKE '%paris%' ESCAPE '\\') ))
ORDER BY node_field_data_created DESC
LIMIT 10 OFFSET 0

Actually, even when I turned on "show all the queries", there wasn't anything about the body field in the entire query list. It seems like the fields (aside from title) are being rendered from some cached data somewhere?

So this is most likely even more closely related to #2149649: Entity rendering/theming does not use the active entity language to render links than I first thought. I think when you have a Fields row plugin, the content fields need to be loaded for the translation. So this is very closely related to the links problem in the other issue (although with row plugin "Content" or whatever it is called to do teasers, the body is actually being loaded from the translation, as you can see from the screen shot I put in the original issue report).

jhodgdon’s picture

I also just filed a related issue #2218025: Field language filter/sort/etc. for Views do not work and are not needed about field:language subfields and filtering... anyone interested in this issue may be interested in that one too.

And we might want to think about how to render fields if there is a filter on a field:language -- which language to use?

Gábor Hojtsy’s picture

jhodgdon’s picture

The two issues aren't really related, except that they are both about translations, fields, and views.

Issue #2218025: Field language filter/sort/etc. for Views do not work and are not needed is about the fact that the Field:language filters are totally broken.

This issue is about the fact that if you have a view of translated nodes (which is by default a view of all the node translations), and you use Fields for the display, the field values are always displayed in English instead of the actual language of the translation. In contrast, the Titles shown are the correct translations, and also in contrast, if you display the Teaser or use another view mode (instead of using Fields for the display), that is translated correctly.

jhodgdon’s picture

FileSize
10.16 KB

OK, so this is interesting.

There is a setting under "Advanced" in the Views UI, for each Display, called "Field language". It gives you the usual language choices, and says it governs the language used for all fields.

The default is "Current users language", which actually means the language chosen by the language switcher and/or language negotiation settings. This is why my Body fields are being shown in English.

However:
- The titles are shown in the language of the node translation, no matter what this setting is.
- The body should also be shown in the language of the node translation. The rows in a node view are node translations. Each Body field value is associated with a particular translation. It makes zero sense to me that on the French or Spanish translation of a node, the French or Spanish title is being shown along with the Body field from a completely different translation. How can that possibly be correct?

Here's another screen shot. In this case, I have one node with "spanish" and "french" translations. The body is the same as the title in all translations. Here's the view output, with the default "Current users language" setting for "Field language":
Field language views output

This does not make sense... The setting needs to be removed, and it should be replaced with the ability to render the fields the same way as the title is being displayed: all fields should be associated and displayed in the View the same as the title: the version needs to be picked out from the language of the translation that is being displayed in that particular row of the output.

Gábor Hojtsy’s picture

@jhodgdon: totally agreed with that :)

jhodgdon’s picture

One more piece of ammunition in the "does not make sense" thread of #5: if you use Content / Teaser to display your body field, it's displayed in the language you'd expect it to be displayed in (the same as the title).

So:

a) Each row of the view corresponds to one language translation.

b) The title is always displayed translated to the language of the row.

c) Teasers (or content in other view modes) are always displayed translated to the language of the row.

d) Individual Field module fields are displayed according to the setting of the "Field language" setting under "Advanced".

e) If you use the "Link to the content" check box on the Title field, the link always goes to node/[nid], which is to say it takes you to the node in its default language or your current language or the site default language or something like that, not to a page that displays that translation of the node.

(d) and (e) feel wrong... (e) might be a different issue though.

jhodgdon’s picture

I did some digging.

There is apparently one place where the language for rendering the field is being decided: in

\Drupal\field\Plugin]views\field\Field::field_langcode()

It is currently deciding this based on that Advanced setting. It instead needs to decide it based on the language of the row in the view.

What I don't know:
- Are other entities the same as Nodes? In other words, if I make a view of another translatable entity type, is each row one translation?
- Can we reliably get the language code of the row from Views?

Actually, fixing this is probably related to #2218065: Need to join fields to the entity field data tables, not entity tables, or filtering increases number of results. The problem on that issue is that when you filter based on some property of a field, the join is being done on node not node_field_data, and it's ignoring language, so you get more results instead of fewer from filtering. Something similar is going on here, in that the language for a row is being ignored when rendering the field, because the field kind of doesn't know what language the row is. Each row (at least in node views) *does* have a particular language associated with it...

jhodgdon’s picture

Oh, finally located the issue that fixed this for entity rendering (e.g., teasers):
#2149649: Entity rendering/theming does not use the active entity language to render links
Oh wait, that was already marked as a related issue. That patch will probably answer some of the questions here.

jhodgdon’s picture

Another note: When you render the Title in a Fields display in Views, and check the "Link this to the node" box, the link does not go to the correct URL -- it always goes to node/1, not es/node/1 or whatever. This works correctly if you are using "Content" not "Fields" display -- the title there links to the right place.

Gábor Hojtsy’s picture

Catching up on this one I just found that advanced setting yesterday and was wondering the same things. It needs to go away and instead the row language should decide the field rendering. We should check that all views data integrations use the data tables now so they have language. Not sure what to do if the base entity does not have language (ie. a contrib entity type?).

jhodgdon’s picture

Issue summary: View changes

Updated summary for the current master plan on the meta-issue #2313159: [meta] Make multilingual views work that this is now a part of.

jhodgdon’s picture

Issue summary: View changes
plach’s picture

FileSize
182.52 KB

Agreed on the plan, just a remark: it wasn't clear to me from the issue summary if we want to extend the entity row renderers mechanism from the entity row plugin to the fields row plugin, but I think we should keep at very least the same UI, if re-using the code is not possible:

About the body field not being present in the query, we have the same logic in D7. See #2218025-27: Field language filter/sort/etc. for Views do not work and are not needed.

Gábor Hojtsy’s picture

@plach: I think we want these three options for fields as well, right? The view should be filtered by a language if we want fields displayed in that language or not filtered, if we want it to depend on interface language, default language, etc. Also I think this "Default language" is the language of the entity not the site, right? We should clean up some terminology on the way somewhere :D

jhodgdon’s picture

The row style options in #15 are what we now have if you use Entity-based row plugins, and I think we want something similar for Field-based row plugins... but with the choices listed in the issue summary. Also we want to replace this for entities to have the same options as Fields.

In other words, the rendering should be giving you the choice of (whether it's fields or entities):
- Specific languages (enabled languages on the site, plus "undefined" and "all languages")
- Current language (the language chosen by URL, language switcher, and other negotiation, for either content or UI I suppose?)
- Row language (the fields/etc. for each entity will be displayed in the language associated with the translation of that entity in the row)

Gábor Hojtsy’s picture

How does this interoperate with the language filter then? Can I filter for German entities and then still display all of them in Italian? Or I filter the view for French entities but always display them in the language of the page (whatever it was negotiated to)? I can see the flexibility there, so that rendering based on the row is just one of the options so there is a separation of rendering language and filtering language... OTOH I can see how people will be confused by some very possible weird combinations. Eg. they would set the rendering language to French and then wonder why they see 5 copies of the same node in French on the page (because they used no filter on the node). I think we should somehow balance the flexibility possible with the ways things can go wrong somehow.

jhodgdon’s picture

Yeah... I think if we make restrictions on it, it's not flexible enough for some crazy use case out there.... I just think we have to make the options very clear, which will clue people in that there *is* a language for the row, for instance.

Gábor Hojtsy’s picture

Issue tags: +Drupalaton 2014

Ok, we can try an see how people understand this...

plach’s picture

Yep, "Default language" is confusing, "Original language" would work better IMHO.

Edit: I think the current default (Translation language) provides a sensible behavior and should let novice users get what they need without struggling too much (or even noticing that setting). However the "Current language" renderer allows for language fallback, if combined with a filter on default_langcode = 1, which is definitely a nice feature. I think also a "Site language" (or "Admin language") renderer could make sense. Think of the node admin view: if I am a site admin and I understand only French, it would be nice if my nodes where displayed in French where available. OTOH this might mess with a "Title" exposed filter, as such a view should be also filtered on default_langcode = 1...

(just thinking loud :)

jhodgdon’s picture

I added #2317975: Update Views help to explain multilingual so that when we get this and a few more of the issues on the meta done, we can remember to update the Views help so this is at least all explained.

jhodgdon’s picture

OH, and regarding #22... We have to be careful thinking about filtering vs. display.

In our current plan (and soon to be reality), each row of a view has a language for each entity, because we're going to be joining on the field_data table for the entity. So any filters go against the fields *in that language*. That's a separate issue anyway...

jhodgdon’s picture

I added a note to the help issue about languages and filters too.

Gábor Hojtsy’s picture

Title: Field rendering does not obey language of nodes in view results » Field rendering does not obey language of nodes in view results, may lead to data loss in admin views

This causes lots of confusion on the node admin view where every instance of the node points to the same language, eg. on a site path prefixes for all langauges and a node translated in English and Hungarian:

Gábor Hojtsy’s picture

Title: Field rendering does not obey language of nodes in view results, may lead to data loss in admin views » Field rendering does not obey language of nodes in view results
jhodgdon’s picture

Yes. This is a near-critical problem and definitely should be fixed soon. Too bad it's not exactly easy. :)

Gábor Hojtsy’s picture

I believe we want to tackle this once #2037979: "Current user's language" views filter label is named very misleading lands, so we can merge that with the existing entity row styles and move the options to row styles for fields as well.

Gábor Hojtsy’s picture

Status: Postponed » Active
Issue tags: +sprint
jhodgdon’s picture

Assigned: Unassigned » jhodgdon

It seems like I should do this... If anyone else prefers to take it on, please feel free to reassign! I probably can make time for it later this week, so speak up now! Please! :)

dawehner’s picture

#2322949: Implement generic entity link view field handlers is kinda a blocker because we should solve this for all our links, which currently float around in node, user and comment.

jhodgdon’s picture

It's not just links, but all of the field rendering in general that is a problem. If that other issue is solving it for links... will it also solve it for everything else?

And... I'm not sure that other issue solves the language problems really? It looks like its purpose is unifying how the links are generated (an issue summary over there would help!). Bbut the problem here is that fields are still trying to obey this stupid field language setting instead of obeying the language of the entities themselves in the view. Will that other issue resolve that problem?

Gábor Hojtsy’s picture

Yeah #15 has a visual demonstration of what this issue is about. Here is another one :)

Gábor Hojtsy’s picture

@jhodgdon said she will not be able to work on this soon. So here is a starter patch that removes the UI, the tests of the UI and the shipped pieces of config / corresponding schema. Will probably fail on some thing. Keeping assigned to @jhogdon as I don't believe I will be able to actually make this work myself. Removing code is easy :)

Gábor Hojtsy’s picture

Also opened #2357995: De-obfuscate entity row language rendering settings in views to clean up the UX of the row based language selection on entities so it makes more sense :) Feedback welcome there too.

Gábor Hojtsy’s picture

Status: Active » Needs review
jhodgdon’s picture

Issue summary: View changes
Status: Needs review » Needs work

It should be fairly scary to everyone that this patch that simply removed all mention/use of this setting didn't break anything. Only one test was edited -- the test that verified the UI for that settings.

Anyway... I applied the patch in #34, and retested this issue, since it was originally filed back in March. The current status:
- If you are using an entity display row display, life is good. Translation is there, and the links are going to the right places (it displays a read more link and an edit link by default).
- If you are using fields display, and you have Title, Body, Edit link, and View link fields added to the view, the Title si translated, the Body is not, and the View/Edit links go to the English page not Spanish.

Also I think Gabor just set this to Needs Review so the tests would run; they (amazingly) passed, but this is obviously not a real viable patch.

Updating summary...

jhodgdon’s picture

Oh, and I'm going to work on this. :)

jhodgdon’s picture

Title: Field rendering does not obey language of nodes in view results » Fields row plugin: Translation is non-uniform for base fields, Field UI fields, links; no way to choose "this row's language"
Issue summary: View changes

So. I did a bit more thinking on this, with YesCT and dawehner.

Here's the problem. If I solve this for Field rows, it is still not solved for Tables, which do not use a row plugin. And since we will also need to solve it for Tables, if we go this route we would have some language-display settings on the Row handlers and some on the Style plugins. Ugh! What a nightmare.

So... What YesCT and dawehner and I (I believe) agree on is that the only way to make sense of this is to move the setting onto the Display, which is where the Field Display Language setting is now, and use that one unified setting for Entity Rows, Field Rows, Tables, and anything else that is grabbing entities and rendering them in some language.

I'm going to update the issue title and summary... Titling the bug to reflect what the real bug is; updating the summary to describe what needs to be done.

plach’s picture

Looks like a sensible approach to me :)

Gábor Hojtsy’s picture

@jhogdon: indeed, spot on! Agreed. Looks like tables are the only rowpluginless ones in core but contrib may have any...

Gábor Hojtsy’s picture

BTW also read the updated issue summary and agree with it. :) Thanks for doing that. So looks like we may "just" rename the field language setting after all and make it have some new options and apply it for entity rows too?

jhodgdon’s picture

Yes. "just"... Well, a few more things: also remove the current entity rows setting and use this one instead. And maybe put the setting in a different place in the UI.

jhodgdon’s picture

I looked into this a bit more today, and I'm really not sure this is possible to fix, given the current architecture of Views. Here is what is happening at render time:

Entity view mode rows (row plugin \Drupal\views\Plugin\row\EntityRow being used)

In this case (which is working fine currently), there is a setting on the row plugin called 'rendering_language'. The raw values of this setting are translation_language_renderer, current_language_renderer, and default_language_renderer. These correspond to classes like \Drupal\views\Entity\Render\TranslationLanguageRenderer (just camelize the raw value name), which actually do the rendering. The way these classes work is that they load and view the entity using either the current row's language (translation), the selected/negotiated language (current), or the entity's base/default language (default), and whatever display mode (teaser, full, etc.) option is chosen on the row plugin.

It would not be a problem, I think, if we moved this setting to the Display instead of it being directly in the Row plugin. We'd also need to change the option machine values I think, and add some more language options... which wouldn't be a big deal, we'd just need to add a new \Drupal\views\Entity\Render\StaticLanguageRenderer class and update the logic in EntityRow::getRenderer() to use that class if one of the static language options was chosen, and probably instead of directly camelizing the machine name of the option, we'd have a switch statement. Should work OK. So far, so good.

Field rows (row plugin \Drupal\views\Plugin\row\Fields being used)

The way the Fields row plugin works is that it just asks each of the fields that is defined in the display to render itself. There are two cases of interest here:

Fields that come from the entity views table (like Title)

These are mostly using essentially \Drupal\views\Plugin\field\FieldPluginBase::render() for rendering, which just does:

  public function render(ResultRow $values) {
    $value = $this->getValue($values);
    return $this->sanitizeValue($value);
  }

So basically, it just says "Get the value from the query result and sanitize". In some cases there is some rewriting happening, but that's basically it.

Which means that it is always going to be getting the value from the query result row. Since the query result rows correspond to translations, this means we're getting the particular translation's value from the entity field data table.

Which as far as I can tell means that we are **always** getting the value that corresponds to the row's language, and since this is generic Views table/field logic, we don't know anything about entities or translations, and there is really not any obvious way to use a different display language. So I do not think we can have the base fields of entities obey a setting for display language at all -- they really do have to be using the language of that row, because that's all we have.

Field API fields

These use \Drupal\field\Plugin\views\field\Field as their field plugin. The logic here is basically that in Field::process_entity(), the first thing it does is call Field::field_langcode() to figure out what language it should be displaying in, and then it calls $entity->getTranslation($langcode). Then it reads field values from that translation.

The logic in the field_langcode() method is that it gets the 'field_langcode' option from the display (that's the Advanced section setting we've been talking about) -- this is either a specific language like English or Spanish, or a code saying "site default language" or "negotiated/selected content language", but in either case it is transformed into a particular static language code based on the page it's being displayed on. Then the entity is given a chance to use a different language (for instance if French is requested but there is no French translation of this particular node). The resulting language is the one used to load the entity translation.

So... This logic could be modified to (a) use a different option name on the display and (b) probably to have options like "the language of this row" like we have on the EntityRow view mode. So that is probably OK.

Conclusion...

I'm not sure what to do about the base entity fields. It doesn't seem like they can be easily translated, since they're not really associated with an entity, the way Views has them set up -- they're just database table fields. So unless we can make them on par with Field API fields, I don't think we can do this.

Thoughts?

jhodgdon’s picture

Issue summary: View changes

Actually, my conclusion from #44 is that we should either:

a) Figure out how to make the entity base fields use the Field API field plugin instead of the base Views plugins.
or
b) Take away the ability for the Field API fields to be displayed in other languages, and make them always use the row language, like the base entity fields do.

Either way, we'd at least be consistent.

Gábor Hojtsy’s picture

Could the base field renderer look at the display settings and unless it says "language of row", delegate it to the more resource consuming field API rendering? Not sure why base field rendering is different but I think it may be a leftover from the good old days when base field did not have widgets, translations, etc. so behave pretty much like any other field. It sounds to me like it would be dangerous to remove the field language rendering feature altogether just so the different Views rendering solutions would work the same...

Given the extent of changes required, maybe we should do this in steps? First move the setting to the global level and extent to static languages. So generalize entity views row handling (#44/1). Then unify field rendering and use the new setting for language selection?

I'll also try to ping this to the views and fields crews if they have opinions.

Gábor Hojtsy’s picture

Discussed with @dawehner on IRC:

[10:04am] dawehner: GaborHojtsy: so yeah noone worked on converting the old d7 thinking of properties and fieldapi fields to something which is inside the d8 world
[10:04am] dawehner: GaborHojtsy: that one https://www.drupal.org/node/2342045
[10:04am] Druplicon: https://www.drupal.org/node/2342045 => Formatter cannot be selected in views for base fields #2342045: Standard views base fields need to use same rendering as Field UI fields, for formatting, access checking, and translation consistency => 3 comments, 2 IRC mentions
[10:06am] dawehner: GaborHojtsy: its certainly good explained!
[10:08am] dawehner: GaborHojtsy: so afaik first step would be to add a way to render base fields using formatters
[10:17am] GaborHojtsy: dawehner: superb, thanks
[10:18am] dawehner: GaborHojtsy: i love your sarcastic humor
[10:21am] GaborHojtsy: dawehner: knowing what next steps are is always WAY better even if the next step is jeeeeez :D
[10:22am] GaborHojtsy: dawehner: can I post your notes from here to the issue?
[10:22am] dawehner: GaborHojtsy: sure!
[10:23am] dawehner: GaborHojtsy: but yeah I think its good that maybe there will be views people meeting at badcamp next week

So looks like the actual field rendering part of this issue needs to be postponed on #2342045: Standard views base fields need to use same rendering as Field UI fields, for formatting, access checking, and translation consistency. However as I said in #46, we can proceed with moving the setting to the display and adding static options and keeping the entity renderer work with those. That will be a sizeable task in itself :) Should we meta-ify this one and open a subissue or just postpone this on that new issue to be opened and #2342045: Standard views base fields need to use same rendering as Field UI fields, for formatting, access checking, and translation consistency?

jhodgdon’s picture

OK. I'm going to go ahead and work on the rest of this issue: the part about unifying all the language settings on the display plugin, and combining the options so you can pick a static language, "site default", content or UI language, translation language, or original language, for both Field UI fields and Entity View Modes.

We'll need that part in any case. Hopefully #2342045: Standard views base fields need to use same rendering as Field UI fields, for formatting, access checking, and translation consistency will get some love in the meantime... I have no idea if it's even possible to do that one, or how.

jhodgdon’s picture

jhodgdon’s picture

Gábor Hojtsy’s picture

Issue tags: -sprint

Should bring back on the sprint when it makes sense. No reason to keep a postponed issue for months on the sprint.

Gábor Hojtsy’s picture

Priority: Major » Critical