Problem/Motivation
Originally reported by @eiriksm at #2457551-23: Regression: optional default configuration is not translatable anymore in locale.
When installing Drupal in a foreign language, several English config source strings are saved as custom interface translations to locale and make it impossible to get overwritten from community translations. When installing in Hungarian for example, the following English source strings end up in locale as custom translations from LocaleConfigSubscriber::saveCustomizedTranslation:
Recent comments
Master
No comments available.
Block
Files
Filter
Reset
Sort by
Items per page
- All -
Offset
Name
Size
Status
Used in
Filename
Entity
Entity type
Use count
All
Adding languages
Language code
Language name
Language
Languages
Translation
User interface translation
Translation language
Search
Translate
Content
Apply
Page
Find and manage content.
Title
Author
Updated
Operations
Published status
Published
Type
No content available.
Recent content
Edit
Delete
More
Feed
Last update
Attachment
Taxonomy term
Users
People
Bulk update
Username
Roles
Member for
Last access
Role
Permission
Active
Blocked
No people available.
more
List
Manage user accounts, roles, and permissions.
User
Output format
Fields
Preview
Proposed resolution
Do not save new translation if its the same as source string. This situation arose now that we store translations in active config, so the values in active config may be the source or the translation and we need to tell those apart. We should only save the same string as the source as a translation if it was changed back.
Remaining tasks
Review. Commit.
User interface changes
None.
API changes
None.
Comment | File | Size | Author |
---|---|---|---|
#22 | interdiff-17-22.txt | 394 bytes | eiriksm |
#22 | english_config_source-2468767-22.patch | 6.88 KB | eiriksm |
#18 | 2468767-18-test-only.patch | 3.31 KB | eiriksm |
#17 | 2468767-test-only.patch | 2.92 KB | eiriksm |
#17 | 2468767-17.patch | 6.5 KB | eiriksm |
Comments
Comment #1
Gábor HojtsyMy fist thought was this would be due to non-translated strings but this fix did not work and in fact exposed that the 69 affected strings are already translated in locale when they get rewritten with their English values. So what's happening here is that the locale data is not propagated immediately to config. Continuing the debugging.
Comment #2
Gábor HojtsySo the problem is with SiteConfigureForm::submitForm()'s side effect. That installs the update.module at a point where locale is already installed and LocaleConfigSubscriber is loaded. Prior to that, locale module is installed last right before the configure form (see install_profile_info()) so this does not happen earlier.
Comment #3
Gábor HojtsySo a possible fix is for LocaleConfigSubscriber to not respond to events in the installer. This may be a bit too drastic if profiles save some kind of translations to config files, but that sounds like an edge case, and they can always do the save via locale instead of config directly. Needs tests of course.
Comment #4
Gábor HojtsyComment #5
eiriksmThis does indeed fix the problem.
Just as a note for others (and a note generally):
Seemed as if I installed Drupal with the locale file pre-downloaded, this bug does not occur (so for example if I tried to install Drupal and then just delete all tables and reinstall, the bug does not appear). Will do a double check on that.
Just let me know if you want to delegate test writing :)
Comment #6
Gábor Hojtsy@eiriksm: tests would be great. I think this is still a problem if module A comes with some default config that it modifies in its hook_install() (assuming hook_install() runs after config import), the fix in #1 would be suitable for that, so I think we should bundle up the two fixes in this issue. Here is an updated patch that includes that too plus a little code comment update.
It definitely does not help if the given source string was not available as a community translation and it was modified but not translated. We cannot distinguish between those. But at least this fix catches the case, when the source was saved before but the target is not yet available and we don't save the same string as the target, so this bug will not occur later on after install either. Wanna take a crack at the tests? :)
Comment #7
Gábor HojtsyThis is a fallout of #2212069: Non-English Drupal sites get default configuration in English, edited in English, originals not actually used if translated.
Comment #8
Gábor HojtsyBTW the reason install and runtime need very different fixes is normally when a .po file is imported (automated or manually) or locale translation changes, there is an immediate config update following. In install, we import the .po file THEN display the site configuration form and then update the config, so there are some requests in-between that are affected. (Sidenote: We don't necessarily need to do it in these places, but its more practical to update config in one pass later because config may change with profile hooks. We can look into importing the .po files later in the same pass as config updates, but that is a different approach).
The runtime fix needs to be different because that is for yet nonexistent community translations that are then still empty, and we should not pollute locale with fake English translations there either.
Comment #9
eiriksmNew patch looks good. Tried to break it with the couple of cases I could think of, but still worked good :)
Not sure where to add the test though? LocaleConfigSubscriberTest? I would have to monkey patch $GLOBAL to get the one if to take effect it seems.
Comment #10
Gábor Hojtsy@eiriksm: For the installer problem, I think a test like InstallerTranslationMultipleLanguageTest may be modified to add further assertions based on strings in this bug. For the runtime use case LocaleConfigSubscriberTest looks like a sane option yup. Save a config file and check if English sourced ended up in locale. Please post the test-only patch to demonstrate that both fails are reproduced by the test.
Comment #12
eiriksmOK, so I ended up writing both tests in the installer, just for convenience really. I could not for the life of me trigger the bug in LocaleConfigSubscriberTest, I guess I would have to add a new test module for that actually.
As it also turns out, there is also another bug not fixed in the patch, reproduced as following:
- Install minimal install profile in another language (norwegian is what i use).
- After the installation, enable a module that has configuration with string already present. For example tour module.
- Strings are now overwritten again. For example the string "Translate" or "Language name" or... well, it is easy to spot.
So, a couple of things to note:
- The installer bug only triggers if you install in another language (obviously). So I added the checks to the "de" installation that we are testing already.
- To trigger the install related bug, one has to import a module with a config string that has already been translated. None of the modules in the testing_multilingual profile end up doing that, so I added the tour module into the mix. Might seem irrelevant, but that was the most transparent method I found to provoke the bug.
- The tests only fail if the patch in #2457551: Regression: optional default configuration is not translatable anymore in locale is applied.
- The tests still fail with the patch in this issue applied in addition to that patch, because of the additional bug mentioned above.
Since I am calling it a day now, I'll just leave the tests here for now.
Comment #13
Gábor HojtsyNot sure how is this possible to detect? The only way I can see to resolve this is to do the langcode change in the active config later in the config update batch if locale is already installed and only keep the langcode change in locale.module for the case when locale itself is installed, so previous ones are updated. Do you think that would make sense?
Comment #14
eiriksm@Gábor Hojtsy Of course, I do not know the reasoning and the development of this system like you do. And you probably have a better understanding of how we would end up in this function, but bear with me for a moment.
This is what we want, right:
Here is a case (attached) where
- The locale translation is not new.
- The existing translation is not empty.
- The new translation if not the existing translation.
- The new translation is the same as the source.
This way, the contributed translation for "Master" ("Hoved") gets overwritten by the "new translation" (not really a translation) "Master".
Is there really a case where we want that to happen? Can't we just add that as a check as well?
Sorry if I am ignoring something obvious here, but it makes more sense to me this way.
Patch attached. This one passes the added tests in #12 also (tests are also included). The patch in #6 does not pass the tests (the test in question that fails is InstallerTranslationMultipleLanguageForeignTest). As said earlier, the tests only "applies" if the patch in #2457551: Regression: optional default configuration is not translatable anymore in locale is applied. Woah that last sentence was a strange one :)
Comment #16
eiriksmHm. OK then. Seems this is deliberate. Reading through issues and code, I see that this seems well thought through, so there is obviously a case I am missing here. I still don't understand why we just say that (shortened for clarity):
Also. This means that no matter how pleased your are with the state of your translated interface, some module can just be enabled and take over your interface.
To test this, I enabled a module with the words "Password" "Username" "Create new account" and so on, to see what happened to the log in form in Norwegian. All texts are now in english :( Which in turn means (sorry for being dramatic), that a module could come along with a config file containing all known user facing text, and completely wipe out all your downloaded translations. More realistically, a lot of the short strings you have translated, could at any point be overwritten by enabling a module at least.
But I am starting to think that it is out of scope of this issue, maybe?
Comment #17
eiriksmRevisiting this after #2457551: Regression: optional default configuration is not translatable anymore in locale got committed.
Here is a patch that combines the fix in #6 with a test. Still find it a bit unfortunate what I point out in #16, but that is maybe out of scope.
Attached test-only and full patch.
Comment #18
eiriksmHah, that test-only was meant to fail of course.
Comment #22
eiriksmTest only failed now, updated patch including my previous mistake.
Comment #23
Gábor Hojtsy@eiriksm:
If the translation is not yet saved, we only save it if different from source (it falls back on the source automatically anyway):
if (($locale_translation->isNew() && $source != $new_translation) ||
If the translation is not new but empty (an empty translation was saved earlier), we only save if different from source (otherwise falls back on source):
(!$locale_translation->isNew() && ((empty($existing_translation) && $source != $new_translation)
If the translation is not empty, we only save it if the new translation is different from the prior one:
|| ((!empty($existing_translation) && $new_translation != $existing_translation)))))
Your change in this later one to not allow changing the translation back to the source string IMHO is incorrect. We cannot say that in 100% of the cases a translation should be different from its source. It should be possible to change a translation back to its source later on. The earlier logic items use the assumption that if the string was empty, it falls back on source (see LocaleLookup::resolveCacheMiss()), which is why it does not save a new translation as same as the source. However, it should be able to set a protected custom translation even to the source value, which is why we don't have the same check later. I don't think your changed logic is correct.
Comment #24
Gábor HojtsyRight, so in locale, everything may be changed with updates in translations (eg. from localize.drupal.org). If you want to protect translations, you can mark them as customized, in which case they should not be overwritten in locale. When config (translation) is changed, even protected translations will be modified of course because you are the initiator of the action. In locale, strings are shared between modules so you only translate things once. This includes default configuration. I don't think that discussion is in scope for this issue.
Comment #25
eiriksmI agree with both your comments. Especially the last one, about not being in the scope of this issue. :)
And I see your point where you disagree. I must admit I am not quite sure what the expected and best way to handle this is. But I am pretty sure about this:
These cases are:
- Probably edge cases
- Very complex to solve in the best possible way that handles all cases.
So I think my last point was. Let's solve this particular issue (which #6 does).
Setting back to needs review. The only difference between your patch in #6 and #22 is the tests. (edit: meaning it does not include the aforementioned changed logic)
Comment #26
Gábor HojtsyAll right. The tests you provided make sense to me :) @penyaskito is looking at this issue I believe, hopefully he an RTBC it if all looks good then :)
Comment #27
penyaskitoSuffered this problem with beta10 and an installation profile in a non-English language.
After applying the #22 patch to beta10 (it still applies), the issue was fixed.
Thanks!
Comment #28
Gábor HojtsyAdded this to that said distro: http://cgit.drupalcode.org/multilingual_demo/commit/?id=eac51fb0d07cfdc1...
Comment #29
alexpottThis issue addresses a major bug and is allowed per https://www.drupal.org/core/beta-changes. Committed b7d1db5 and pushed to 8.0.x. Thanks!
Fixed spelling mistake on commit.
Comment #31
Gábor HojtsySuperb, thanks!