Problem/Motivation
When a drupal core release comes out the accompanying translation files will not be available immediately. It takes at least a few hours (up to days) before translation files of the (point) release. During this time the installation of Drupal in a non English language fail with a requirements fault "The %language translation is not available."
Currently the installer (install_get_localization_release()
) only provides a fallback in case the release is a development snapshot. For example 8.0-dev falls back to 7.0 and 8.2-dev falls back to 8.1.
The main problem we need to solve is the user experience. We need to make sure that a translation is found. Even a quite old translation would be acceptable. Once the Interface translation (local) and Update module are installed and enabled, a more intelligent update mechanism is available and more recent translations will be downloaded and applied.
Proposed resolution
* Implement simple (hardwired) fallback rules for core translation releases.
* Allow for multiple suggestions and multiple 'download' attempts.
Manual test of the patch
Normal scenario
- Start the Drupal 8 installation as usually at example.com/install.php
- In the first installation step, select a non-English language. (e.g. Dutch (Nederlands))
- Stop in the second step (select installation profile).
- Observe the translation file downloaded into sites/default/files/translations.
- This file should be: drupal-8.0-alpha3.nl.po (or perhaps drupal-8.0-alpha4.nl.po)
To re-run the test, remove any *.po file from the sites/default/files/translations directory and re-start the installation procedure at example.com/install.php
Alternative scenarios
- Change the drupal VERSION string in core/lib/Drupal.php [updated, file moved] (line 82) from '8.0-dev' to any of the following versions: '8.0-alpha1', '8.0-alpha2', '8.0-alpha4', '8.0-beta1', '8.0', '8.4'.
- Start the Drupal 8 installation as usually at example.com/install.php
- In the first installation step, select a non-English language. (e.g. Dutch (Nederlands))
- Stop in the second step (select installation profile).
- Observe the translation file downloaded into sites/default/files/translations.
- This file should be:
- 8.0-alpha1 -> drupal-7.0.nl.po
- 8.0-alpha2 -> drupal-8.0-alpha2.nl.po
- 8.0-alpha4 -> drupal-8.0-alpha3.nl.po
- 8.0-beta1 -> drupal-8.0-alpha2.nl.po
- 8.0 -> drupal-7.0.nl.po
- 8.4 -> drupal-7.0.nl.po
- Prepare for the next test by removing all files from the sites/default/files/translations directory.
Comments
Comment #1
Sutharsan CreditAttribution: Sutharsan commentedIMO this is must have:
This is nice to have:
Edit: modified 8.n-dev nice fallback
Comment #2
Sutharsan CreditAttribution: Sutharsan commentedThis is as far as I got...
install_get_localization_release()
calculates several release alternatives. Somewhere between the above must and nice scenario's.Comment #3
Sutharsan CreditAttribution: Sutharsan commentedThis is a working patch. It uses this fallback scenario:
Testing this patch is difficult since we only have one version (8.x-dev) available now. Only by injecting a Drupal 7 version number into the call to
install_get_localization_release()
this patch can be tested. The example below (line 2005, 2006 in install.core.inc) will result in downloading and installing the drupal 7.9 release into the translations directory.Some test scenario's:
Tip: to test the fallback you can check the content of the translations directory, no need to compleet the full installation and check the translation status page. Whipe the content of the translations directory before you start a new installation test.
[edit] blockquote replaced by code
Comment #4
Sutharsan CreditAttribution: Sutharsan commentedI realised I missed one scenario: What to fall back to in the first hours/days after 8.0 is released. We can fall back to 8.0-rc1 or perhaps hardcode another rc-release. I propose this additional rule:
Comment #6
Gábor HojtsyFor the 8.0 release I think we'll need to hard-code the prior RC release indeed. Otherwise the fallback rules look good to me :)
Comment #7
Sutharsan CreditAttribution: Sutharsan commented#4 fallback rule added.
Comment #8
YesCT CreditAttribution: YesCT commentedI think the rules are good. Can we just make this description match those? And take care of this todo?
spacing is off here.
If looking to make a list, 1354 talks about that.
http://drupal.org/node/1354#lists
needs @param with type (string?)
comments confusing.
the first release alternative... is no alternative.
Also. E.g. is confusing, lets just spell it out (I had to google it): for example.
Also, one uses a : and the other .
Anyway: I'd suggest:
this format is weird. just put it on one line (well, wrap it at 80 chars).
missing space after case.
Verb tense should probably be: Extracts
merge with the @param?
similar to previous note about 1354 standard for list.
Release extra text (e.g. "alpha").
huh? sounds contradictory.
what happens if the first file found that matches the language does not patch the major version number?
Is this just noting what version the file was that was picked?
some comments more inline in the code might help.
lets update this to match what's happening and also correct the spelling of desired. I think this can clarify by saying that means "online" and that is what that variable is for.
Check if any of the desired translation files are available and if the translation server can be reached. In other words, check if we are online and have an internet connection.
(but wrapped at 80 chars)
----
I can do some of this. Will need checking by @Sutharsan to see if I got the meanings correct.
Comment #9
YesCT CreditAttribution: YesCT commentedTried to clarify comments and fix standards for them.
Patch attached.
More questions to follow though.
Comment #10
YesCT CreditAttribution: YesCT commentedis "core" really the core major version number? the 8 in 8.2.
is "version" really the core minor version number? the 2 in 8.2.
or is "core" literally "8.x" for 8.0, 8.1 and 8.2 etc.
and version is the release of a translation alternative?
Perhaps some actual examples might be useful in the description.
like: for a core version of 8.2, an array might be returned:
[0][core]="8.x"
[0][version]="8.2"
[1][core]="8.x"
[1][version]="8.1"
[2][core]="8.x"
[2][version]="8.0"
[3][core]="8.x"
[3][version]="8.0-rc"
[4][core]="8.x"
[4][version]="7.0"
oops. extra newline.
what is the insert doing? why?
why only get the first file?
what happens if it's the wrong file? is it just as if an out of date translation is imported, or is no translation imported?
Comment #11
YesCT CreditAttribution: YesCT commentedComment #12
YesCT CreditAttribution: YesCT commentedthis issue title mentions install, but will this also fallback for when languages are added to an already existing site? that would be really nice for usability testing, to have adding a language actually automatically import the translation (even an old translation) for the ui.
Comment #13
KartagisI think in the patch on http://drupal.org/node/1914070#comment-7110328, on the line 'version' => $info['major'] - 1 . '.0', '.0' should vary according to the latest minor version.
K.
Comment #14
Sutharsan CreditAttribution: Sutharsan commentedAnswering #10
"core" is the core compatibility version number as defined in a .info.yml file. So literally "8.x" for 8.0, 8.1 and 8.2 etc. It is not the major version number.
"version" is the core release version number as defined in a .info.yml file. (Not necessarily the .info.yml of this installation, but that was the hole point of this fallback of course.) It is not the minor version number.
Each record with core/version info is a fallback alternative, sorted in order of preference, the most preferred first.
The example in #10 is not correct. What about this:
Detected version is "8.2":
[0][core]="8.x"
[0][version]="8.2"
[1][core]="8.x"
[1][version]="8.1"
[2][core]="8.x"
[2][version]="8.0-rc1"
[3][core]="7.x"
[4][version]="7.0"
Different scenario, detected version is "8.2-dev":
[0][core]="8.x"
[0][version]="8.1"
[1][core]="8.x"
[1][version]="8.0"
[2][core]="8.x"
[2][version]="8.0-rc1"
[3][core]="7.x"
[4][version]="7.0"
We pick the first file because we expect only one translation file per language. If there are multiple, either the person who installs has manually place another file there or the installation profile was shipped with multiple. The installation system will not download a second file from the same language. The worst that can happen in either case is that we import an outdated translation file. The next time the translations get updated, this will corrected.
This insert is to tell the automatic import process which project is installed for which a translation should be imported. An API-function would be nicer, but none is available.
Answering #12
This code is only used for installation and not intended for runtime. At runtime we use version data from the update module, but at install time this is not available. At runtime we currently only use one translation version and one check for the presence of a translation file for each project/language combination.
Answering #13
+ // All releases may a fallback to the previous major release (e.g., 8.1 falls
+ // back to 7.0).
+ $releases[] = array(
+ 'core' => $info['major'] - 1 . '.x',
+ 'version' => $info['major'] - 1 . '.0',
);
I wish we knew the latest version of the previous release. But remember this is the very last resort and will only be used as long as there is no rc1 release. This exactly what the current code does.
Comment #15
KartagisI made that comment and I still think the minor version should be calculated somehow, because I occasionally git pull drupal 8 and install it just to be able to port my module successfully. I sometimes install in my own language as well to see if anything breaks. I couldn't help noticing the translation for cron options mostly includes 0 sec as translation is an ongoing process and those strings probably weren't translated at that time, and also new strings might be added from time to time.
Comment #16
YesCT CreditAttribution: YesCT commented#9: install-language-version-fallback-1914070-9.patch queued for re-testing.
Comment #17
Sutharsan CreditAttribution: Sutharsan commented#9: install-language-version-fallback-1914070-9.patch queued for re-testing.
Comment #19
wusel CreditAttribution: wusel commentedAs of #2030537-3: Translations not downloaded when adding a new language:
If I install the alpha 2 or a alpha2-xxx.dev of the D8-core and select German language during the installation, then I get the german D7-po-file (drupal-7.0.de.po) but nor D8-po-file. A German D8-po-file is available on the translation server of d.o. at http://ftp.drupal.org/files/translations/8.x/drupal/drupal-8.0-alpha2.de.po
Wusel
Comment #20
jair CreditAttribution: jair commentedComment #21
pp CreditAttribution: pp commented#7: install-language-version-fallback-1914070-7.patch queued for re-testing.
Comment #22
pp CreditAttribution: pp commentedI rerolled the patch. The $server_available variable is nowhere used, I deleted it.
Comment #23
Gábor HojtsyComment #24
Gábor HojtsyComment #25
pp CreditAttribution: pp commentedUse named subpatterns
Comment #26
shnark CreditAttribution: shnark commentedPatch applied, removing needs reroll tag.
Comment #27
shnark CreditAttribution: shnark commented[duplicate comment]
Comment #28
Gábor HojtsyI think the needs manual testing tag pretty much applies to this still.
I'm not entirely sure what kind of automated tests to provide. We can do a set of tests to install in a foreign language. There is already a test to install in German(?) I believe. We can seed such a test with different files and see if it loads it or not. Anybody wanna help with this?
Comment #29
Sutharsan CreditAttribution: Sutharsan commentedRerolled the patch.
\Drupal::VERSION
now used instead ofVERSION
.Comment #30
Sutharsan CreditAttribution: Sutharsan commentedThis patch adds a unit test for install_get_localization_release() which calculates the fallback versions.
The todo's will be resolved in the next patch which will add fallback versions for alpha and beta releases too.
Comment #31
Sutharsan CreditAttribution: Sutharsan commentedVersion fallback now includes alpha and beta releases too. Tests have been modified to incorporate this too. While working on this, I've simplified code and comment of
install_get_localization_release()
.The modified fallback schema:
More fallback details can be found in the test class:
InstallerTranslationVersionUnitTest
[edit] dev-fallback added
Comment #33
Sutharsan CreditAttribution: Sutharsan commented#31: install-language-version-fallback-1914070-31.patch queued for re-testing.
Comment #35
Sutharsan CreditAttribution: Sutharsan commentedManual installation failed in an endless loop because FileTranslation::findTranslationFiles() could not detect filenames like 'drupal-8.0-alpha2.nl.po' only 'drupal-8.0.nl.po'.
Comment #36
Sutharsan CreditAttribution: Sutharsan commentedThe endless loop I mentioned in #35 was caused by a code failure. The filename of the downloaded file was not recognised. To prevent an endless loop and I've added an extra check in the translation requirements check. Because this is an error that can not be solved by the user, I did not add a new error message, but reused the existing.
Comment #37
Sutharsan CreditAttribution: Sutharsan commentedPatches in #35, #36 look weird. Something gone wrong with re-rolling?
Comment #38
Sutharsan CreditAttribution: Sutharsan commentedThanks to the interdiffs in #35 and #36 it was easy to make a proper patch.
Comment #38.0
Sutharsan CreditAttribution: Sutharsan commentedUpdated issue summary.
Comment #38.1
Sutharsan CreditAttribution: Sutharsan commentedUpdated issue summary.
Comment #39
Sutharsan CreditAttribution: Sutharsan commentedUpdated the issue summary with manual test scenario's.
Comment #40
jpd4nt CreditAttribution: jpd4nt commentedHi. Reviewing and testing this patch.
The test examples work as specified.
Looking at the patch, is there any reason why it does not loop through all the minor version?
Say if alpha5 came out, the po file it would get is the 7.x version, not the alpha3 which is newer.
This could reduce admin over head for new releases at the expensive of install time length.
Also which is probably a separate issue, none of the release bundles use version other than -dev so this patch is not being used to work out the version number from alpha releases.
Comment #41
Gábor Hojtsy@jpd4nt: so this is happening in the installer. We want to avoid doing too many HTTP requests in the installer, because that takes a lot of time. Ideally, we would have one endpoint on the server, where we tell the version number and it directs us to the right file, or all the possible version numbers would have a file that have exported status of translations at that phase, so you would be sure to get the translation right from the first load. But that requires more server side solutions, and we'd ideally need a stop-gap solution at least for now that lets people test Drupal 8 with new translations :)
Comment #42
jpd4nt CreditAttribution: jpd4nt commentedHi Gabor.
As a stop gap solution to provide translation, this patch seems to work nicely.
Those two points were the only ones that jumped out at me when reading through.
Comment #43
bserem CreditAttribution: bserem commentedI can confirm that patch in #38 works for me too.
Comment #44
penyaskitoI did a code review and didn't find any nitpick. But the way this patch adds a lot of value in terms of readability and understandability of the logic behind.
The only question that came to my mind was #40, but Gábor already argumented on #41.
Some users already reported that they tested this, so I'm very tempted to RTBC.
Comment #45
Sutharsan CreditAttribution: Sutharsan commented#38: install-language-version-fallback-1914070-38.patch queued for re-testing.
Comment #45.0
Sutharsan CreditAttribution: Sutharsan commentedUpdated issue summary.
Comment #46
alexpottUnused variable $info
I don't think this change is necessary to fix the problem and is unrelated.
Missing line at end of file
Comment #47
YesCT CreditAttribution: YesCT commentedI might be missing something, but where is the version set? I wanted to see what the alphas thought their version was.
Drupal.php moved to core/lib/Drupal.php and I dont see VERSION in that file.
grepping for other stuff... didn't find it either.
Comment #48
andypost@YesCT
const VERSION = '8.0-dev';
incore/lib/Drupal.php
Comment #49
Sutharsan CreditAttribution: Sutharsan commentedAll remarks by alexpot in #49 processed.
@YesCT, alternatively you can change the version in all *.info.yml files to the desired version number.
Comment #50
vijaycs85Thanks @Sutharsan. The interdiff looks fixing exactly what we need to address in #46. Back to RTBC now.
Comment #51
vijaycs85sorry for the quick RTBC on a 0 bytes patch :( @Sutharsan can you please check and attach the real patch please? :)Thanks @alexpott for checking it and letting me know :)
Created new issue #2102069: Throw validation error on 0 bytes file upload. to not to test 0 bytes patches.
Comment #52
Sutharsan CreditAttribution: Sutharsan commentedSorry :S
Comment #53
vijaycs85Thanks again @Sutharsan. The code looks good to me. I just send the test only patch(at #1970534-12: Patch testing issue )to see if it fails without patch. Can RTBC once it is RED :)
Comment #54
vijaycs85As per https://qa.drupal.org/pifr/test/642163 the tests failing. So RTBC. Thanks @Sutharsan.
Comment #55
Gábor Hojtsy#52: install-language-version-fallback-1914070-49-1.patch queued for re-testing.
Comment #56
Gábor HojtsyComment #57
Gábor Hojtsy#52: install-language-version-fallback-1914070-49-1.patch queued for re-testing.
Comment #58
Xano#52: install-language-version-fallback-1914070-49-1.patch queued for re-testing.
Comment #59
Gábor HojtsyElevating to major because Drupal 8 keeps downloading Drupal 7 translations instead of Drupal 8 translations, and that is ridiculous. We cant get people test the actual Drupal 8 translations without this.
Comment #60
alexpottCommitted 7b08fd6 and pushed to 8.x. Thanks!
Fixed some minor spelling mistakes on commit
Comment #61
alexpottForgot to add my spelling fixes... so reverted and recommitted :( d3d34a5 and pushed to 8.x. Thanks!
Comment #62
Gábor Hojtsyhttps://drupal.org/node/2113953 is the change notice I wrote up. This change has significance for all the people trying out Drupal 8 and as said above is not yet the final solution. Eg. even after Drupal 8 alpha3 is released, it will still download the alpha2 translations, unless the fallback patterns are manually updated in the code. We either need a release process where that is maintained or a server side solution. So far the people I talked about preferred a server side solution. Opened #2113957: Build server side version fallback system for translations and #2113955: Rely on proper server side version fallback for translations for that.
Comment #63
Gábor HojtsyYay, did I say that although this is not a final version, this will be an enabler for us to try out things and find bugs. :D See I found a pretty big one right away: #2114069: Routing / tabs / actions lack way to attach context to UI strings :)
Comment #64
wusel CreditAttribution: wusel commented@#19
I have installed drupal-8.0-alpha4 and drupal-8.x-dev_2013_10_24 on xampp 1.8.2-2 and Win7-64bit.
Thank you all, it now downloads the file "drupal-8.0-alpha2.de.po", if I choose German during install.
Wusel
Comment #64.0
wusel CreditAttribution: wusel commentedfile version is in moved.
Comment #66
hansfn CreditAttribution: hansfn commentedVersion fallback is currently broken by the move to semantic version numbers.
Reopen until #2349263: Add support for semantic version numbers in installer is fixed.
Comment #67
Sutharsan CreditAttribution: Sutharsan commentedNot sure why this issue needs to be reopened. The other issue pretty well covers the problem that fallback scenarios added in this patch don't work anymore. Reverting status.
Comment #68
hansfn CreditAttribution: hansfn commentedThis was opened because it's much more likely that other people with the same problem find this issue (if it is open), than the other issue that I filed. I just wanted to avoid duplicate issues ... Anyway, thx for adding the related issue.