We discussed providing locale support for the installer. Here is a patch, which works for me, but obviously needs some discussion. What did I do to support a locale in the installer:

  • Wrapped all strings in the installer files in st() calls. Some having placeholder text were already wrapped.
  • Made possible to run the PO import in installer mode. Theme() and t() are not defined in this mode, so setting of error messages need to be dealt with as a special case for normal operation and for the installer.
  • The PO import in installer mode is only interested in st() calls now (no format_plural support). This does not seem to be a problem yet, since format_plural() is not needed in the installer.
  • In installer mode, the strings are stored in an array and retrieved from the st() function.
  • The tricky part is indentifying the locale to use. I stepped this over for now. The installer tries to read an installer.po file from a folder named 'locale'. If it does not exist, English is used; if it does exist, the translations in this file are used. There is no support for installer language selection in this patch. First I would like to discuss the approach taken here before stepping further.

I have already updated the extractor.php script to handle st() functions, and generate an installer.pot file with all the st() wrapped strings found throughout core, regardless where they are. This enables translation teams to provide that installer.po file st() is trying to look for.

Comments please!

Comments

gábor hojtsy’s picture

StatusFileSize
new8.73 KB

This is how an installer.pot file looks now, unsuprisingly.

moshe weitzman’s picture

i took a read through the code and it looks terrific to me. nice work. i'll try to test soon ... we should add automatic locale selection to core anyway. presumably we default to the browser supplied language (like google, and many others).

gábor hojtsy’s picture

@Moshe: we are in talks with Jose on integrating some parts of i18n into core, and a language selector will be among these parts. We should be able to do it so that the installer can also make use of it.

killes@www.drop.org’s picture

I am ok with the changes to locale.inc

Steven’s picture

Making st() more like t() is not advisable, as st() does automatic escaping and adds <em> to the %foo substitutions, while t() does not. This is rather inconsistent.

I was never really happy with st() in the installer though. The only reason it was left in is because it makes a lot of code shorter and more readable.

Perhaps it is time to upgrade t() a bit. There are, today, three kinds of substitutions with %vars:
- Insertions of pure HTML (insert as is)
- Escaped text insertions (call check_plain and pass to t())
- Placeholder insertions (call theme_placeholder, which also calls check_plain, and pass to t()).

We could make a lot of code more readable if we could merge these calls into t(). It would also make it harder to forget them. This could be achieved by defining substitute prefixes (e.g. "%foo" = escape as placeholder, "@foo" = escape only, "!foo" = insert as is).

Also, theme_placeholder() should really only be used for user-submitted parts of a sentence that you want to emphasise. st()'s blind use of it means that there are a lot of unsightly emphasised words in the installer (e.g. "Drupal has been installed").

This might seem out of place here, but let me clarify: I'm working on installer requirements checking, which needs to work both at install time and at run-time. So far, the only method I've found that works is to set $t to "st" or "t" and then call $t("string", ...). This only works if st() and t() behave exactly the same though.

Even more ideal would be to remove st() all together, and make it so that t() works during the installation as well. I think we can just move t() to bootstrap.inc, but I'm not sure.

Either way means a lot of search-and-replace in core, but IMO it's worth it.

gábor hojtsy’s picture

Good idea Steven! Forked the t() extension suggestion into a new issue, and submitted a patch (with a slightly different prefix format).

Ontopic here is that we need a reliable way to extract strings used in the installer. If we can identify files in which all t() calls should be marked as used by the installer, then it is fine to get rid of st() altogether. Otherwise seeking st() calls in the Drupal source is the way now to spot installer messages which need to be extracted to a separate POT file.

gábor hojtsy’s picture

StatusFileSize
new30.97 KB

After we forked this patch, and the t() changes got applied, here is another round with this patch. Reread #1, if you need to see the basic picture. The concept is still the same. The patch evolved a bit.

I changed _locale_import_one_string(), so that now it gets a clean $op parameter, and it received 'db-store', 'db-report' and 'mem-store', 'mem-report' operations. This way it can store strings in the DB or in the memory, and later storage stats (or in case of memory storage, the strings themselfs) can be queried. The lot of changed-looking lines in _locale_import_one_string() were not changed a bit, they were just indented 4 spaces further, to properly place them inside the switch() needed for the operations.

It still stands that I have stepped over the PO file selection problem for now. I would welcome suggestions on where should we look for the PO file, and whether we should present a language selection interface in the installer or not. What should we base such a selection on? Scanning for PO files in some folder? What folder?

This patch was tested with Drupal HEAD a few minutes ago, and both the installer and the site seems to work fine.

kkaefer’s picture

Hmm, just a quick thought, but what about including the installer translations directly into core? There are not that many strings for each language. We move all the install-related files to /install, and have a subdirectory "translations" with all the po files there. The installer looks what files are there and presents them at the very first step. You can choose a language and the po file is loaded.

gábor hojtsy’s picture

I also gave some more serious thought into file placement. There are several problems with shipping Drupal with installation translations. First I am sure most translations will not be ready on day one for the release. We would also always need a gatekeeper with core write access to commit translations.

Worse is that if you select a language for the installer, you expect the installed product to speak that language right? This is how it works in desktop systems and such... But we are definitely not going to ship with any complete language translation. They are really big. To put this into perspective, Drupal-4.7.3.tar.gz is 473.6 KB, and the hu-4.7.2.tar.gz (which is also valid for the 4.7.3 version) is 124.1 KB. One single complete translation is quarter the size of the whole core (both being compressed)!

What we can do in the installer is to search for a PO file for the installer and the same language PO file for the complete interface. People could download these in a package separately from Drupal core, and these would extract nicely into the Drupal core directory tree. The installer could use the language found and could import the interface translation in that language at the end.

Interface (and installer) translations are bound to the install profile used, since the install wizard is provided by the profile, the modules turned on are set by the profile, etc. We could think about treating PO files as .css and .install files are treated now by core: create one PO file for the default turned on modules and spread module specific PO files as module specific .css and .install files are spread among modules. Then if users enable that module, the PO file would import automatically.

Ps. we could support setting up of multiple languages in install time, but importing more then one language will surely time out PHP (the above Hungarian translation unzipped is half a megabyte, and it all needs to be read in and parsed by Drupal).

Opinions?

gábor hojtsy’s picture

StatusFileSize
new36.13 KB

Here comes a (hopefully) core ready version, with a locale selection as requested. Works this way:

  • You first select the profile to use (or the only one is autoselected for you).
  • The system 'greps' for all .po files for that profile, if more then one is available, you get the options to choose.
  • The locale you have choosen if used to present the installer UI. Later on the locale module is also enabled by default, if any locale was available (either used or not) during install time.

This patch continues on the assumption that Drupal is not going to get shipped with any installer PO files. You download your language pack, and it is going to have this new folder structure in it, having profiles/default/it.po (or an Italian translation) in it for example. You unzip that into the Drupal folder before installing. Once you start the installer, it will automatically be Italian if you only have this locale or will offer all available languages if you have more locales there.

I would really appreciate reviews of this patch, so we can get this into the next Drupal version. Automatic OI import on module installs can be easily done later (without core modification if we would not be able to make it with that feature). But without this patch, people will not be able to start off their install process in their own language.

I have tested this patch locally with some dummy translations (which give nice error messages because of being malformed PO files), and a small Hungarian sample translation. It works.

gábor hojtsy’s picture

StatusFileSize
new12.39 KB

Here comes an updated POT file, so you can test with it (add in some sample translations, name it something like de.po and put it into the profiles/default folder).

Stefan Nagtegaal’s picture

Goba, could you make some screenshots too?

Steef

gábor hojtsy’s picture

StatusFileSize
new36.21 KB

Updated patch, which supports selection of the built in English locale too. Screenshot coming.

gábor hojtsy’s picture

StatusFileSize
new97.94 KB

Here are two shots showing the locale selection (which shows the locales available for the profile) and the database setup screen after selecting the Hungarian locale.

Stefan Nagtegaal’s picture

Goba,

i think that 'Select your installation locale' is a little weird. I'm not a native English speaker, but I think something 'Choose your preferred language', is a little better.. though, I'm not sure...

gábor hojtsy’s picture

It was worded after 'Select your installation profile' which is the exact title displayed if you have one or more profiles to choose from. I am open for better ideas :)

gábor hojtsy’s picture

Status: Needs review » Reviewed & tested by the community
StatusFileSize
new36.18 KB

Patch wording fixed as Stefan requested. Also fixed a small glitch in locale name printing, plus made English less of a special case (nicer code). Tested it and it works fine. I feel this is RTBC.

drumm’s picture

Status: Reviewed & tested by the community » Needs work

Parse error: syntax error, unexpected ';' in /home/drumm/drupal/drupal/C-HEAD/includes/install.inc on line 274

gábor hojtsy’s picture

StatusFileSize
new36.82 KB

Tested the patch more deeply. Fixed the issue of $locale colliding with the bootstrapped Drupal $locale global (needed to rename the installer one to $install_locale), plus made the successful install page t()-able.

gábor hojtsy’s picture

StatusFileSize
new53.51 KB

Screenshots of the installer: 1. language selection, 2. chmod error in Hungarian, 3. successful installation messages (with chmod warning)

gábor hojtsy’s picture

Status: Needs work » Needs review
jose reyero’s picture

StatusFileSize
new10.85 KB

I've tried the patch and it works just fine.

So a big +1 for this one.

The only thing I miss is that when you get back to the site configuration -create the first account, etc- it is again in English, so maybe the default install profile should take care also of enabling the locale module and give a chance to the user to upload the .po file.

However, this patch will allow as it is to provide pre-packaged language installations, just adding some custom profile, so I'd commit it right away and would think about more complex extensions later.

Btw, partly translated install po file for Spanish attached.

drumm’s picture

Status: Needs review » Fixed

Committed to HEAD.

gábor hojtsy’s picture

Jose, the locale module is already enabled by this patch in case you have any language additionaly to English for the installer. The modules to import their .po files in their _install() hooks is the next step, and is a subject of another issue.

gábor hojtsy’s picture

Status: Fixed » Reviewed & tested by the community
StatusFileSize
new638 bytes

In the meantime, get_t() was introduced (by Steven?) to do exactly what I have done in a ternary operator here. So use get_t() instead. A oneliner.

drumm’s picture

Status: Reviewed & tested by the community » Fixed

Committed to HEAD.

Anonymous’s picture

Status: Fixed » Closed (fixed)