Download & Extend

Plural exports could be broken

Project:Drupal core
Version:8.x-dev
Component:locale.module
Category:bug report
Priority:normal
Assigned:Unassigned
Status:closed (duplicate)

Issue Summary

My request is not only for D5 but D6 also.

I'm using 7 UI languages for my site with many contributed modules which aren't at the same level of translated material. Of course your module is of great help here.

It would really help me to have a "extract all strings from all modules in one file" option. It's already there for the sub-modules. That would spare me clicking on each module separately.

Thanks

Comments

#1

... and would be nice to have some kind of level indicator of the completeness of the translation for a specific module, the same manner as for the main language screen (ex 30/50 (60%)).

I don't know if it doable but woulb be a +

Thanks

#2

Status:active» closed (won't fix)

Well, you can easily export translations for all strings for all modules with the locale module interface (built into Drupal). It does not provide the same file:line information for strings but otherwise it is the most complete output you can get. So this is already built-in.

Status indicators for all languages is unfortunately not easily possible since we are parsing modules when you request and it takes lots of time. So we would need to parse all modules first and then gather their status from the database one by one. It is not impossible, we can store all extracted strings in the database and compare, but it is very time consuming to get that data and its a minor thing IMHO.

#3

Oh !! my fault,

I didn't even look in the core module. I tried it but the file seems corrupted when using with poEdit.

It does not provide the same file:line information for strings

What are the consequences ??

Nevermind for the "level of completeness" option which is in fact no so important, especialy if its' time consuming.

Thanks

#4

- "corrupted" is not much information... what's the error message?
- consequence of some file:line info missing is just that you cannot directly go and see the line of the file where it was used, but you need to search for it yourself

#5

Here you go (attached image) with the poEdit message.

AttachmentSizeStatusTest resultOperations
poedit-error.png17.31 KBIgnored: Check issue status.NoneNone

#6

Here you go (attached image) with the poEdit message.

#7

Title:Extract all» Plural exports could be broken
Project:Translation template extractor» Drupal core
Version:5.x-2.x-dev» 6.x-dev
Component:User interface» language system
Category:feature request» bug report
Status:closed (won't fix)» postponed (maintainer needs more info)

Please copy here any msgid_plural marked line which is followed by simply msgstr instead of msgstr[0], msgstr[1] etc. I am not sure this can be fixed (it depends on how the strings are discovered), but this is surely a core issue. Moving to the core queue. (I am pretty sure that this bug exists in both 5.x and 6.x since we did not change anything with this code AFAIR, so I mark it for 6.x).

#8

Will that be useful ?

#:  modules/poll/poll.module:336;416
msgid "1 vote"
msgid_plural "@count votes"

Of course, this is using the extract all strings form everything, I usually don't see that when using normal extraction

#9

Actually, we need the msgstr line as well :) so all lines before the next #:

#10

Couldn't find one "msgid_plural" line with msgstr !?!?! Maybe it's because there's no tranlsation yet?? Here's another example :

...

#:  modules/search/search.module:213
msgid "There is 1 item left to index."
msgid_plural "There are @count items left to index."

#:  modules/search/search.module:118
msgid "search content"
msgstr "rechercher dans le contenu"

#:  modules/search/search.module:118
msgid "use advanced search"
msgstr "utiliser la recherche avancée"

...

#11

Hm, you should have imported a .po file before, did you?

#12

Well, yes. For each module i've installed, I looked in the po dir for a fr.po for the translated strings which I imported. Wasn't that the right thing to do ?? In fact, what was my need in my first post was to have a complete po file, including all the added modules installed, for translation, and that, for every language enabled on my site.

#13

Hm, that was a good thing to do, for sure. For the plural translations to show up properly in exports, you need the plural forms in the database, which is only possible with importing a proper .po file for that language right now. Can you copy here the plural and formula fields from your locales_meta table for this language? From the above, it looks like the plurals field will be 0, and the formula might be broken as well. Let's see!

#14

Status:postponed (maintainer needs more info)» active

I've run into some issues with plural forms and looks like the problem is this:

If you import any po file without a proper Plural-Forms header it will delete the plural form in the language table. Though it will be fixed again if you import a file with the proper value, this introduces some randomnesss: The plural formula in your site will be that of the last po you imported (or none).

This is an issue too when you start translating from scratch on your site / l10n server. All produced po files will miss that plural formula.

Still worse, the problem propagates. If your last imported po had no plural formula, your exported one will miss it too. Then any one importing that file will have the formula deleted.

All this is creating random issues with translations all around, specially since the l10n server which many of us use for translations has this issue too. After checking some contrib modules, there are random po files in the repository missing that plural formula. Importing any of them will cause the site to lose its formula.

So I think we should fix this in the po import code, maybe not deleting the formula when there's already one. This should gradually be fixing all the translations, as in this case only the good ones (with formula) will propagate.

The current code is this (locale.inc).

<?php
       
// Get the plural formula and update in database.
       
if (isset($header["Plural-Forms"]) && $p = _locale_import_parse_plural_forms($header["Plural-Forms"], $file->filename)) {
          list(
$nplurals, $plural) = $p;
         
db_query("UPDATE {languages} SET plurals = %d, formula = '%s' WHERE language = '%s'", $nplurals, $plural, $lang);
        }
        else {
         
db_query("UPDATE {languages} SET plurals = %d, formula = '%s' WHERE language = '%s'", 0, '', $lang);
        }
?>

It looks to me like we could safely drop the second part (else) so the missing formulas don't propagate anymore...

#15

The other problem is that there are still .po files with a broken formula. So those, who have those, or their first imported .po file had a broken/incorrect formula, will not be able to fix that by importing a good one.

#16

#17

Status:active» needs review

Here is a patch implementing #14.

AttachmentSizeStatusTest resultOperations
locale-212749-17.D6.patch766 bytesIgnored: Check issue status.NoneNone

#18

Why does not contains the http://localize.drupal.org/l10n_server.xml the values of the plurals and the formula? Only when a language is switching to on should be set these constant values. I think it would be much simpler, and a few similar errors could be closed.

#19

Drupal core does not deal with the file at http://localize.drupal.org/l10n_server.xml, so whatever is included in there is unrelated to fixing issues in Drupal core.

#20

If we do a similar function as the _locale_get_predefined_list? For example: _locale_get_plurals_exception, which only contains the language codes, where the definition is different from the default "Plural-Forms: nplurals=2; plural=(n!=1);\n".

<?php
function _locale_get_plurals_exception() {
  return array(
   
"ar" => array("plurals" => 6, "formula" => "n==0 ? 0 : n==1 ? 1 : n==2 ? 2 : n%100>=3 && n%100<=10 ? 3 : n%100>=11 ? 4 : 5"),
    ...,
    ...,
  );
}
?>

Or we can list all items in a new _locale_get_predefined_plurals function. The help is available: http://translate.sourceforge.net/wiki/l10n/pluralforms

And then we can modify the insert command in the locale_add_language, and we never update these 2 fields later. What do you think?

#21

Yes, this can be done for Drupal 8. For Drupal 6 and 7, http://drupal.org/project/l10n_pconfig handles it. Drupal 6 and 7 are in API/functionality freeze and such changes are not going to get in. Drupal 6 and 7 were recently (this summer) hardened to not overwrite existing good plural formulas in #566390: Apply locale import protection to the plural forms.

#22

@Gábor Hojtsy:

Is this a D8 issue now?

#23

Version:6.x-dev» 8.x-dev
Status:needs review» active

Drupal 7 and 6 was I think fixed to not update plural formulas by default (if you use the default string import method). If you use the overwrite string import option it will overwrite the plural form in that case. Since looks like that was the only thing worked on in this issue, looks like this should not be applicable to any supported Drupal version anymore.

However, the discussion included the problem that plural string variants are not properly detected runtime, which is a design problem in how format_plural() works. It would be great to get that fixed, which is I think still applicable to 8, 7 and 6.

#24

Component:language system» locale.module

#25

D7 still contains the code Jose mentioned earlier:

<?php
         
if (isset($header["Plural-Forms"]) && $p = _locale_import_parse_plural_forms($header["Plural-Forms"], $file->uri)) {
            list(
$nplurals, $plural) = $p;
           
db_update('languages')
              ->
fields(array(
               
'plurals' => $nplurals,
               
'formula' => $plural,
              ))
              ->
condition('language', $lang)
              ->
execute();
          }
          else {
           
db_update('languages')
              ->
fields(array(
               
'plurals' => 0,
               
'formula' => '',
              ))
              ->
condition('language', $lang)
              ->
execute();
          }
?>

I just got a .po file with missing Plural-Forms and now we've lost the, before properly set, formula in the db.
I think independent if you use the overwrite string import option it shouldn't change the plural formula if the header isn't available or invalid.

#26

Status:active» closed (duplicate)

Duplicate of #655048: Plural formula information blanked when importing a poorly-formed .po file where the backporting happens. (It is fixed in D8, RTBC for D7).