Hi,

I'm using 'l10n_update' on roughly a dozend of non-English Drupal sites; generally, the module seems to be quite reliable, however on one site the translations seems to "deteriorate" for a couple of weeks. The symptoms are that translations are not properly updated, and/or existing translations are lost. The result is a crude mixture of localized and English strings in the UI. The "lost" translations apply as well to core as to contrib modules.

Some examples:

Access denied page for anonymous users; these strings from core definitely exist in the translation files, but I'm getting the English version when "German" should be the default language:

Access denied
You are not authorized to access this page.

Similar, the search page isn't localized anymore:

Search
Enter your keywords:

Weekday and months from CCK 'date' module in a Views listing:

Friday, 6. Januar 2012 (01. KW) 
Thursday, 5. Januar 2012 (01. KW) 
Wednesday, 4. Januar 2012 (01. KW) 
Dienstag, 3. Januar 2012 (01. KW) 
Monday, 2. Januar 2012 (01. KW)

Here, the month and one weekday is translated, the other weekdays appear in English. Since I'm using the same View on other sites as well, I definitely know that those strings are translated in the module. They were lost during translation updates.

According to ./admin/build/translate, the site has lots of untranslated interface strings:

  • Französisch 10253/40928 (25.05%)
  • Deutsch 12635/40928 (30.87%)

Another site with a similar set of modules and configuration reports:

  • Deutsch 14547/23281 (62.48%)

If these figures are correct, every second translated string was lost.

The problem with this bug report is that I can not provide a procedure to reproduce the issue; as on all sites, I'm using several dev releases, so #1001554: Make it possible to fetch .po files for dev modules/core can not be the root issue. I'm using Drush, but since there were no core updates for a while, #817500: Core update deletes core .po files probably isn't the root issue, as well.

Last but not least this issue has two components:

a) How to fix translations? How to restore the lost 30% of translated strings? Obviously I don't have any idea how to restore a state with valid translations on the site; according to ./admin/build/translate/update, there are no pending translation updates available. Any help on this is greatly appreaciated!

b) Feature request. Similar to #1016508: Add option to change reload all translations I'd strongly suggest to implement a mechanism to force a refresh of all translation files, like it was provided by the depreacted Po-Re-Importer module. That might at least fix symptoms like those that I'm encountering.

Thank you!

CommentFileSizeAuthor
#8 translations-before-reset.jpeg40.96 KBasb
#8 translations-after-reset.jpeg40.45 KBasb

Comments

asb’s picture

asb commented

Two small additions: I tried to trigger translation updates by adding another language (fr) to the site (doesn't restore the missing translations). After removing the new language (fr) via ./admin/settings/language/overview I ran ./admin/build/translate/update which claims for every (!) module that local translations exist ("Lokal sind Aktualisierungen verfügbar"), and the module starts to download 267 french .po files. That results in a large list of errors ("the selected language doesn't import the import"):

Die ausgewählte Sprache wird nicht für den Import unterstützt.
Importierte Übersetzungsdatei /tmp/translation-o4j71z konnte nicht importiert werden.

Clearing all caches via 'admin_menu' or 'Drush'' (cc + all) fixes this.

Second observation: The affected site uses the 'Quick Tabs' module, my comparison sites don't. Only on the affected site I have two "text groups" ("Textgruppe"): "integrated interface" ("Integrierte Oberfläche"), and "Quick Tabs". I don't know what these "text groups" are, and if they are supported by 'Localization update'.

sutharsan’s picture

sutharsan commented

The meaning of "Deutsch 12635/40928 (30.87%)" is:
12635 string translations en-de available. These are imported or locally translated.
40928 translatable strings are detected. This is not the sum of all translatable string in core and installed contrib modules. Only when code is executed strings are 'detected' by Drupal and stored in the database. You may have noticed that this number grows over time even when you don't update or enable modules. The second site (.../23281) is probably a more recent site.

I'll try to write some quick and dirty code to reset the translations.

asb’s picture

asb commented

I'm not sure what you mean by "The second site [...] is probably a more recent site" (core and module versions are identical on both sites), but thanks a lot for the quick reply and the hope for a fix ;)

sutharsan’s picture

sutharsan commented

I mean that the site with these figures 'Deutsch 14547/23281 (62.48%)' is probably more recently build. The more modules you add and remove or upgrade the more strings are detected and the higher the number of detected string (40928) get.

asb’s picture

asb commented

I understand, thanks. But no, the 62.48% site is definitely older (started in 2006), the 30.87% site is from 2008/2009.

For comparison: Some more values from ./admin/build/translate/overview for the language "Deutsch" for

  • sites started around 2005: "13538/19922 (67.96%)", "15116/27992 (54%)"
  • sites started around 2006: "14291/20601 (69.37%)", "12484/17622 (70.84%)"
  • sites started around 2007: "12049/17642 (68.3%)"
  • sites started around 2008: "12139/60380 (20.1%)"
  • sites started around 2009: "13205/18993 (69.53%)"
  • sites started around 2010: "10900/13367 (81.54%)", "14093/27629 (51.01%)"

Very confusing and hard to see a pattern. However, values around 20-30 seem to be exceptions.

sutharsan’s picture

sutharsan commented
Status: Active » Postponed (maintainer needs more info)

I've written an module to delete the translations. Localization Update Reset module, you'll find it in my sandbox: http://drupal.org/sandbox/sutharsan/1397740. It has a 7.x (master) and 6.x (6.x-1.x) branch. You can use this module to reset your translations. Create a database backup of (at least) the locales_source and locales_target tables!! Use it at own risk.
btw, I'm not sure if this module will ever be released, it has to prove its usefulness first.

The module flushes the locales_source and locales_target table per language and respects local changes as per configuration of the Localization Update module. To reset the Localization Update module status, it flushes the l10n_update_files table.

I'm very interested in the differences you will find before and after flushing. And especially in any further loss you suspect.

asb’s picture

asb commented

Thanks!

How to get the 6.x-1.x branch:

git clone --branch 6.x-1.x http://git.drupal.org/sandbox/sutharsan/1397740.git localization_update_reset

After enabling the module, I get the message "Your translation update mode does not allow translations to be deleted. Change this setting before you continue" at ./admin/build/translate/delete ("Change this setting" points to ./admin/settings/language/update without further specification what has to be changed).

Further the admin page suggests, for example: "Deutsch (12649 translations)"; clicking the button "Delete translation" results in the message "Todo ... translations deleted" (green) and below "Your translation update mode does not allow translations to be deleted. Change this setting before you continue" (yellow). How do I allow to delete translations?

asb’s picture

asb commented
Status: Postponed (maintainer needs more info) » Active
StatusFileSize
new translations-after-reset.jpeg40.45 KB
new translations-before-reset.jpeg40.96 KB

OK, the message quoted above seems to be misleading. A couple of days later, "new" translations were detected for all core/contrib modules, and downloaded by 'Localization update'.

After processing, ~87% of detected strings were translated (opposed to ~30% before). Screenshots of ./admin/build/translate/overview before and after the are attached. The localized site looks properly now, including strings from 'date' module.

Except for the mentioned oddities, the sandbox module works well and solves the issue for me. So many thanks again for this fix!

sutharsan’s picture

sutharsan commented

Thanks for the before/after numbers :) This shows me that no translations were lost. Before: 12649, After: 12650. The difference is the number of source strings: Before: 40961, After: 14407. This does not explain why your strings are not translated, but if the translation is available on the site, is unlikely that it is caused by Localization Update.

asb’s picture

asb commented
Status: Active » Closed (works as designed)

OK, closing the issue.

Since your sandbox module fixed the issue for me, it'd be great if you'd consider to integrate the "reset" functionality into 'Localization update'. It makes 'Localization update' even better if it can fix issues that it does not cause ;)

sutharsan’s picture

sutharsan commented

If people will continue to tell us that they need this, we will consider including it.

asb’s picture

asb commented

Follow-up issue: #1016508: Add option to change reload all translations