In the export we discoverd an empty string.
The real problem is that the string was generated. That we could not discover.
But anyway we believe it should not end up in exports, because this causes a bug that is very hard to discover.
When an empty msgid is imported in the function _locale_import_one_string it will be detected as a header, which it is clearly not.

function _locale_import_one_string($op, $value = NULL, $mode = NULL, $lang = NULL, $file = NULL, $group = 'default') {
  static $report = array('additions' => 0, 'updates' => 0, 'deletes' => 0, 'skips' => 0);
  static $headerdone = FALSE;
  static $strings = array();

  switch ($op) {
    // Return stored strings
    case 'mem-report':
      return $strings;

    // Store string in memory (only supports single strings)
    case 'mem-store':
      $strings[$value['msgid']] = $value['msgstr'];
      return;

    // Called at end of import to inform the user
    case 'db-report':
      return array($headerdone, $report['additions'], $report['updates'], $report['deletes'], $report['skips']);

    // Store the string we got in the database.
    case 'db-store':
      // We got header information.
      if ($value['msgid'] == '') {
        $languages = language_list();
        if (($mode != LOCALE_IMPORT_KEEP) || empty($languages[$lang]->plurals)) {
          // Since we only need to parse the header if we ought to update the
          // plural formula, only run this if we don't need to keep existing
          // data untouched or if we don't have an existing plural formula.
          $header = _locale_import_parse_header($value['msgstr']);

          // Get the plural formula and update in database.
          if (isset($header["Plural-Forms"]) && $p = _locale_import_parse_plural_forms($header["Plural-Forms"], $file->filename)) {
            list($nplurals, $plural) = $p;
            db_query("UPDATE {languages} SET plurals = %d, formula = '%s' WHERE language = '%s'", $nplurals, $plural, $lang);
          }
          else {
//EDIT: here it will unset the db values in the plural processing
            db_query("UPDATE {languages} SET plurals = %d, formula = '%s' WHERE language = '%s'", 0, '', $lang);
          }
        }
        $headerdone = TRUE;
      }

      else {
        // Some real string to import.
        $comments = _locale_import_shorten_comments(empty($value['#']) ? array() : $value['#']);

        if (strpos($value['msgid'], "\0")) {
          // This string has plural versions.
          $english = explode("\0", $value['msgid'], 2);
          $entries = array_keys($value['msgstr']);
          for ($i = 3; $i <= count($entries); $i++) {
            $english[] = $english[1];
          }
          $translation = array_map('_locale_import_append_plural', $value['msgstr'], $entries);
          $english = array_map('_locale_import_append_plural', $english, $entries);
          foreach ($translation as $key => $trans) {
            if ($key == 0) {
              $plid = 0;
            }
            $plid = _locale_import_one_string_db($report, $lang, $english[$key], $trans, $group, $comments, $mode, $plid, $key);
          }
        }

        else {
          // A simple string to import.
          $english = $value['msgid'];
          $translation = $value['msgstr'];
          _locale_import_one_string_db($report, $lang, $english, $translation, $group, $comments, $mode);
        }
      }
  } // end of db-store operation
}

What happens is the first time you do an export and import the translation all seemed to be passed well. But it didnt. In fact the empty string will cause an empty header to be loaded and the plural and its formula will be unset in the database. Next time (a few weeks later in our case) we tried to import another generated file which also contained an empty msgid we got this error:

"The translation file fr.po contains an error: "msgid" is unexpected on line 8756."

Since the new file is generated without plural and formula in the db it lacks support for the plural strings, which causes the import to fail.

There is a simple core patch attached that prevents exports to generate empty strings. Unfortunaly we did not find the real cause why the empty string could be generated in the db.

How it could get there is unclear because there is clearly a check that prevents this:

function locale($string = NULL, $langcode = NULL, $reset = FALSE) {
  global $language;
  static $locale_t;

  if ($reset) {
    // Reset in-memory cache.
    $locale_t = NULL;
  }

  if (!isset($string)) {
    // Return all cached strings if no string was specified
    return $locale_t;
  }

  $langcode = isset($langcode) ? $langcode : $language->language;
...

Anyway adding the patch in the export doesnt hurt anything and prevents this bug from happening.

Comments

domidc’s picture

StatusFileSize
new2.89 KB

Queue patch for automatic testing.

domidc’s picture

Dont understand why the patch is ignored. Can someone explain why it doesnt work?

domidc’s picture

Status: Active » Needs review

Status: Needs review » Needs work

The last submitted patch, locale-empty-export-string.patch, failed testing.

gábor hojtsy’s picture

Title: empty msgid in export » Do not kill plural formula if the one in the .po file was broken or missing
Version: 6.19 » 8.x-dev

Well, the gettext po format defines the header as the msgstr pair for an empty msgid (look into any credible .po file), so I'd argue if you had a .po file where that entry did not represent the header, you had a broken .po file to start with. Also, the code above illustrates, that the plural formula is only ever overwritten if you *specifically* asked for existing data to be overwritten (which is not the default on import).

I agree it should not kill the plural formula in the DB if there was no plural formula or it was malformed, so that would be great to fix. That should accidentally also fix your issue, and empty msgid items will not be exported anymore. Retitling for that.

gábor hojtsy’s picture

Status: Needs work » Closed (duplicate)

#655048: Plural formula information blanked when importing a poorly-formed .po file actually exists with a better patch suggestion, so closing down as duplicate of that.

plach’s picture

Component: language system » locale.module