Problem/Motivation

For the moment there's no check if the file is UTF8 or not

Steps to reproduce

Try to upload file on with non UTF-8 encoding on admin/config/regional/translate/import, it uploads without any validation.

Proposed resolution

Include new validation plugin to check if the uploaded file is encoded with UTF8.

Remaining tasks

Fix errors in MR
Add tests

User interface changes

API changes

Data model changes

Release notes snippet

Issue fork drupal-1630568

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.2.x-dev

Drupal 8 is end-of-life as of November 17, 2021. There will not be further changes made to Drupal 8. Bugfixes are now made to the 9.3.x and higher branches only. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.2.x-dev » 9.3.x-dev
catch’s picture

catch
he/him
Primary language English
commented
Title: Check if uploaded .po is UTF8 » Validate that uploaded .po files are UTF8
Component: language system » locale.module
Category: Bug report » Task
Issue summary: View changes
Issue tags: +Needs issue summary update

This seems like a task. Moving to locale module since language system doesn't handler interface translation.

rootwork’s picture

rootwork
he/him
Primary language English
Location 🇺🇸 US-Pacific 🌎 Chinook (Multnomah, Clackamas) & Cowlitz lands 🌹 Portland, OR
commented
Issue tags: +Novice, +Portland2022

Tagging as novice just for the issue summary update (though feel free to work on it beyond that)

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.15 was released on June 1st, 2022 and is the final full bugfix release for the Drupal 9.3.x series. Drupal 9.3.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.4.x-dev branch from now on, and new development or disruptive changes should be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

anjali rathod’s picture

anjali rathod
Primary language English
Location India
commented
Assigned: Unassigned » anjali rathod
anjali rathod’s picture

anjali rathod
Primary language English
Location India
commented

Do we have to write a test for it @catch ?

Laureatus made their first commit to this issue’s fork.

Laureatus opened merge request !2658

quietone’s picture

quietone commented
Version: 9.4.x-dev » 9.5.x-dev
Issue summary: View changes
Status: Active » Needs work

I have added that standard issue template. It still needs to be updated and that is suitable for a novice, leaving tag.

@Anjali Rathod, good question about tests! In nearly all cases, the answer is yes. And that is true for this issue. There are several steps, or gates, that an issue must pass before it is marked RTBC. For most issues following step 10 in the Review a patch or merge request task of the Contributor guide is sufficient. The complete list of core gates has more topics.

Laureatus’s picture

Laureatus
he/him
Primary language German
commented

I have made the code changes you requested and renamed the function to file_validate_encoding. I also checked with #UX and asked for some feedback on the error message. I think "The file is not encoded with UTF-8." is the best solution.

ravi.shankar made their first commit to this issue’s fork.

ravi.shankar’s picture

ravi.shankar commented

Addressed drupal CS issue of MR.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

quietone’s picture

quietone commented
Assigned: anjali rathod » Unassigned
Issue summary: View changes
Issue tags: -Novice

I rebased the MR onto 11.x but there are errors to fix.

sokru made their first commit to this issue’s fork.

sokru’s picture

sokru commented
Issue summary: View changes
Status: Needs work » Needs review
Issue tags: -Needs issue summary update

Not the most elegant solution, but does the job.

smustgrave’s picture

smustgrave commented
Status: Needs review » Needs work
sokru’s picture

sokru commented
Status: Needs work » Needs review

All issues should be resolved now.

smustgrave’s picture

smustgrave commented
Status: Needs review » Reviewed & tested by the community
Issue tags: +Needs Review Queue Initiative

Ran test-only feature

1) Drupal\Tests\locale\Functional\LocaleImportFunctionalTest::testStandalonePoFile
Behat\Mink\Exception\ResponseTextException: The text "The file is not encoded with UTF-8" was not found anywhere in the text of the current page.
/builds/issue/drupal-1630568/vendor/behat/mink/src/WebAssert.php:907
/builds/issue/drupal-1630568/vendor/behat/mink/src/WebAssert.php:293
/builds/issue/drupal-1630568/core/tests/Drupal/Tests/WebAssert.php:975
/builds/issue/drupal-1630568/core/modules/locale/tests/src/Functional/LocaleImportFunctionalTest.php:257
FAILURES!
Tests: 6, Assertions: 136, Failures: 1.

Which shows the coverage.

Believe all feedback on the MR has been addressed.

quietone’s picture

quietone commented
Status: Reviewed & tested by the community » Needs work

This is almost there. I left a few comments to look at.

sokru’s picture

sokru commented
Status: Needs work » Needs review

I think other threads are solved, but still open to suggestions if the FileEncoding constraint $encoding should be list of encodings.

quietone’s picture

quietone commented
Issue tags: +Needs tests

Sorry that I missed that this will need tests of the new constraint validator.

sokru’s picture

sokru commented
Issue tags: -Needs tests

I added tests for FileEncodingConstraint. Instead of list I converted $encoding into an array, IMO it makes the code cleaner: no need for implode or preg_* that FileExtensionConstraint is using.

nicxvan’s picture

nicxvan commented

I reviewed this and it looks great.

All of the threads seem to be resolved.

I tried to run the test only job, but it wouldn't run due to:
This job could not start because it could not retrieve the needed artifacts.
Learn more about dependencies and common causes of this error.

The actual tests are passing.

smustgrave’s picture

smustgrave commented
Status: Needs review » Reviewed & tested by the community

Rebased and got test-only to run https://git.drupalcode.org/issue/drupal-1630568/-/jobs/2659732

Re-looking at believe all feedback has been addressed

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented
Status: Reviewed & tested by the community » Fixed

Committed 02b324c and pushed to 11.x. Thanks!

  • alexpott committed a61ad676 on 11.x 
    Issue #1630568 by sokru, Laureatus, quietone, smustgrave, ravi.shankar:...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

quietone closed merge request !2658