Problem/Motivation

locale_string_is_safe() is used to check that a string is safe to be added or imported as a translation. It check if html tags contained in the string are white listed or not. It uses Xss::filter which returns well-formed HTML entities. Due to this HTML correction, non-well formed html tags are refused by locale_string_is_safe(). If a translated string contains, for example <br/>, the string is rejected. While strings with <br /> are accepted.

Proposed resolution

Make locale_string_is_safe() ignore white space in html tags.

Remaining tasks

t.b.d.

User interface changes

none

API changes

none

Data model changes

none

Comments

gábor hojtsy’s picture

Project: Drupal.org site moderators » Localization client
Version: » 6.x-1.x-dev
Component: Localize.drupal.org » Code

http://api.drupal.org/api/function/locale_string_is_safe/6 is the function used to check these strings. br is an allowed tag, so not sure why this would be a problem. Can you post the whole string? Moving to l10n_client.

gábor hojtsy’s picture

Status: Active » Postponed (maintainer needs more info)

Doh, you did provide the link to the string. It is:

To install a new module or theme, either enter the URL of an archive file you wish to install, or upload the archive file that you have downloaded. You can find <a href="@module_url">modules</a> and <a href="@theme_url">themes</a> at <a href="@drupal_org_url">http://drupal.org</a>.<br/>The following archive extensions are supported: %extensions.

So I'm thinking it might be that a space is missing before the slash. However, http://api.drupal.org/api/function/_filter_xss_split/6 does seem to allow not having a space there. Interesting. Can you try with a space before the slash?

palik’s picture

Version: 6.x-1.x-dev » 7.x-1.0-beta1

Well, i played with this and it seems that "<br/>" is not always treated the same way,

1) when i set l10nclient to send strings to LDO it goes to l10n server properly, but in locale database on my site it is rejected (not saved);

2) on LDO, after i accept string with <br/> and export .po file and try to import it to my site, I get "One translation string was skipped because it contains disallowed HTML." error.

3) If i then change from <br/> to <br /> in .po file everything goes smoothly and strong gets properly saved into db.

sorry form my english i tried to write this as clearly as i could :)

PS. we (polish team) have more than 94% of D7 translated :)

gábor hojtsy’s picture

Title: problem with ID#572009 string - BR tag prohibited? » filter_xss() will not allow self-closing tags without whitespace
Project: Localization client » Drupal core
Version: 7.x-1.0-beta1 » 7.x-dev
Component: Code » base system
Priority: Minor » Normal
Status: Postponed (maintainer needs more info) » Active

Well, then looks like we should fix the XSS checking in core to allow for the space to be omitted (as it is possible to be omitted as per the standard). Moving to the core queue then.

meba’s picture

I somehow agree. I also had issues with other tags not allowed (like div). Are we able to dig up a specific reason why we introduced this test?

gábor hojtsy’s picture

@meba: In locale? If there is no XSS check, translators can introduce XSS on your site via translations (which are often treated as trusted data and sometimes contain HTML markup themselves).

chx’s picture

So what happens is that filter_xss converts <br/> to <br /> and so the string does not match up. You want to mess with the "Is there a closing XHTML slash at the end of the attributes?" section. Mess with it at your own peril, though.

spiffl’s picture

Status: Active » Needs review
StatusFileSize
new1.19 KB

The attached patch-file modifies the function locale_string_is_safe() by
first comparing as before the patch, i.e. the resulting strings must be equal after filter_xss.

If this fails, both strings are run through a remove-whitespace-regex and are then compared again.

thedavidmeister’s picture

Version: 7.x-dev » 8.0.x-dev
Issue summary: View changes
Status: Needs review » Needs work
Issue tags: +needs test

The behaviour appears to be identical in d8, so bumping version number.

This type of patch will definitely need tests before it could be considered for production.

sutharsan’s picture

Issue tags: +Needs reroll
sutharsan’s picture

Issue tags: -needs test +Needs issue summary update
StatusFileSize
new1.95 KB

These are tests that prove that the problem still exists in D8.
This patch does NOT include the re-roll of #8.

babruix’s picture

Issue tags: -Needs reroll
sutharsan’s picture

Status: Needs work » Needs review
StatusFileSize
new3.35 KB
new1.57 KB

This patch adds the reroll of #8.

Status: Needs review » Needs work

The last submitted patch, 13: drupal-xss-white-space-953582-13.patch, failed testing.

sutharsan’s picture

Status: Needs work » Needs review
StatusFileSize
new1.67 KB
new3.33 KB

Fixing tests.

I matched the tests with the code. But not 100% sure if we should cater for tabs and spaces, or only spaces.

sutharsan’s picture

Component: base system » language system
Issue summary: View changes

Updated issue summary.

sutharsan’s picture

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.2.x-dev

Drupal 8 is end-of-life as of November 17, 2021. There will not be further changes made to Drupal 8. Bugfixes are now made to the 9.3.x and higher branches only. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.15 was released on June 1st, 2022 and is the final full bugfix release for the Drupal 9.3.x series. Drupal 9.3.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.4.x-dev branch from now on, and new development or disruptive changes should be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

acbramley’s picture

Version: 9.4.x-dev » 9.5.x-dev
StatusFileSize
new1.33 KB

Re-wrote the Xss::filter test coverage and added locale_string_is_safe coverage. Hopefully this will demonstrate where the issue lies in that Xss::filter returns markup with the whitespace added and therefore the comparison fails.

acbramley’s picture

Issue tags: +Bug Smash Initiative

Status: Needs review » Needs work

The last submitted patch, 29: 953582-29-test-only.patch, failed testing. View results

acbramley’s picture

Title: filter_xss() will not allow self-closing tags without whitespace » locale_string_is_safe() will not allow self-closing tags without whitespace

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.