Language information where available may be an important criterion when applying filters.

For example, the wordfilter module allows filtering of specified words or phrases in content. #265565: Multilanguage support (at least for 6.x) is a feature request for adding language-specific word replacement, but this is not currently feasible because filters are not language-aware.

Attached is a small patch adding a $language argument to check_markup(). Modules implementing hook_filter() will receive a $language argument and may use this information to choose filter behaviour.

Comments

moshe weitzman’s picture

We cache filtered string by md5(text). do we need to add language to the cache key?

nedjo’s picture

@moshe weitzman: Yes, I think you're right.

In url() and l() we can pass a language in the $options array, but in that case it's a language object and not a string. Maybe for consistency we should be passing an object here as well?

stella’s picture

StatusFileSize
new5.18 KB

Here's an updated patch that includes the language string in the cache key.

Cheers,
Stella

nedjo’s picture

Thanks Stella!

A minor point. I'm thinking we should use $langcode instead of $language for the variable name, for consistency in the rest of core where $language tends to be a language object and $langcode the code, as used here.

stella’s picture

Status: Needs work » Needs review
StatusFileSize
new5.19 KB
new1.03 KB

Here's an updated patch which uses $langcode instead. I've also attached a patch to update the hook_filter() api documentation.

dries’s picture

Status: Needs review » Needs work

It would be good if you could describe in the phpdoc (i) the format of the language code and more importantly (ii) why the language code matters and what it could/would be used for.

I'm also wondering if this affects any of our tests -- it should.

stella’s picture

Status: Needs review » Needs work

I'm using the latest version of Drupal from HEAD. I ran all of the simple tests before I made the change and again afterwards. No additional simple tests failed because of the application of the patch.

Will take a look at improving the php documentation.

Cheers,
Stella

stella’s picture

Status: Needs work » Needs review
StatusFileSize
new1.17 KB
new5.32 KB

Updated patch with improved function documentation.

I was wondering though, if no language code is provided to check_markup(), should check_markup() provide the code of the current language to hook_filter() or should it be left to the hook_filter() to determine the current language code if it requires it? Currently, the patch assumes the latter.

Cheers,
Stella

moshe weitzman’s picture

I know that I got us started down this road, but I really was asking if the langcode is needed as a cache key. If the md5($text) is a perfect match, can't we be sure that the cached text is good and we need not run filters?

dries’s picture

The documentation improvements look good. I'm still curious what language-specific filter you're working on.

Let's wait for the reply to Moshe's question in #10 too.

nedjo’s picture

@dries: The need for passing a $langcode to filters came up in #265565: Multilanguage support (at least for 6.x), a feature request on the Word filter module.

That module enables filtering of specified words. For example, all occurrences of "dime" might be replaced with "quarter".

The issue is that many words have the same form in different languages. (Words that are spelled the same in different languages are technically called "interlingual homographs"). E.g., "dime" is "ten cent coin" in English but is also a word in Spanish (it means "tell me").

Whether or not the meaning is the same in different languages, the correct replacement likely is different. To know whether or not "dime" should be replaced with "ten cent coin" using the Word filter module, we need to know what language we're working with.

[But maybe, given the history of dorpje > drop > druppel > Drupal we want to be careful about how much we fix the problem of interlingual similarities--sometimes they prove to be a spur to creativity :) ]

@moshe: The theoretical use case for including the $langcode in the cache ID would be that the full string being processed is identical in more than one language, but should be handled differently. This is definitely an edge case, but I guess it would occur if we had a single-word string, or else some unusual string that is mostly language neutral (a company name, language, etc.) but includes an "interlingual homograph". So, I guess, we're safer including the $langcode.

@stella: re whether we should feed the current language, my initial thought is, it may provide more flexibility and therefor be better to leave that to the filter implementing the hook, as the current patch does. That way, a filter can decide how to respond specifically to an unknown language (e.g., by skipping filtering).

moshe weitzman’s picture

wow - good answer. i'll support a langcode in the key.

sun’s picture

+1 for this change.

First thought:

I'd like to propose (ugh, slightly scope creep) to revert to "language", but don't do:

check_markup($block->body, $block->format, '', FALSE);
check_markup($comment->comment, $comment->format, '', FALSE);
check_markup($node->body, $node->format, $node->language, FALSE);

Do this instead:

check_markup($block, FALSE);
check_markup($comment, FALSE); // Change $comment->comment into $comment->body!
check_markup($node, FALSE);
nedjo’s picture

@sun: I agree that having better parallelism between our different object types would be good, but, yes, it's beyond the scope for this issue.

And there are a couple of reasons that this wouldn't work even if we did have parallel naming.

First, it may not be an object at all that's being filtered--check_markup is used in many different contexts in contrib.

Second, even if it is an object, a given object type may have more than one field that needs filtering, so we can't assume it's a body that we're filtering. Comment is an example:


      $comment->signature = check_markup($comment->signature, $comment->format);

dries’s picture

Status: Needs review » Needs work

@nedjo: thanks for the clarification -- that is really helpful. It seems like a fairly special case, but I do support the change nonetheless. It is a valid use case after all.

I've run the tests, and this looks like it is ready to be committed. I committed it to CVS HEAD. Two things left to do before this issue can be marked fixed:

* We have to commit the documentation update.
* We have to update the module upgrade instructions in the handbook.

Thanks!

stella’s picture

Status: Needs work » Needs review

I've committed the API documentation update.

Suggestion for module upgrade instructions:

Parameters to hook_filter() have changed
hook_filter() now accepts an additional parameter, $langcode, which allows filters to be language aware. This means modules can now implement language specific text replacements.

Drupal 6.x:

<?php
function my_module_filter($op, $delta = 0, $format = -1, $text = '', $cache_id = 0) {
  // Apply filter...
}
?>

Drupal 7.x:

<?php
function my_module_filter($op, $delta = 0, $format = -1, $text = '', $langcode = '', $cache_id = 0) {
  // Apply filter...
}
?>

Parameters to check_markup() have changed
check_markup() now accepts an additional parameter, $langcode. This is needed for filters which implement language specific text replacements.

Drupal 6.x:

$node->body = check_markup($node->body, $node->format);

Drupal 7.x:

$node->body = check_markup($node->body, $node->format, $node->language);
stella’s picture

Status: Needs review » Fixed

Module upgrade instructions updated on http://drupal.org/node/224333

Cheers,
Stella

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.