Hello, I am wondering why not all the different stemmer modules in Drupal (11 - http://drupal.org/search/apachesolr_multisitesearch/stemmer?filters=ss_m...) are not working together to create an international stemmer. The main part of all the code is based on variations of the porter stemmer and not really drupal related.

With all those coding mantainers it would be far easier to create a standard module that could include a system to add languages and focus on its integration with international sites. Also this module is useful not only for searching but for automated tasks like linking with http://drupal.org/project/linkintel

I'm going to send issues to all the other stemmer modules in case anyone is interested. I think it would be a very benefitial step for everyone and specially all the other languages that are not yet "stemmed".

Thanks

Comments

jhodgdon’s picture

jhodgdon
she/her
Primary language English
commented
Status: Active » Closed (won't fix)

The amount of code that would be common to all the stemming modules is really very minimal. Nearly all the code in the Porter Stemmer project is specific to English-language stemming rules, and there are only a few lines that are the interface with Drupal.

So while I applaud the thought, I don't think it's all that useful to build an international stemming project.

jhodgdon’s picture

jhodgdon
she/her
Primary language English
commented
Status: Closed (won't fix) » Closed (duplicate)

Furthermore, this has been proposed before:
#839868: Stemmer API

daniorama’s picture

daniorama commented

Mmm, I couldn't find it in my searches because I looked for "international" :D

Well, the thing is that although the common code is very minimal right now, it could be other way. The German stemmer for example have really nice additions and the problem with multilingual sites is still on.

Also, other modules that use stemmer for automatic tasks would need to just add the compatibility with a main one, and not to make people create multiple modules for every language and do the same.

I really think there is a lot of points in favor to think about an international stemmer.