Apache Solr Multilingual

About

The goal of this project is to cleanly extend Apache Solr Search Integration to provide:

• better support for non-English languages
• support for multilingual search
• cross-language information retrieval (CLIR)
• an easy-to-use administration interface

Apache Solr Multilingual does not replace Apache Solr Search Integration but is based on it.

Details

Motivation

Apache Solr Search Integration provides a (more or less) easy way to use Apache Solr as a powerful search engine for Drupal. Unfortunately, the only language that works well with it, out-of-the-box, is English.

So if you run a non-English website, you need to tweak all the configuration files by hand or you lose some of the advantages that Solr provides compared to Drupal's built-in database-driven search. Doing so requires a deep knowledge of Solr and search technology in general.

The entire process gets even more complicated if you run a multilingual website.

That is why we started thinking about an additional module called Apache Solr Multilingual, to hide most of the complexity from a Drupal website's administrator.

Language-Specific Problems

• Stop Words
  Words you want to exclude from your search index are called stop words. The list of words strongly depends on the focus of your website and, of course, on your site's language.
• Stemming
  Every word in the search index is stored in a reduced form called a word stem. This strategy enables the user to find content, independent of the key word's inflection, e.g. singular or plural. Unfortunately, the stemming algorithm is different from language to language.
• Protected Words
  In some cases, you'll want to exclude certain words from the stemming described above. These protected words are language-specific, like stop words.
• Compound Word Splitting
  Languages like German frequently combine words (e.g. "Dampfschifffahrt"). In order to deal with that problem you need to split such words into parts depending on language-specific word catalogs.
• Spell Checking
  No doubt that spell checking should be language-specific.

Apache Solr Multilingual

Apache Solr Multilingual provides an out-of-the-box solution to language-specific problems (described above) or supports the site administrator by providing a user interface that hides some of the complexity.

Additionally, Apache Solr Multilingual provides a way to offer language-specific searches for different languages at once on multilingual websites. Therefore, Apache Solr Multilingual integrates with Drupal's standard multilingual features provided by core modules and the Internationalization module.

As a special feature, Apache Solr Multilingual can be configured to deal with the translations of nodes and taxonomies on multilingual sites. That means that you can find content in any language, no matter which language was used to enter the search phrase. It's currently a simple implementation of CLIR, but our plan is to extend this feature.

Current State

Apache Solr Multilingual 6.x-2.x requires Apache Solr 6.x-2.x, which is currently in beta.
Apache Solr Multilingual 6.x-1.x is not well supported anymore (unless somebody has time to backport the features of 6.x-2.x).
Apache Solr Multilingual 7.x-1.x will not be released until the major multilingual features in Drupal 7 Core are fixed.

Sponsors

• Cocomore AG - For Drupal-related services, visit drupal.cocomore.com
• bio.logis GmbH

Additional Resources

• Sep 2011: Language-Specific and Multilingual Full-Text Searching at Drupal City Berlin (English slides).
• Feb 2011: Non-English and Multilingual Search with Apache Solr Multilingual at Drupal Dev Days Brussels (English slides).
• May 2010: Apache Solr Multilingual session at Drupal Dev Days Munich (German slides).

Downloads