Multilingual Search

About

The goal of this project is to cleanly extend Apache Solr Search Integration to provide:

  • better support for non-English languages
  • support for multilingual search
  • cross-language information retrieval (CLIR)
  • an easy-to-use administration interface

Apache Solr Multilingual does not replace Apache Solr Search Integration but is based on it.

Details

Motivation

Apache Solr Search Integration provides a (more or less) easy way to use Apache Solr as a powerful search engine for Drupal. Unfortunately, the only language that works well with it, out-of-the-box, is English.

So if you run a non-English website, you need to tweak all the configuration files by hand or you lose some of the advantages that Solr provides compared to Drupal's built-in database-driven search. Doing so requires a deep knowledge of Solr and search technology in general.

The entire process gets even more complicated if you run a multilingual website.

That is why we started thinking about an additional module called Apache Solr Multilingual, to hide most of the complexity from a Drupal website's administrator.

Language-Specific Problems

  • Stop Words
    Words you want to exclude from your search index are called stop words. The list of words strongly depends on the focus of your website and, of course, on your site's language.
  • Stemming
    Every word in the search index is stored in a reduced form called a word stem. This strategy enables the user to find content, independent of the key word's inflection, e.g. singular or plural. Unfortunately, the stemming algorithm is different from language to language.
  • Protected Words
    In some cases, you'll want to exclude certain words from the stemming described above. These protected words are language-specific, like stop words.
  • Compound Word Splitting
    Languages like German frequently combine words (e.g. "Dampfschifffahrt"). In order to deal with that problem you need to split such words into parts depending on language-specific word catalogs.
  • Spell Checking
    No doubt that spell checking should be language-specific.

Apache Solr Multilingual

Apache Solr Multilingual provides an out-of-the-box solution to language-specific problems (described above) or supports the site administrator by providing a user interface that hides some of the complexity.

Additionally, Apache Solr Multilingual provides a way to offer language-specific searches for different languages at once on multilingual websites. Therefore, Apache Solr Multilingual integrates with Drupal's standard multilingual features provided by core modules and the Internationalization and the Entity Translation modules.

As a special feature, Apache Solr Multilingual can be configured to deal with the translations of nodes and taxonomies on multilingual sites. That means that you can find content in any language, no matter which language was used to enter the search phrase. It's currently a simple implementation of CLIR, but our plan is to extend this feature.

Drupal 7

Apache Solr Multilingual 7.x-1.0 is currently in beta state. The feature list is mostly complete for European languages. Different languages are on the road map. Track the #1702354: Apache Solr Multilingual 7.x Roadmap to keep up to date with the development.

Apache Solr Multilingual depends on or uses some other modules:

Apache Solr Multilingual supports content translation and entity translation and therefore interacts with these modules if available:

If you're interested in helping us, there are different ways. Beside code contributions we need native speakers for each language as testers and to help us defining default settings for different languages.

Drupal 6

The only combination of Apache Solr Search Integration and
Apache Solr Multilingual known to work are the 6.x-2.x versions of both modules!

Even if the 6.x-2.x branch is not listed at the project page, you can download apachesolr 6.x-2.0-beta5

Unfortunately the maintainers of Apache Solr Search Integration decided to discontinue the 6.x-2.x branch, but to enhance the 6.x-1.x branch and to start a new 6.x-3.x branch to backport the 7.x-1.x features. Depending on the effort and the interest of potential sponsors we will provide an implementation based on 6.x-3.x.

Sponsors

  • bio.logis offers a comprehensive spectrum of genetic analyses.
  • pgsbox.com - Personal Genomics Services

Additional Resources

Project Information

Downloads