Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.Quick overview of my use case:
- Site has language-specific (French or English), and language-neutral content.
- Language-specific content must only appear in results when its language is the same as the current language.
- Language-neutral content must be displayed in results regardless of the current language.
- The actual language of language-neutral content can be any of the two languages. We cannot assume it will be all French or all English.
And some thoughts:
- CLIR is of no use in this case, because English content must not appear if the current language is French, and vice versa.
- Apache Solr Multilingual's mapping options are inadequate in this case. If a mapping language is selected, fulltext fields of language-neutral documents are only indexed (and searchable) for that language. If no mapping at all is applied, fulltext fields are simply not getting indexed specifically for any of the site's languages.
So I'm experimenting with the following solution:
- Added a new mapping option: "All languages". With that option, language-neutral documents get enriched with fulltext fields for each of the languages (for example, a document gets indexed with both i18n_content_fr and i18n_content_en).
- Search pages are configured with the "Limit search to current language by default" and "Show language-neutral/undefined results by default" options. Queries will only search fields matching the current language, so having duplicate fields for other languages has no impact on search results.
The attached patch is an attempt at this solution. Opinions on this approach?
Of course, users searching in French will get good search results with language-neutral documents that contain French text (because that text will have gone through a French index analyzer), and so-so results with English text (because it will have gone through the same French analyzer). However, I guess it is not too much of a surprise to have to switch the current language to English for optimal English text search, and to switch it to French for optimal French text search.
| Comment | File | Size | Author |
|---|---|---|---|
| #6 | 2066405.patch | 4.57 KB | mkalkbrenner |
| #2 | apachesolr-multilingual-map-all-2066405-2.patch | 6.75 KB | mvc |
Comments
Comment #1
mkalkbrennerThat's a valid feature request. I'll review your patch ...
Comment #2
mvcsame patch, re-rolled against tag 7.x-1.0-rc1, in -p1 format for drush make.
Comment #3
mkalkbrennerThe patch already looks good for "common fields". But you missed the callbacks for node, user, entity and term references.
But in general I think that the original requirements stated in that issue can already be achieved without a patch:
Enabling "Limit search to current language by default" and "Show language-neutral/undefined results by default" of the Multilingual Query Settings on the corresponding Search Page Settings should do the job. If not, we have to look for a bug.
Comment #4
David Lesieur CreditAttribution: David Lesieur commentedIf I remember correctly, the patch ensures that language-neutral content gets indexed (or one could say duplicated) in language-specific fields for all of the languages. That allows the content to get through every language's analyzers (e.g. indexing in both 'i18n_content_fr' and 'i18n_content_en', instead of just 'content'). None of the current mapping options does that.
In my use case, language-neutral content is not consistently in a single default language. It can be in any of the languages enabled on the site. So with the new mapping option, better search results are obtained when the current language matches the actual language of language-neutral content.
Comment #5
mkalkbrennerOK, I got you. For this use case additional searching through the language neutral fields at query time is not sufficient. (But you need to know that you increase the false positives as well.)
No, only for fields that are not references. Term references needs to be handled because terms might be translated. The other references are simply missing in the "duplicates". This might cause various errors during searches. Therefor the patch needs work.
Comment #6
mkalkbrennerI modified (and simplified) the code to handle the callbacks as well.
Can you please test my patch against the latest dev?
Comment #7
mkalkbrenner