Document that language is not set on search keyword preprocessing

First we need to establish that this is actually a problem.

So, we have this function search_simplify() in search.module, which takes a $langcode argument, and that is what eventually calls hook_search_preprocess with the $langcode argument ($langcode is not currently otherwise used in search_simplify()). The idea is that both during indexing and during search keyword submit, you should do the same preprocessing (called "simplify" for historical reasons) so that the search will find matches.

So, during indexing: NodeSearch plugin finds node translations and calls search_index(), passing in the language, for each one. This calls search_index_split(), passing $langcode, and that calls search_simplify(), passing in $langcode. So far, so good. #1964428: Search module should keep language information on snippet content. may modify this slightly.

During searching: SearchController gets the submitted keywords, and calls the setSearch() method on the active search plugin. The base method (not overridden by NodeSearch plugin) saves the keywords and request attributes on the plugin, which might or might not include a language? Not sure.

Then during execute() on the plugin, NodeSearch actually does the work of searching in its findResults() method. It passes the keywords to the SearchQuery extender class by calling its searchExpression() method. Only the submitted keyword string is passed in there.

SearchQuery::parseSearchExpression() is what parses this. But it just breaks things into words and calls search_simplify() without a $langcode parameter, since it has no idea about the language.

So. This does seem to be a problem. Updating summary. Next will think about how to fix it properly.

It seems to me there are two possible ways to solve this:

a) Documentation: Document in hook_search_preprocess() that if $langcode is NULL, this is user input in an unknown language, and let modules decide what to do. They can presumably (??how??) figure out some default langcode to use and act accordingly.

b) Change SearchQuery::setSearch() to take a $langcode argument, and have it pass this into search_simplify(). Then we'd have to figure out what language NodeSearch should give it, or maybe have SearchController make the decision. Based on what? I'm not sure, but the same question as "act accordingly" in (a) except we would have the Search system make the decision instead of leaving it up to the individual preprocessing module.

As a note, it is only hook_search_preprocess that is language-dependent. The other parts of search_simplify(), at least so far, are not.

Yes, the only use is in the hook implementations.

I think I agree with this. (a) is the thing to do -- let the hook implementations decide.

In which case, I think all we need to do here is improve the documentation. I can make a quick patch. I noticed there is not much stated in the current docs on any of these that says that $langcode could be null or what to do about it.

Agreed, docs! I will make a patch.

OK, here's a patch that fixes up the docs headers for all the functions/hooks in search.module and search.api.php that have $langcode parameters. I think it makes it clearer when/why this would be unknown in the hook. Review welcome!

Good ideas. So yeah, the search will not find results if the same preprocessing is not applied to both keywords and indexed text. Reworded slightly. And added sample code.