Resolve differences in the <indexDefaults> directive

We increased maxFieldLength because large docs, e.g. extracted PDFs, were truncated. Possibly it should be even larger.

Lucene in Action suggests making this unlimited, but I can't remember if Solr passes through that setting.

maxMergeDocs being set is likely carried forward from an old config.

Because we are setting the lucene match, we need to also set the mergePolicy to restore that older default - Nick_vh did load testing and the default Solr 3.5 policy gave slower searches.

Here's the comment from the example config:

The Merge Policy in Lucene controls how merging is handled by
Lucene. The default in Solr 3.3 is TieredMergePolicy.

The default in 2.3 was the LogByteSizeMergePolicy,
previous versions used LogDocMergePolic

The apachesolr module's config seems fine here, I'd readily adopt that in the Search API. An unlimited setting for maxFieldLength would probably be great, too (unless there are general disadvantages besides possible memory usage). Maybe someone could test or research that?

I should remember always to refresh beofre commenting …

Some discussion here : http://grokbase.com/t/lucene/solr-user/10by60t7jf/how-to-set-maxfieldlen... + http://osdir.com/ml/solr-user.lucene.apache.org/2011-11/msg00573.html + http://www.gossamer-threads.com/lists/lucene/general/140440

It is actually deprecated (http://lucene.apache.org/core/3_6_0/api/all/org/apache/lucene/index/Inde... ) and should not be used. We should use LimitTokenCountAnalyzer (http://lucene.apache.org/core/3_6_0/api/all/org/apache/lucene/analysis/L...)

Not sure how to implement LimitTokenCountAnalyzer in the config but I think the path forward is to throw this value out the config and test if is cut off

Well, we want to have a config that works reasonably for 1.4 still and more or less the same as 3.5.

Also, not clear that 3.5.0 supports LimitTokenCountAnalyzer.

It doesn't seem like there is an "unlimited" setting that Solr accepts, but we could e.g. increase to 100000 or more from 20000.