Adapt schema.xml and solrconfig.xml for Solr 3.x

No, I haven't given this any thought, as I was completely unaware of this until now. Guess I should keep more in touch with developments in the software I'm integrating with … Thanks for bringing it to my attention!

Therefore, no, I don't know whether this module is compatible. From a quick glance at the release notes, I even suspect that it isn't, as seemingly the HTMLStripWhitespaceTokenizerFactory, which is used in the schema.xml, was removed.
You can definitely expect that to be fixed, though, the newest Solr release should of course be supported.

Regarding using new features I'm more hesitant, though. First off, we probably shouldn't just change requirements now and expect a newer Solr version. And also, I don't really see any major source for possible improvements. As the Search API doesn't support custom location fields, there also doesn't seem a way to take advantage of the new spatial searches. And while range facets for numbers do look interesting, there isn't even a real plan to implement them for dates.

I think it would be pretty hard to support custom types in a generic way. However, supporting this with a contrib module, or with custom modules on specific sites, should be no problem, I think. I don't intend to implement such a module myself, though.

Subscribing.

Ah, sorry, forgot about this.
The attached patch works with Solr 1.4 – does it also work with 3.1?

#6 works for for me on solr 3.1 but with some warnings.

If we specified luceneMatchVersion to at least the version 3 we remove a lot of warning.

One is about solrconfig, org.apache.solr.request.XSLTResponseWriter is remplaced by org.apache.solr.response.XSLTResponseWriter.

We should also specified a queryConverter.

I don't know if we those change can be make without breaking old version.

WARNING: the luceneMatchVersion is not specified, defaulting to LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.4 emulation is deprecated and will be removed in 4.0. This parameter will be mandatory in 4.0.
WARNING: Deprecated syntax found. <highlighting/> should move to <searchComponent/>
WARNING: WhitespaceTokenizerFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0
WARNING: WhitespaceTokenizerFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0
WARNING: LowerCaseFilterFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0
WARNING: WhitespaceTokenizerFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0
WARNING: LowerCaseFilterFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0
WARNING: LowerCaseFilterFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0
WARNING: LowerCaseFilterFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0
WARNING: StandardTokenizerFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0
WARNING: StopFilterFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0
WARNING: LowerCaseFilterFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0
WARNING: LowerCaseFilterFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0
WARNING: no default search field specified in schema.
WARNING: StopFilterFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0
WARNING: org.apache.solr.request.XSLTResponseWriter is deprecated. Please use the corresponding class in org.apache.solr.response
WARNING: No queryConverter defined, using default converter

Thanks for reviewing!
Just post a patch (or the altered file(s)) here and I'll see if they have negative effects on Solr 3.0. (If there are still warnings you couldn't get away, I can also try to address those.)

Subscribing...

But I guess there are still some warnings when starting Solr? Could you maybe try to fix those and then post an updated patch here? It seems gagarine isn't interested in this issue anymore …

Ah, OK, good to know.
Maybe this only happens with 3.1 and 3.2 is more tolerant?

Well, anyways, it's better than nothing, so I just committed this. If there are warnings with 3.1, a) it's still better than before and b) people can re-open this issue.

sub

Sorry I forget about this issue. I assigned the issue to myself, I should not forget about it again.

For a new project I use solr 3.3. It works great but I get some warnings (I filtered out line without warning).

I don't things is possible to make a config file than works for 1.4 and 3.x without using deprecated syntax and features. Of course we can add a luceneMatchVersion to hide warning but I will be more for creating another config for 3.0 than will also work for 4.0.

WARNING: the luceneMatchVersion is not specified, defaulting to LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.4 emulation is deprecated and will be removed in 4.0. This parameter will be mandatory in 4.0.
WARNING: Deprecated syntax found. <highlighting/> should move to <searchComponent/>
WARNING: WhitespaceTokenizerFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0
WARNING: WhitespaceTokenizerFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0
WARNING: LowerCaseFilterFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0
WARNING: WhitespaceTokenizerFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0
WARNING: LowerCaseFilterFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0
WARNING: LowerCaseFilterFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0
WARNING: LowerCaseFilterFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0
WARNING: StandardTokenizerFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0
WARNING: StopFilterFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0
WARNING: LowerCaseFilterFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0
WARNING: LowerCaseFilterFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0
WARNING: no default search field specified in schema.
WARNING: StopFilterFactory is using deprecated LUCENE_24 emulation. You should at some point declare and reindex to at least 3.0, because 2.x emulation is deprecated and will be removed in 4.0
WARNING: org.apache.solr.request.XSLTResponseWriter is deprecated. Please use the corresponding class in org.apache.solr.response
Sep 24, 2011 1:35:49 PM org.apache.solr.response.XSLTResponseWriter init

It asks to reindex the data without the LUCENE_24 emulation because this it's deprecated and going to be removed in feature release.

A patch. I set the luceneMatchVersion to LUCENE_33. Perhaps it will be clever to set to a more compatible LUCENE_3.

With this new solrconfig I pass from a 20ms to 4ms latency and you get certainly lot of other improvement using LUCENE_33 instead of LUCENE_24.

+++ b/solrconfig.xml
@@ -32,6 +32,14 @@
+  <luceneMatchVersion>LUCENE_33</luceneMatchVersion>

This will works for all version

<luceneMatchVersion>LUCENE_CURRENT</luceneMatchVersion>

But as the log say: "WARNING: You should not use LUCENE_CURRENT as luceneMatchVersion property: if you use this setting, and then Solr upgrades to a newer release of Lucene, sizable changes may happen. If precise back compatibility is important then you should instead explicitly specify an actual Lucene version."

@morningtime it works but you should have warnings (mixed with a hundred of other message on starting).

If you don't put luceneMatchVersion you are going to use the lucene 2.4 instead of 3.3. So it will works but you of course lose the benefit to be in 3.3. When you change luceneMatchVersion you have to reindex.

Since we want to add Geofield support in the near future, we'll probably have to add a separate Solr 3 schema.xml anyways. Would that solve the problems here, or at least most of them?
If so, could you please provide a schema.xml (and, if necessary, solrconfig.xml) that work without warnings with Solr 3, gagarine? I guess I'll just commit those, then (and update the INSTALL.txt accordingly, of course).

Voilà

Warning I use

<luceneMatchVersion>LUCENE_CURRENT</luceneMatchVersion>

But the INSTALL.txt should indicate than you have to replace by your actual version to avoid problems on solr update and reindex if you change this value.

No change on schema.xml is require.

+1

But the INSTALL.txt should indicate than you have to replace by your actual version to avoid problems on solr update and reindex if you change this value.

And how can one find out what version is the „actual“ one which should be used?

Also, at least for me, this solrconfig.xml also works with Solr 1.4, without any warnings. Therefore it would seem we don't need to have two solrconfig.xml files after all. Can a few people please test and confirm this? With what versions does the solrconfig.xml in #27 work for you?

sub

The attached config file in #27 seems to work fine Solr 3.4.

I did however get this in the log file:

Nov 21, 2011 12:19:10 PM org.apache.solr.core.Config parseLuceneVersionString
WARNING: You should not use LUCENE_CURRENT as luceneMatchVersion property: if you use this setting, and then Solr upgrades to a newer release of Lucene, sizable changes may happen. If precise back compatibility is important then you should instead explicitly specify an actual Lucene version.
Nov 21, 2011 12:19:11 PM org.apache.solr.core.SolrConfig
INFO: Using Lucene MatchVersion: LUCENE_CURRENT

Yes, that warning is exactly what gagarine means with:

Warning I use
<luceneMatchVersion>LUCENE_CURRENT</luceneMatchVersion>
But the INSTALL.txt should indicate than you have to replace by your actual version to avoid problems on solr update and reindex if you change this value.

Good to hear this works for you too, though. I just need an answer for #29 and I guess we can commit this.

I didn't find a doc for luceneMatchVersion :(. Some valid values are LUCENE_CURRENT, LUCENE_24 (this is the fallback if you don't provide a value, it's an emulation of 2.x and a lot slower), LUCENE_3, LUCENE_31, LUCENE_4 and perhaps other...

Copy past from http://lucene.472066.n3.nabble.com/Solr-1-4-1-compatible-with-Lucene-3-0...

In short: Solr is only compatible to the version it is shipped with (or the other minor patch levels). So Solr 1.4 works with Lucene 2.9, so you must have any Lucene 2.9.x in your classpath.
Lucene 3.0 is a major release of Lucene and had removed all backwards compatibility layers, that are still existent in Lucene 2.9 (functionality is identical in 2.9 and 3.0). Unfortunately, Solr 1.4 still used some of the deprecated code, so it's not compatible.

About one year ago, Lucene and Solr merged to one Apache project and now Lucene and Solr is released together with same version number. The latest Lucene and Solr release in 3.1. But again, Solr should only be used with the version it is shipped with. There was never a separate release of Solr for Lucene 3.0, you can only upgrade or downgrade.

matchVersion parameters have less to do with API compatibility (sometimes it is used also for that to keep backwards), but it's more used to control behavior of some analyzers (means how text tokenization is done to make analysis compatible between different versions).

I think we can marque this as reviewed.

No, we really shouldn't commit this without at least a note in the INSTALL.txt about what to enter there. And maybe a similar note in the file itself – along with a link to some resource showing the appropriate values. Or „The release notes of your Solr version will contain the correct Lucene version to use.“
We should also update the version note in INSTALL.txt to reflect we now support Solr 3.x as well.

Here's the patch for the INSTALL.txt file.
The config at #28 is referenced as solrconfig.3.xml in the INSTALL.txt file.

Damn i'm using Solr 3.3 with Search API Solr search
Indexing sounds ok but seaching with solr admin console fails

I got this error @statup
ATTENTION: no default search field specified in schema.

So a simple search XXXX:8983/solr/select/?q=utinam&start=0&rows=10&indent=on&fl=*,score

Fails, while

XXXX:8983/solr/select/?q=utinam&start=0&rows=10&indent=on&fl=*,score&qf=t_body:value
Works

how can i tell solr to search in all node fields?

+1 for alternate configuration.

Not that it is duplicate but this #1509380: Adopt common solrconfig.xml and schema.xml files for Apache Solr Search Integration and Search API Solr could help all of us. And it solves the 3.x and 1.4.x version problem.

Not that it is duplicate but this #1509380: Adopt common solrconfig.xml and schema.xml files for Apache Solr Search Integration and Search API Solr could help all of us. And it solves the 3.x and 1.4.x version problem.

Obviously changing the schema.xml and solrconfig.xml might require search_api users to re-index, but I agree with Nick here that we have invested a lot of time looking at the 7.x config files in apachesolr to improve them, and having them be common would be good for everyone.

+ for making the configs the same as apachesolr module. Being someone who runs a number of sites that use the apachesolr module currently, this would make it easier for me to try out search_api, since it would require that I set up a separate core.

I can confirm that #27 works for Solr 3.6. While I would love to see unified solr configs, it seems that in the meantime this module should at least provide some sort of out of the box solr 3.x support, because the Solr 1.x binaries have been removed from the Apache mirrors. So, getting this module up and running requires you to either use the patched config above or track down a copy of 1.4 somewhere. In my rough tests 3.6 seems way faster than 1.4, so it seems like a good deal all around to have intermediate support for 3.x while a unified config is worked out.

Just my 2c... Probably not even worth that ;)

#27 works for me too + documentation patch #36 is ok.

<indexDefaults> and <mainIndex> are deprecated and we have to use <indexConfig> instead. Just a heads up.

Be careful, you want to keep your compatibility with Solr < 3.6. Deprecated (at least in the Solr world, they do a good job for backwards compatibility) does not mean it does not work anymore afaik.

The common configs are now done.
Could you please test the latest patch in #1509380: Adopt common solrconfig.xml and schema.xml files for Apache Solr Search Integration and Search API Solr if it works with both 1.4 and 3.5+?

Current version is 3.6.1 warnings

WARNING: You should not use LUCENE_CURRENT as luceneMatchVersion property: if you use this setting, and then Solr upgrades to a newer release of Lucene, sizable changes may happen. If precise back compatibility is important then you should instead explicitly specify an actual Lucene version.
дек 04, 2012 4:17:42 PM org.apache.solr.core.SolrConfig <init>
WARNING: <indexDefaults> and <mainIndex> configuration sections are deprecated (but still work). Please use <indexConfig> instead.
дек 04, 2012 4:17:42 PM org.apache.solr.core.SolrConfig <init>

Maybe better make a conf folder and provide a config files for each major version?

The other issue, #1509380: Adopt common solrconfig.xml and schema.xml files for Apache Solr Search Integration and Search API Solr has been committed.