Treat cardinality=1 fields as single-value + caching stats response

I can increase the cardinality of a field in the UI at any time, right? So just as the SQL schema supports multiple values for all fields, so should Solr.

I agree that's a little wacky for booleans, but seems to be what Drupal core provides.

What's the use case where this is causing a problem for you?

I actually got into this problem when I was trying to make the slider. In order to get stats to return from solr for a specific field it has to be a single. This means that one should programmatically modify the facets to make them "multiple = false" to make this stats property work. Any integer field was a multiple and I had to make sure it wasn't doing setting this field as a multiple.

This is indeed a very tricky problem and you can't change the field in Solr as easily as the drupal field. It might make sense to see if there is only 1 value in the UI and add both a single and a multiple to the index? (Overhead problem...)

After some discussion we probably need to add a singular field for any numeric type AND for the date. The module will only take the first value from the multi valued field and any component that is using stats or sorting can then use this until Solr solves the problems it has with statistics and sorting on multivalued fields

This patch fixes the cardinality problem somehow. When doing stats queries you can rely on this functionality that there is always a singular field as well.
Attention : Only valid for integer fields

A typo + leftover variable. This patch also fixes a small issue in hook_entity_update

While we are at it, we need to fix the query used by the term component and might want to handle caching that result

The following patch handles very basic/limited caching (we might want to use cache_set and cache_get?) + it will use the singular field for any multivalued integer/float indexed value

Cache_get and cache_set seem to work way better since it is not run on every page load. I feel this patch is ready for review

Whitespace problem

Committed this patch. Reindexing is needed, since this will add new fields to your solr index

Why is the function moved to the .inc file? I think you may end up including it on every page load now.

Follow up

Committed the follow up

Commited #12 as backport patch

Committed #15 to 6.x-3.x

I have an other use case for adding single-valued indexes for fields with cardinality=1: sorting. Solr appears to be unable to sort on multivalued fields (correct me if I am wrong), but in my use-case I want to sort on custom string fields that come from a predefined list of strings.

I suppose, then, that my question is: why only add these single-valued fields for numeric and date fields, only?

Nothing prevents you from doing a mapping_alter and also define singularity for specific text fields?

Fair enough, but I think it would be expected behavior to make single-valued indexes for single-valued fields, no? This would take general users a lot of effort by default.

The problem (if you have read this thread) is that in Drupal a single value field can easily be transformed into a multi value field. There is no way we can do that with solr so we decided to put all of them by default in a multi field and the first value of the field also in a single solr field.

I think for the 90% usecase this is more than sufficient so I'm a bit reluctant to open up new options and allow people to break things. text fields and facets are not a good combination because they have been processed/tokenized and do not serve facets very well.

Yes, I have read this thread, and I understand the problem. But I still don't know why it's been decided that only numeric and date fields have an extra single field.

By default, a string field is not tokenized:

  <fieldType name="string" class="solr.StrField" sortMissingLast="true" omitNorms="true"/>

And when a string field comes from a list or predefined strings, it facets rather well, I think.

I understand your reluctance, but if we do it well, it will not break things. At most, it will add some extra indexes to Solr. I will look into this to see if I can find a more generic solution. I understand the way Drupal stores fields internally, and that it makes sense to translate this storage model to Solr, but I think the Solr index should reflect the widget settings, rather than the field data storage settings, as it has direct implications on the way the facet search works.

Same problem here.

What I'm personally doing is overriding the default mappers and have my own cardinality check, then set the appropriate single or multiple solr index.

For me, if a field is single in the admin, it should be single in Solr.

Only single fields can be used for sorts, which is important for what I'm doing. Also I would assume they are more performant in general.

Here's my snippet I'm using in the indexing functions to set the appropriate index.

function yt_apachesolr_check_cardinality($index_key, $field_info) {
  // if the cardinality of the field is 1, store in solr as single field.  
  if (isset($field_info['field']['cardinality']) && $field_info['field']['cardinality'] == 1) {
    $singular_field_info = $field_info;
    $singular_field_info['multiple'] = FALSE;
    $index_key = apachesolr_index_key($singular_field_info);
  }
  return $index_key;
}