field_apachesolr_field_mappings() doesn't provide mappings for fields of type text, text_long, or text_with_summary. This doesn't affect standard fulltext search, but it means that these fields aren't available as facets via FacetAPI or as Views fields or filters via Apache Solr Views.

Comments

nick_vh’s picture

Status: Closed (works as designed) » Active

This is expected behavior because text processing takes a very long time and this would not be optimal for most of the installs

An easy code snippet solves this for your site :

/*
 * Implements hook_apachesolr_field_mappings_alter().
 */
function MYMODULE_apachesolr_field_mappings_alter(&$mappings) {
  $mappings['text'] = array(
    'indexing_callback' => 'apachesolr_fields_default_indexing_callback',
    'map callback' => '',
    'index_type' => 'string', // Do not set these to text because a facet and text processing do not make sense
    'facets' => TRUE,
    'facet missing allowed' => TRUE,
    'dependency plugins' => array('bundle', 'role'),
    'hierarchy callback' => FALSE,
    'name_callback' => '',
    'facet mincount allowed' => FALSE,
    // Field API allows any field to be multi-valued.
    // If we set this to false we are able to sort
    'multiple' => FALSE,
  );
}
nick_vh’s picture

Status: Active » Closed (works as designed)
becw’s picture

Status: Active » Closed (works as designed)

I think that makes sense for fields of type text_long fields and text_with_summary, which if they are stored should definitely use 'index_type' => 'text', but the text field type is 255 characters or less, and storing it as 'string' means that Apache Solr does not do any processing of the values.

Anyway, thanks for explaining the reasoning! Implementing hook_apachesolr_field_mappings_alter() does solve my problem :)

becw’s picture

Is there a way to index a single Drupal field two different ways in Solr? For example, as type 'text' for fulltext search in a single field, and as type 'string' for facets and sorting?

darren oh’s picture

Category: bug » feature
Status: Closed (works as designed) » Needs review
StatusFileSize
new836 bytes

It makes no sense to not have mapping for basic text fields. Keeping it out means unnecessary complexity and duplication of effort.

twigley’s picture

I'm having some problems with the fix Darren Oh has provided above.
Adding these changes did mean the text fields are indexed (I can see them in the search index report, listed with index type "string").
However, they don't appear as Facets within the facet settings.
Do I need to do anything else to make them appear as facets other than set facets to true as above?
Any help would be really appreciated.

twigley’s picture

All fixed. It's a very large data-set, they came through automatically after a couple of minutes.

nick_vh’s picture

Text fields should not be mapped and certainly not be used for facets. The reason for this is due to the inner workings of solr. You should map the fields as if they were strings, not text.

Strings (ss_*)
Literal representations of values, for example balloons=balloons

text (ts_*)
Parsed representations of values that go through analyzers. eg. (query) balloons = balloon.

If you map them, map them to strings, and if you want text analysis, be sure to know the implications before you map them to text fields. But, if we do this automatically, such as the patch implies, we risk getting massive strings. You surely don't want a text blob of 300+ characters to be a literal string that you facet on?

I think we need to find a more pragmatic solution here instead of mapping all text fields as strings.

darren oh’s picture

My patch only affects the text field type, which is limited to 255 characters. It makes it possible to index the text field type, but does not automatically do so. People will still need to enable indexing for the specific fields they wish to use as facets. It is reasonable to expect that people will index specific fields because they really want to.

jordiserra’s picture

Just FYI,

Search API allows you to index text fields by default, and facets work like a charm. Fulltext fields does not work, but this is coherent with this post.

nick_vh’s picture

Status: Needs review » Reviewed & tested by the community

After much thinking this actually makes sense and if we can restrict it to that small text field setting it seems like a reasonable feature request and as such I will add it to the dev version.

nick_vh’s picture

Version: 7.x-1.x-dev » 6.x-3.x-dev
Status: Reviewed & tested by the community » Patch (to be ported)

We should port this to 6.x-3.x. Committed for 7.x-1.x

mkalkbrenner’s picture

Assigned: Unassigned » mkalkbrenner
Issue summary: View changes
Status: Patch (to be ported) » Needs review
StatusFileSize
new712 bytes

From my point of view that's already implemented in 6.x-3.x in combination with content.module:

/**
 * Implements hook_apachesolr_field_mappings() on behalf of content module.
 */
function content_apachesolr_field_mappings() {
  $mappings = array(
    // ...
    'text' => array(
      'indexing_callback' => 'apachesolr_index_content_text_indexing_callback',
      'index_type' => 'string',
      'facets' => TRUE,
      'query types' => array('term'),
      'query type' => 'term',
      'facet mincount allowed' => TRUE,
    ),
    // ...
  );
  return $mappings;
}

But in opposite to 7.x-1.x the text is not handled by apachesolr_clean_text(). I attached a patch to do that.

  • Commit 0c9c39a on 6.x-3.x by mkalkbrenner:
    Issue #1433366 by mkalkbrenner: apply apachesolr_clean_text() on text...
mkalkbrenner’s picture

Status: Needs review » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.