How to define addtional fields in schema.xml?

Hi Anbu,

schema.xml has certain fields defined which are always there in drupal (tid, uid, modified, created, etc). In addition, there are "dynamic" fields. Solr lets you add fields to your index (database) while indexing them! Which means, you can just add fields to the XML sent to solr, and they become facets.

It's not so simple though, because they must be in a certain format / name, etc. This module will find all the cck fields in your database, and they will be added as facets (which you can filter on).

So just add some CCK fields to your nodes, and they will show up as apache solr blocks on the block page.

Any chance your module can put that information in a CCK field?

I guess there *could* be a hook in apachesolr to enable other modules to put in extra fields...? This is how the faceted_search module works.

I foresee having (in Drupal8?) all sorts of hooks for search.. for plugging in search engines, handling facets, results cache, attachment indexing, and custom content like this case. Perhaps even having several search engines simultaneously? =) Oh and whilst I'm dreaming... Views will format search results as we wish =)

I'll put in a vote for a hook to allow other modules to add fields to the indexed document.

Although I think a better solution would be for a more general drupal implementation with 2 hooks:

• first, collect information into a keyed array of fields (like, body, text, etc, );
• second, actually index that data

There are hooks to do that already. Though I think there are some issues with it.

apachesolr.module only does text and options widgets. The reason I think is misleading cause you can have an integer and options widgets but the view of that is text. So why not just open it up to use all options widgets.

Anyway, here's what I did to make it work. There are 2 parts to this depending on what you are trying to accomplish.

First, you have to define the mappings for the cck fields that are not text and options widget. Since they are not text and apachesolr module will need to know how to add it to the document.

function yourmodule_cck_field_mappings() {
return array(
'number_integer' => array ('index_type' => 'string')
// 'date' => array ('index_type' => 'date')
);
;

I have trouble defining a date field so currently don't know how that works yet. What this basically does is take any number_integer cck fields and submit to solr as a string type. And this is what I find as another limitation. It would be great if we can also do a if number_integer and options widget. But at this point, you can't.

Next, you need another hook function for the other cck fields that you still want and other non-cck fields as well.

function yourmodule_apachesolr_update_index(&$document, $node) {
// Update solr index.
try {
$fields = array('model','sell_price');
$cck_fields = array(
'field_publisher',
'field_contributor_name',
'field_contributor_name'=>array('multiple'=>1));

foreach ((array)$node as $key => $value) {
if (in_array($key, $fields)) {
$document->$key = $value;
}
if (in_array($key, $cck_fields)) {
if (is_array($value) && count($value) > 0) {
foreach ($value as $field) {
if (!empty($field['view'])) {
if ($cck_fields[$key]['multiple']) {
$document->setMultiValue($key, $field['view']);
}
else {
$document->$key = $field['view'];
}
}
}
}
}
}
}
catch (Exception $e) {
watchdog('yourmodule', $e->getMessage(), WATCHDOG_ERROR);
}
}

If you have fields that are not CCK, you would defined it in the $fields array. For cck fields, you would define it in $cck_fields. Note that you only need to do this if your cck fields are not text and options widgets and in yourmodule_cck_field_mappings. Also, the contributor section is a multi-value field in solr and there is currently a bug with multivalue (http://drupal.org/node/295314).

Once you have done that, apachesolr should be adding the documents in. But solr wouldn't know what to do with some of them so you need to modify schema.xml.

For cck fields, they will be indexed by the dynamic fields section. For the non-cck fields or the cck fields that are explicitely defined in $cck_fields, you will have to add them in manually in schema.xml like so..

field name="text" type="text" indexed="true" stored="false" <-- add after the last like of that field section
field name="model" type="string" indexed="true" stored="true"
field name="sell_price" type="sfloat" indexed="true" stored="true"
field name="field_publisher" type="string" indexed="true" stored="true"
field name="field_contributor_name" type="text" indexed="true" stored="true"
multiValued="true"

I have to take out the less than and greater than signs so they would show, but you will need to add them back in.

Save the file, restart solr, set apachesolr to reindex and this should be all set.

If the CCK is an options type then the mapping is automatically done, right?

yes, it is done automatically.

Note that, keyword searches (e.g. /apachesolr_search/xxxx) only match what appears in your node's body, title, and taxonomy terms (what's put into $document->text at index time).

If you set a CCK field display to "hidden", keyword searches won't match that field.

(E.g.: You have a CCK field "city"--if that's hidden then a search for, say, "chicago" would not match nodes containing "chicago" in that field)

A minor issue, really =) Hope I made sense =)

Hello,

I think this relates to an issue that I am having... I am trying to get write a custom module to add a product's list position (ordering weight) to the index so that I can achieve sorting by this. While I know that there is an Ubercart Integration module and this code should probably belong there, I would like to ask how I can achieve this since I don't believe that the list position drop-down is a cck field. I tried to copy the code from the Ubercart Integration module, but I am not having any success getting it to show up as an option for sorting.

Thanks.

Sean

There's no need to modify schema.xml to add a new field. schema.xml defines dynamic fields:

   <dynamicField name="is_*"  type="integer" indexed="true"  stored="true" multiValued="false"/>
   <dynamicField name="im_*"  type="integer" indexed="true"  stored="true" multiValued="true"/>
   <dynamicField name="sis_*" type="sint"    indexed="true"  stored="true" multiValued="false"/>
   <dynamicField name="sim_*" type="sint"    indexed="true"  stored="true" multiValued="true"/>
   <dynamicField name="sm_*"  type="string"    indexed="true"  stored="true" multiValued="true"/>
   <dynamicField name="tm_*"  type="text"    indexed="true"  stored="true" multiValued="true" termVectors="true"/>
   <dynamicField name="ss_*"  type="string"    indexed="true"  stored="true" multiValued="false"/>
   <dynamicField name="ts_*"  type="text"    indexed="true"  stored="true" multiValued="false" termVectors="true"/>
   <dynamicField name="tsen2k_*" type="edge_n2_kw_text" indexed="true" stored="true" multiValued="false" omitNorms="true" omitTermFreqAndPositions="true" />
   <dynamicField name="ds_*" type="date"    indexed="true"  stored="true" multiValued="false"/>
   <dynamicField name="dm_*" type="date"    indexed="true"  stored="true" multiValued="true"/>
   <dynamicField name="tds_*" type="tdate"    indexed="true"  stored="true" multiValued="false"/>
   <dynamicField name="tdm_*" type="tdate"    indexed="true"  stored="true" multiValued="true"/>
   <dynamicField name="bm_*"  type="boolean" indexed="true"  stored="true" multiValued="true"/>
   <dynamicField name="bs_*"  type="boolean" indexed="true"  stored="true" multiValued="false"/>
   <dynamicField name="fs_*"  type="sfloat"  indexed="true"  stored="true" multiValued="false"/>
   <dynamicField name="fm_*"  type="sfloat"  indexed="true"  stored="true" multiValued="true"/>
   <dynamicField name="ps_*"  type="sdouble" indexed="true"  stored="true" multiValued="false"/>
   <dynamicField name="pm_*"  type="sdouble" indexed="true"  stored="true" multiValued="true"/>

   <dynamicField name="tis_*"  type="tint"  indexed="true"  stored="true" multiValued="false"/>
   <dynamicField name="tim_*"  type="tint"  indexed="true"  stored="true" multiValued="true"/>
   <dynamicField name="tls_*"  type="tlong" indexed="true"  stored="true" multiValued="false"/>
   <dynamicField name="tlm_*"  type="tlong" indexed="true"  stored="true" multiValued="true"/>
   <dynamicField name="tfs_*"  type="tfloat"  indexed="true"  stored="true" multiValued="false"/>
   <dynamicField name="tfm_*"  type="tfloat"  indexed="true"  stored="true" multiValued="true"/>
   <dynamicField name="tps_*"  type="tdouble" indexed="true"  stored="true" multiValued="false"/>
   <dynamicField name="tpm_*"  type="tdouble" indexed="true"  stored="true" multiValued="true"/>
   <!-- Sortable version of the dynamic string field -->
   <dynamicField name="sort_ss_*" type="sortString" indexed="true" stored="false"/>

So, if you are adding a field for sorting, you would use one of the dynamic fields for sorting. I assume the position is an integer, and each product has only one. In that case, use the tis_* dynamic field. To add the value to the index, you need to implement hook_apachesolr_update_index:

function mymodule_apachesolr_update_index(&$document, $node, $namespace) {
  $document->sis_product_position = $node->field_product_position[0]['value'];
}

Above, I assumed your product position was stored in a CCK field called field_product_position. If that is not the case, correct the code as necessary.

Now, you just need to add that field as a sort option. I show how in #653148: Custom sorting. wonder95 provides a similar example in #785592: Data not indexed on cron run when field added in hook_apachesolr_update_index. If you have more questions, please follow up in #653148: Custom sorting, as that issue relates to sorting.

I would suggest using apachesolr_index_key()

/**
 * array('index_type' => 'integer',
 *        'multiple' => TRUE,
 *        'name' => 'fieldname',
 *        ),
 */
function apachesolr_index_key($field)

so

function mymodule_apachesolr_update_index(&$document, $node, $namespace) {
  $key = apachesolr_index_key(array('index_type' => 'integer', 'multiple' => FALSE, 'name' => 'field_product_position'));
  $document->{$key} = $node->field_product_position[0]['value'];
}

Except you want to call apachesolr_index_key with array('index_type' => 'sint', ...) in this case.