Biblio fields are not indexed by search

There are a couple of reasons this might be happening...

1. You are not running a regular cron job to update your index.
2. Automatic indexing on save or update is turned off... Check the setting in the "Search" section on the '/admin/settings/biblio' page, the last option turns on reindexing on saving or updating a biblio node.

Hope that helps

Ron.

Hi Ron,

This issue was posted by a support engineer from Acquia as we bought a subscription for the Acquia Search service (=Apache Solr search). The problem was that Biblio fields were not indexed, only node common fields like title etc. The support engineer tested some cases and found out what is described in the first post above. In this post I included some of the posts from the Acquia ticketing system.

Is this something that can be fixed by you? Or is something else going on? I tried to apply your suggestions but that did not do the trick...

Regarding your suggestion to run cron: because of Acquia Search cron is called every 5 minutes...

Posts from the Acquia ticketing system regarding Biblio fields not being indexed:

***********************************
Hi,

For a customer we created a Drupal portal. We have migrated content from different systems into this portal. Because the content is library related content we used the Biblio module for this. Because we wanted to use the advanced search functionality of Apache Solr but in a user friendly way, we bought a subscription to Acquia Search.

It looks now that not all content within the portal is being indexed. It looks that the fields used in the Biblio module are not indexed. Can you confirm this is not being indexed? If so how can this be solved?

All content (about 22.000 nodes) in the portal is Biblio content and only the title is properly indexed, we want ALL biblio fields (biblio authors, publication language, publisher etc.) to be indexed.

Can you please help us? Without Biblio items being indexed the portal has not so much use.

Best regards,
Christian
---------------------
Hi Christian,

Regarding the Biblio module, my testing shows that by default, only a sub-set of fields are exposed to search. This is by design, and not a limitation of Acquia Search or Drupal core search.

To allow other Biblio fields to be exposed to search, those fields must be set to "Common" in the Biblio fields configuration, on the "Administer > Site configuration > Biblio settings > Fields > Common" page (/admin/settings/biblio/fields):

Fields that are not set to "Common" will still appear on the node form, under the "Other biblio fields" fieldset, but are not exposed to search or displayed to the end user in the node view.

After changing the visibility of these fields, you may need trigger a reindex of your site from the "Administer > Site configuration > Acquia Search > Search index" page (/admin/settings/apachesolr/index).

Regards,
Peter Bull
Acquia Client Advisory Team
---------------------
Hi Peter,

It doesn't look well. I have changed the biblio module settings like you suggested and reindexed all content (>22000 nodes, that took a while). But still none of the biblio fields that were set to common and made visible at publication type level are being found in any of the searches. Only words from the title of a biblio node are correctly found.

Best regards, Christian
---------------------
Hi Christian,

I'm experiencing different behavior on your site than on my test site. I notice that you are using the development release of Biblio 6.x-1.x-dev -- was there functionality in the dev release that made you use that version instead of the stable 6.x-1.14 release?

I'm going to set up a new local test environment using the Biblio 6.x-1.x-dev release that you are using, and I will report back.

Also, regarding your Solr faceting question, I just saw that there is an Apache Solr Biblio module (http://drupal.org/project/apachesolr_biblio). It's a development release but may provide the functionality you need.

Regards,
Peter Bull
Acquia Client Advisory Team
---------------------
Hi Peter,

The dev release will soon be the next stable release of Biblio (6.x-1.15). The dev release has the functionality of exporting biblio items to Rich Text Format (RTF), that is a functionality that we use.

My experience with the dev releases of biblio is that they are quite stable already...

Hopes this helps.

I'll have a look at the module you are suggesting. The question I have (without looking at the project) is will it work with Acquia Search?

BR, Christian
---------------------
Thanks for the update Christian. Still testing on my end.

My initial findings... with a clean Drupal install and Biblio 6.x-1.x-dev (2010-Oct-20) my biblio nodes were NOT being indexed by search. Node titles were indexed, but the body of the node was not being sent to the search index.

Disabled and uninstalled the dev module, deleted my search index, then installed Biblio 6.x-1.14 - my biblio nodes were now being indexed and words the appeared in the biblio fields were returning search results.

Updated from 6.x-1.14 to 6.x-1.x-dev. New biblio node content was not being indexed. Old (existing) nodes still appeared in the index.

Rebuilt the search index (still using Biblio 6.x-1.x-dev). Now no biblio fields are being indexed. So I can safely say at this point that there has been a regression in the dev release of the Biblio module that has broken search.

My next step will be to investigate the Apache Solr Biblio and report back my findings.

Regards,
Peter Bull
Acquia Client Advisory Team
---------------------
Hello Christian,

Also, yesterday I tested the Apache Solr Biblio module and found it was not compatible with the current Biblio 6.x-1.x-dev release.

I will reach out to Ron Jerome today and post to the issue queue. Thanks!

Regards,
Peter Bull
Acquia Client Advisory Team
---------------------
Hello Christian,

I ended up working on this throughout the day and achieved a possible solution. This was using the current 6.x-1.x-dev release of the Biblio module in CVS, although it should also work with the dev release that you are using.

The fields are not being indexed because the them functions that display those fields will only display biblio fields on a node page. They do not display them to a teaser, or when the node is rendered and passed to the search index. (Regardless of whether a site uses core Drupal search or an external solution such as Solr.)

The solution will involve adding an "update index" case to the nodeapi hook. So the function biblio_nodeapi() in biblio.module would look something like this:
function biblio_nodeapi(& $node, $op, $a3, $a4) {
if ($node->type == 'biblio') {
switch ($op) {
case 'delete revision' :
db_query('DELETE FROM {biblio} WHERE vid = %d', $node->vid);
db_query('DELETE FROM {biblio_contributor} WHERE nid = %d AND vid = %d', array($node->nid, $node->vid));
db_query('DELETE FROM {biblio_keyword} WHERE nid = %d AND vid = %d', array($node->nid, $node->vid));
break;
case 'insert':
if (variable_get('biblio_index', 0)) {
_node_index_node($node);
search_update_totals();
}
break;
case 'update':
if (variable_get('biblio_index', 0)) {
// _node_index_node performs a node_load without resetting the node_load cache,
// so it would index the old version. We reset the cache here.
// Don't assign node_load to $node because node_load resets e.g. the menus mlid etc.
$mynode = node_load($node->nid, NULL, TRUE);
_node_index_node($mynode);
search_update_totals();
}
break;
case 'view':
if ($node->type == 'biblio' && variable_get('biblio_hide_bibtex_braces', 0) && !empty($a4)) {
drupal_set_title(filter_xss($node->title, biblio_get_allowed_tags()));
}
break;
case 'update index':
// render the biblio fields as html and return the string for indexing
$style = biblio_get_style();
$base = variable_get('biblio_base', 'biblio');
return theme('biblio_long', $node, $base, $style);
break;
}
}
}

I posted an abbreviated version of this information to the Biblio issue queue as well: http://drupal.org/node/955258

Regards,
Peter Bull
Acquia Client Advisory Team
***********************************

Does this help you with the problem? If you need more info feel free to contact me (you have my email address already).

Best regards, Christian

Ahh, I did a diff between the -dev and 1.14 version and I now know what caused this regression. I'll fix it and make a 6.x-1.15 release.

Ron

The fix has been committed... http://drupal.org/cvs?commit=446008

And just for the record... The fields don't have to be set to "Common" to be indexed but they do have to be EITHER "Common" for all publication types OR "Visible" for a particular publication type (accessed from admin/settings/biblio/fields/type)

Cheers,

Ron.

Thanx Ron, a lot! I will try the fix and report back if it works! Once again thank you!

Best regards, Christian.

Looks like that did the trick! Now waiting for all those cron runs to index all nodes!

Best regards, Christian.

Good, sorry about the regression, but it never occurred to me that node_index would be calling hook_view with both $teaser and $page set to NULL.

Ron.