Hey,

In an implementation of ApacheSolr, i have come across the need to index fields beyond the core node fields which are included by the module, but yet are not CCK fields. This seems like something that might be useful to others beyond just my case, and hence have created a patch here.

My goals were:

  1. Ability to update the index of a specific node upon insert and modification
  2. Ability to index and query custom attributes of a node that may be added to the node object, but not be CCK fields

What was updated:

  1. Updated the 'update_index' and 'getNodesToIndex' methods to allow the passing of 'nid' as an optional secondary parameter
  2. In the 'update_index' method of the ApacheSolrUpdate class, added a check for additional fields to look for. This was accomplished in a similar fashion to how CCK fields were already being done, but this was done by exposing a new hook, so that each module can add their own custom attributes.
    Ex. $addl_fields = module_invoke_all('apachesolr_node_fields');

How the features are implemented (via the search module that would be implemented, ex apachesolr_search.module):

  1. I do not have to implement the hook_update_index().
  2. In hook_nodeapi(), I can now invoke the updating of the index for a single node. Ex: ApacheSolrUpdate::update_index('apachesolr', $node->nid);
  3. Implemented the new hook_apachesolr_node_fields() to add the new attributes that one of the modules add to the node object via hook_nodeapi / load. Ex:
    function modulename_apachesolr_node_fields() {
      $fields = array();
      $fields['asset_category'] = array('name' => 'asset_category', 'field_type' => 'text');
      return $fields;
    }//end - function
    

The result, after applying the patch:

  1. The module is completely backwards compatible with the original implementation of the module.
  2. But, applying this patch, and implementing the new hooks, we can now search by the custom attributes without modifying the module, or changing the schema.xml. Querying these new attributes is done in the same way as querying CCK fields was done, as:
    ssasset_category:tops
    

    This can also be done by adding these fields as blocks I suppose much like other CCK fields can be added as filters.

  3. One other thing that i am working on now is adding the ability to sort via this search box, so that we can do something such as "solrsort:nid desc"

    This is needed because I need to define a default sort order, since the score in my case is not useful.

    Please let me know your thoughts, etc, and if you think this could be incorporated into the full module. It would be great to be able to know that I can keep up to date with the module if these necessary changes can make the way in.

    -T

CommentFileSizeAuthor
custom_attributes.patch5.9 KBsocki

Comments

JacobSingh’s picture

Hello,

Thanks for contributing!

A couple thing:

1. I don't think you need to modify the query to get nodes to index. When patching, try to change only a minimal amount and specific to your index.
http://drupal.org/node/10263 is a good resource.

2. I believe you can already do what you are doing without adding additional hooks. see hook_apachesolr_update_index() and hook_apachesolr_modify_query().

Best,
Jacob

socki’s picture

Hi,

Thank you for the response. My intention was not to suggest a rewrite to the module. I understand the idea of keeping things simple, efficient and useful to all and ideally changing as little as possible. That being said, we're half way there!

Querying / Searching

I was actually able to utilize the hook_apachesolr_update_index(). I apologize, I had confused that with the standard search module's hook_update_index(). So I was able to remove the addition of the new hook I created to handle the additional fields.

Indexing

Unfortunately, I still don't get how you suppose i limit the updating of the index to a single node. I don't see the hook_apachesolr_modify_query() being instantiated anywhere and when i defined it, it didn't seem to get called anywhere. From the description of it though, this appears to be limiting data on output, and not upon index.

What i need to do is only index content upon creation or update. The rationale behind this is that the site I am working on currently contains over 6 million nodes. I do not need any of that past content indexed. In addition, I only need new and updated content of a specific node type to be indexed. For this reason don't implement hook_update_index(). To handle this, I'd like to invoke the update to the index via hook_nodeapi() which is the reason for adding nid as a secondary / optional parameter to those two methods in the class:

/**
 * Implementation of hook_nodeapi
 **/
function modulename_nodeapi(&$node, $op, $a3 = NULL, $a4 = NULL) {
  //switch on operation
  switch ($op) {
    case 'insert' :
    case 'update' :
      ApacheSolrUpdate::update_index('apachesolr', $node->nid);
      break;
  }//end - switch
}//end - function

Thanks... Let me know what you think. I can create an updated patch if this sounds right to you.

-T

flexer’s picture

Socki,

looking into apachesolr_search.module I see:

        foreach (module_implements('apachesolr_modify_query') as $module) {
          $function_name = "{$module}_apachesolr_modify_query";
          $function_name($query, $params);
        }

just before the query being issued. I'm using the -dev 6 version.

I'd like to index a specific node via nodeapi, too :)

socki’s picture

I'm using the Drupal 5 version.

You're right... so i see where it's being called now in the code, but it's being called from within the 'search' hook of hook_search() which is a tad late for where I need to call it as this would merely filter the data upon output.

Just for a little more background, i'm not doing a full sitewide search, as may be the standard implementation. That being said, it might be useful to have a little more granular control of sorts in what gets indexed... If the update query was applied to the query that gets executed within ApacheSolrUpdate::update_index() then that would be excellent, as of right now, it doesn't appear possible, unless I'm missing something. Hence the patch. :-)

You seem to have the same issue as I do... If its helpful, I'll upload an updated patch in a bit which only enhances this one aspect.

JacobSingh’s picture

Status: Needs review » Closed (duplicate)

I think this is long since resolved in other ways. The apachesolr_og module is likely a good example of non-cck custom fields.