Hi,

I was wondering if the apachesolr module can handle fields from solr that are not added by drupal.

If not, is there a chance that this will be integrated? The Roberson Library has developed a solr module that retrieves solr information that is ingested with non-drupal (no nodes or cck fields) data. https://github.com/Islandora/islandora_solr_search . It's not ready to be ported as its own module on drupal.org yet - it would need a redesign.

Just wondering if you think this functionality could be integrated in apachesolr or if it should stay it's own module.

Cheers,
Danny

CommentFileSizeAuthor
#7 islandora_solr_search.jpg24.53 KBdanny_joris

Comments

nick_vh’s picture

It is certainly possible to do this.
Example :

//we need an additional query for the statistics of the field
// We can optionally specify a Solr object.
$solr = apachesolr_get_solr();
$query_stats = apachesolr_drupal_query('apachesolr_stats', array(), '', '', $solr);
$query_stats->addParam('stats', 'true');
$query_stats->addParam('stats.field', $this->facet['field']);
$query_stats->addParam('stats.facet', $this->facet['field']);
$response_stats = $query_stats->search();
if ($response_stats->response) {
  $stats_minmax = $response_stats->stats->stats_fields->{$this->facet['field']};
}
danny_joris’s picture

Thanks - I don't really see the context of this snippet yet, but nice to see it is possible. I'll do some research.

We could be working with cases with up to 10 different requestHandlers and up to 200 different custom fields. We don't need to store drupal content at all, but if so, would it be advisable to combine our custom data with drupal data in one solr table or is it possible to address 2 solr instances?

Would using the apachesolr module bulk the code to achieve this result?

nick_vh’s picture

My bad, I didn't completely pasted the snippet. This is querying the statistics from a field but you can do that with other filters and parameters.

nick_vh’s picture

Oh, and apachesolr 7.x-1.x allows you to have multiple solr environments/instances. So you can fetch the solr that is appropriate to you. Please dive into the code to get a better understanding I'd propose.

If you have more specific question, don't hesitate to ask!

nick_vh’s picture

Status: Active » Fixed

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

danny_joris’s picture

Title: Pulling non-drupal solr data. » php
Status: Active » Closed (fixed)
StatusFileSize
new24.53 KB

Mind if I re-open this as it's related to the same custom solr index? (and I finally found time to explore this further)

So I've been playing around a bit with both the D6 and D7 version. In the D7 version I can point to our custom solr instance. I've been checking most of the DrupalApacheSolrService methods and I'm able check luke successfully, which is a good sign. I'm still unable to perform a successful query on the normal search results. It's because the way we set it up is using requestHandlers defined in solrconfig.xml. I managed to get the right results by calling <?php $solr->search('myQuery', array('qt' => 'standard', 'defType' => 'dismax')); ?> programatically. Is there a way we can select a requestHandler and defType (always dismax in our case) for the default search results? Also it still limits the results by 10 - I don't know where that's defined. How does apachesolr handle this? Does it use dynamic requestHandlers? It seems to just return every field?

The islandora_solr_search module search results are pluggable, meaning you can create a custom module that hooks into islandora_solr_search, which you can use to leverage a custom display. (lists, solr results on maps etc.. based on the same solr query) I would assume we can use 'multiple displays' in apachesolr by creating multiple solr environments that point to the same solr instance. How would we theme/render the output of these different displays? Can we hook into it to preprocess the data?

If I'm able to pull out the solr results, I assume I can hook into the facet api to leverage our own facets? Probaby using the snippet you provided earlier in this tread?

In islandora_solr_search we're able to use a 'simple search' which performs a query from one textfield which uses a defined requestHandler and dismax. We also have a second form that doesn't use dismax, but instead performs queries directly on specific solr fields. It would have a dropdown for every textfield which sets the solr field to search on and it has an AND/OR dropdown between query strings (see attachement). Is this kind of form available in apachesolr too?

Thanks!
Danny

danny_joris’s picture

Title: Pulling non-drupal solr data. » php
Status: Closed (fixed) » Active
danny_joris’s picture

Title: php » Pulling non-drupal solr data.

not sure what happened there.

danny_joris’s picture

I'm still trying to pull the right data from my solr index. I created a custom module with this hook_apachesolr_query_alter() function:

<?php
function custom_apachesolr_query_alter($query) {
  $query->addParam('qt', 'standard');
  $query->addParam('defType', 'dismax');
  // without dismax adding some fields of which I know exist, doesn't work either:
  $query->addParam('fl', array('mods_title_s', 'mods_topic_s')); 
}
?>

Adding dismax does return the correct number of results, but all results are empty (just 3 dots: ...). The pager does get rendered correctly.

It also throws this error for every result on the page:
Notice: Undefined property: stdClass::$id in apachesolr_search_process_response() (line 1163 of /home/quickstart/websites/example.dev/sites/all/modules/apachesolr/apachesolr_search.module).
or
Notice: Undefined property: stdClass::$entity_type in apachesolr_search_process_response() (line 1176 of /home/quickstart/websites/example.dev/sites/all/modules/apachesolr/apachesolr_search.module).
Which makes me think that even in the search results it's assuming drupal specific content.

nick_vh’s picture

Title: php » Pulling non-drupal solr data.
Status: Closed (fixed) » Active

Why would you want to change the query type? I don't see the advantage since this is a lesser version than edismax. Edismax is enabled by default.

danny_joris’s picture

I didn't know Edismax was enabled by default. I can't return anything with it. Would I need to update our schema.xml or solrconfig.xml? Or do I need a different query?

Seems like edismax is only available in solr 3.x? I'm trying to access a solr 1.4 instance.

Does dismax vs edismax have anything to do with the errors that are thrown?

danny_joris’s picture

The standard requesthandler in solrconfig.xml of the sample data is :

<requestHandler name="standard" class="solr.SearchHandler" default="true">
<!-- default values for query parameters -->
<lst name="defaults">
  <str name="echoParams">explicit</str>
  <str name="fl">PID, mods_title_s, mods_topic_s, mods_geographic_s, mods_publisher_s, mods_genre_s, mods_subjectName_s, mods_dateIssued_dt,  mods_coordinates_s</str>
  <str name="q.alt">*:*</str>
  <str name="qf">mods_title_it^2.5 mods_topic_it^2.0 mods_publisher_it^2.0 mods_description_t^3.0 mods_subjectName_it^3.5 mods_genre_it^3.5  mods_geographic_it^2.0 PID^0.5</str>
</lst>
</requestHandler>
nick_vh’s picture

I'm mistaken! My bad. It is possible to enable edismax more easily but it is not default. Dismax is.
So you should also not set it in your query object :-)

danny_joris’s picture

np - so dismax can be triggered without it being in the solr query? It seems like this is the GET request that gets called. It's what the SolrBaseQuery->solr method returns.
/solr/select?start=0&rows=10&spellcheck=true&q=california&fl=id,entity_id,entity_type,bundle,bundle_name,label,is_comment_count,ds_created,ds _changed,score,path,url,is_uid,tos_name,mods_title_s,mods_topic_s,PID,pid,nid ,tm_node&spellcheck.q=california&wt=json&json.nl=map
Dismax isn't included in this request and it also doesn't return anything from my solr instance (because dismax isn't included?)

Sorry for all the questions.

nick_vh’s picture

Dismax is already set in the solrconfig.xml

  <requestHandler name="dismax" class="solr.SearchHandler">
    <lst name="defaults">
     <str name="defType">dismax</str>
     <str name="echoParams">explicit</str>
     <bool name="omitHeader">true</bool>
    </lst>
  </requestHandler>
danny_joris’s picture

Odd - because it's also set in the solrconfig.xml of our solr instance. I'll look into it.

Issue remains that when I do add the dismax defType parameter, I can pull the results, but the rendering goes wrong. It would be really awesome if we could pull our custom data, but it seems like there are a lot of drupal-specific calls and checks.

nick_vh’s picture

Status: Active » Closed (works as designed)
danny_joris’s picture

Fair enough.

nick_vh’s picture

Just as a last comment :
drupal.org/project/sarnia

This is meant to read out fields that are non-drupal.