This module is working really well for me, but I've had a number of problems which all boil down to a single issue. As stated in the README, the module requires a unique identifier to be mapped for the search result entities. I'm not sure the documentation makes it clear, but this also has to be an integer (or the entity system complains about a type error). My problem is that the data I am storing in SOLR does not have a guaranteed unique integer field. There are nearly-unique integer fields, and a unique String field, but that is all.

So, I am currently using one of the nearly-unique fields, but it is a bit of a hack. Ideally, I would like the Sarnia module to generate an unique entity ID for me. Is this possible?

Also, I guess I can assume the unique ID only needs to be unique in the scope of the search results? i.e. it does not matter if a node in Drupal that is not indexed in SOLR has the same ID number as a SOLR result? If so, would the SOLR results document index be a sufficient identifier?

Thanks for your time. Andy.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

becw’s picture

Status: Active » Needs work

IIRC, something in the entity system (either Drupal core or Entity API) assumes that entity ids will be integers. I'm not sure that it would make sense to generate integer entity ids using Sarnia (... and store them in the Drupal database?) Do you know if there's a way to generate a serial field in Solr? I'd consider that a hack too, though... ideally you'd be able to use your string field.

This issue needs some research into non-integer entity ids; if they're allowed now (Entity API is evolving), then we need to make sure Sarnia supports them; if they're still not supported, maybe there are some Schema.xml techniques that we could recommend.

Also, how can we make the integer id requirement more clear in the Sarnia documentation? Maybe we need a section on Schema.xml configuration as well as configuring Search API for Sarnia?

anj’s picture

In the documentation, I guess "You must choose a field with unique values" could read "You must choose a field with unique integer values", which would at least mean that small but important piece of information is there and not only in the README.

There is an SOLR ord() function that very nearly does the right thing, but seems to be very memory hungry.

I guess I'll have to re-index and create a suitable integer for each item. I'd be happy to live with a hack for now, if anyone knows where in Sarnia the returned ID is mapped to the Entity.

becw’s picture

Ok, I've updated the Sarnia documentation.

It looks to me like Solr's ord() function is a property-level function, not result-level: it returns the index of a value in a multivalue Solr property. Also, generating IDs based on a particular query would conflict with caching--the same Solr record could be returned with different ids depending on the query.

krlucas’s picture

First, this module is the best thing ever.

Second, the documentation for the entity id property says:

id: The name of the property that contains the primary id of the entity. Every entity object passed to the Field API must have this property and its value must be numeric.

Notice it says "numeric", not integer. My Solr index is composed of very similarly structured records from different sources with potentially overlapping integer ids. However, by adding an integer "source" id to my index I was able to to create a composite float id of "$source_id.$item_id".

So far so good...

Antti J. Salminen’s picture

Actually, as that quote says there is no requirement for entity ids to be numeric if they are not fieldable. See here: http://drupal.org/node/1027908#comment-4636438

remydenton’s picture

Right, only fieldable entities require numeric IDs. Unfortunately, that doesn't help us much since Sarnia entities are fieldable.

Also, whatever the documentation may say, entity_property_verify_data_type() validates fieldable entity IDs specifically as integers and will throw an error if you've got decimal points in there.

My solr index actually uses namespaces in the field that's supposed to contain the unique ID (so the unique value is something like "islandora:001"). Obviously, that won't pass muster as integer or number, and I can't even think of a good way to generate a unique integer from a text string like that in Solr. Does anyone have a clever solution for this?

The alternative is to just bluster on through and use strings as the entity ID anyway. Hardly ideal, but the only issue I've run into so far with this approach is that I get a validation error when I try to add the entity ID as a field in the view. As long as I remove that field, things seem to work okay. Maybe I just haven't done enough testing yet...

anj’s picture

For what it's worth, this problem has more or less disappeared for me, because we're moving to Solr4. In order to support SolrCloud, each document has an internally generated _version_ long integer timestamp, and this can be used for the entity ID.

ibakayoko’s picture

you need to install a solr 4, create a new collection and import the index from your solr 3 core.

and use sarnia to query the new collection.

tombraider’s picture

Hi,

I'm trying to display data into my drupal website using sarnia to retrieve external/non-drupal indexed data from Solr. I followed the steps in this link https://drupal.org/node/1379476 but I'm somehow stuck with the "ID field" select box. My select box is not showing any option, therefore I cannot proceed with the next step. This is because I don't have a unique integer id. I only got mongoid which is a type string that I use as unique id from my non-drupal application. I'm currently using Solr 4.3 and I have the _version_ field of type long in my schema.xml file.

I'm wondering how we can use that as the entity id since no option is showing up in the select box. Any help regarding this would be very much appreciated.

andrewmacpherson’s picture

becw's change to the handbook page (comment #3) is alos needed in the module's README.txt file.

Patch attached.

andrewmacpherson’s picture

Status: Needs work » Needs review
jmdeleon’s picture

Status: Needs review » Reviewed & tested by the community

Patch of corrected documentation accepted.

jmdeleon’s picture

Adding an additional patch to this: Since the unique ID field is no longer required to be an integer value, I made a small change to the contextual filter handler to treat all arguments as strings, where they can further be processed (and converted to integer if necessary).

jmdeleon’s picture

jmdeleon’s picture

Status: Reviewed & tested by the community » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.