Hey,

Currently the views integration is a bit disappointing as it is not possible to access individual CCK fields. #455742: Node row plugin demonstrates that the node output is possible, so it should already be possible to access individual fields via the theming functions.

My suggestion:
Instead of reimplementing the handlers for each and every CCK field as in #576286: Add general SQL support (which unfortunatelly does not work for me right now), wouldn't it be easier to create a relationship to the found nodes?

Obviously from the performance perspective this approach is inferior compared to rendering the found fields directly, but it should allow us to access all CCK fields and use their full functionality.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Scott Reynolds’s picture

Status: Active » Postponed (maintainer needs more info)

wouldn't it be easier to create a relationship to the found nodes?

How?

Been one thing I have been talking about. The problem is we can't allow the arguments and filters to be brought in. As they won't work, you cant mesh two filtering schemes (Solr and DB) together.

Your idea isn't bad, just we need to build a views_query object for the database and give it access to all fields it knows about already. Its non-trivial

But im listening for ideas.

digi24’s picture

Status: Postponed (maintainer needs more info) » Active

Hi Scott,

thanks for your comments. I have looked further into this issue (personally, I have some experience with the hooks in views and external data).

As you have said, we need some table to use with the JOINs. I am currently looking at the faceted search module, which implements a nice way of achieving this. They create a temporary table in memory, fill it with the search results and join it with the node table.

What do you think?

martin

Scott Reynolds’s picture

Doesn't address the important problem

The problem is we can't allow the arguments and filters to be brought in. As they won't work, you cant mesh two filtering schemes (Solr and DB) together.

So the best way to describe how a temp table won't solve this is an example. I create a View, make a relationship to the node table using the temp table technique. I then apply Node type = 'story' using the Node relationship.

I execute a search string and it makes the temporary table bringing in the ten best matches. What if none of the ten best matches are 'story'? What if only some of them are 'story'?

In order to get this technique to work you would have to pull in ALL Solr documents and then apply the filter on it.

Same type of scenario for sort. If i create a View add the node relationship and sort by nodequeue position. What if none of the best matches are in any nodequeue? What if only some of them are?

What really needs to happen is the Solr Views Query needs to be able to pull in additional fields and fields only. And it can do that by have a standard Views query as part of its class and execute that query for more fields.

digi24’s picture

Thanks for your comments. I really appreciate this, as I need to at implement a quick and dirty solution asap, and I am trying to avoid going the wrong way.

Regarding your examples:
I do not really see the big difference. Any result to be achieved using your suggested procedure can also be achieved via the temporary table approach. In both cases we are first executing the solr query and then combining it with other information. There are in my view only two differences:

1. the temporary table allows the user to define "stupid" views queries that may not lead to the desired results. In your example, the filter by type can be achived purely with solr, using the node relationship is "stupid". Other node filters may not be achieved through solr and suffer the problems you have mentioned. (I do not really care about this, but I would probably have another perspective if I were the maintainer facing support tickets).
2. please tell me if I am wrong, but when we want to start pulling in additional fields, a handler must be defined for each field type within the solr_views module. I do not think this is consistent with the drupal logic of keeping modules indepedent of each other.

Or is there a hook or something similar where I could just loop through all availible fields loading the appropriate definitions?

digi24’s picture

Status: Active » Postponed

I will post any progress experiences, feel free to comment, but I am postponing the feature request.

Scott Reynolds’s picture

2. please tell me if I am wrong, but when we want to start pulling in additional fields, a handler must be defined for each field type within the solr_views module.

In this case its not true. See the existing field handlers for CCK using standard Drupal queries, uses things like add_field(). Solr query object hasn't defined those functions. So those handlers can be used almost independent of the the Query object. The query object just has to have those methods and then parse the input, build a standard SQL view query object, execute it and then blend the field results into the $result object.

1. the temporary table allows the user to define "stupid" views queries that may not lead to the desired results. In your example, the filter by type can be achived purely with solr, using the node relationship is "stupid". Other node filters may not be achieved through solr and suffer the problems you have mentioned. (I do not really care about this, but I would probably have another perspective if I were the maintainer facing support tickets).

Right, I can't put that into the module because it would open up all these holes and not function as people expect.

There is a great discussion with the FieldAPI guys and merlinofchaos about this because the new FieldAPI allows for your fields to be stored somewhere else besides the SQL database. Can't find it right now.

domidc’s picture

subscribing

ayalon’s picture

What was you solution digi24?

Scott Reynolds’s picture

The simplest way is to add a CCK build type. Exclude the fields you don't want displayed and choose the formatter for the fields you want displayed. Then set up the View and use the Row plugin Node and select your newly created node build type.

kenorb’s picture

tbenice’s picture

sub

tbenice’s picture

Here's a patch that worked for me so far. However I get each field twice in the views ui, might be: http://drupal.org/node/992598. This approach also gives other relationship support like to the content profile of the user.

I think the patch also includes a solution for: http://drupal.org/node/449190#comment-4155902, which I found necessary.

cheers,

Ted

picxelplay’s picture

@tbenice, does this actually work for you for adding relationships? If so, what relationships have you successfully achieved? It doesn't work for me at all.

rjbrown99’s picture

Status: Postponed » Needs work

There's a patch here now, so I'm changing status to make it more visible to others.

tbenice’s picture

Hi, the patch adds a new relationship to the node. From there it allows you to add any relationship that the node can make. My goal was to pull content profile data (picture, full name etc) into apachsolr views. For that it works fine.

However, I've moved back to views 2 for the time being.

rjbrown99’s picture

Thanks, I might check it out at some point. Right now I'm focused on integrating the flag module with arguments as a way to filter content. I originally started by thinking of bringing in the flag as a relationship (similar to how you would do it in traditional views) but it's going to work better as an argument for me.

picxelplay’s picture

All I get for a Relationship option is 'Apache Solr: Node Object', then click to add, then I get Broken/missing handler.

tbenice’s picture

let me look into it. you might be missing the handler .inc.

rjbrown99’s picture

To continue my earlier post, I'm going to be working on a flag relationship since arguments won't get me where I need to go. It will likely be based on the flag vista module which has expanded views support for flag.

picxelplay’s picture

If I just needed to show flag/unflag links in my apachesolr_views (and not have any sorting), would I need to expose Flag and Flag Vista to apachesolr_views, or only Flag Vista?

rjbrown99’s picture

If you want to do anything with flags in apachesolr views, it's going to take implementing flag support for apachesolr views. It does not exist in either module at this point.

tbenice’s picture

Hi again,

you're probably missing the attached handler file. Extract it into /apachesolr_views/handlers. Let me know how that works. I've only tested this patch/handler with content profile via the node's author, but i suspect it might work to integrate flags. Sorry if this is wasting your time... :)

-Ted

rjbrown99’s picture

Thanks for posting #22, this is proving to be helpful. Rather than re-implementing flag support from scratch, I decided to give your code a try (using the relationship to the base node) and then using flag_vista's Node: flaggings with a relationship back to the original node. In theory this would accomplish what I am after, a filtered view of all nodes flagged by a specific user.

In reviewing the code in #12, specifically in apachesolr_views_query.inc I see this:

+          //add relationships
+          if (!empty($this->relationships)) {
+            
+          }  

Is something missing there? A call to add_relationship?

tbenice’s picture

@rjbrown99: I think that might be an orphan. I had to add add_relationship because views was calling it inside the class...same for adjust_join. I take it this isn't working for you as-is or you would have posted that it worked. Could you please expand a bit on the behavior you're seeing? Thanks!

Let me do some testing, or you can go ahead and work on it too ;). I can try adding flags- i'm not familiar with flag_vista, but use flag all the time.

rjbrown99’s picture

Thanks for the follow-up. My issue is I never saw the query function in the handler called. I added a debug statement right at the top of query but nothing ever arrived there. I did globally clear the cache as well. It's a pretty basic view and I have tried a few different iterations of working to get it to call the query function (second relationship via flag_vista, changing from node teaser views to fields and trying to add fields directly, etc.)

Flag vista was just a shortcut way for me to make an easier view of flags. The core flag module would likely work as well, it just takes an extra relationship to arrive at a final view of flags by user.

I'm using the patch in #12 and the handler in #22 and I haven't changed either of them yet.

rjbrown99’s picture

Disregard my earlier comment, it is true that it is not working for me but it IS getting to query(). That was a mistake between the user and keyboard not in the code :)

picxelplay’s picture

12 + 22 tested and confirmed for adding flag/unflag links.

rjbrown99’s picture

I axed flag vista and went back to the traditional flag module, using 12+22 patches. My view is the traditional setup for showing nodes flagged by the current user -

1) Relationship to Apache Solr: Node Object
2) Relationship (node object) Flags: my flag (by all users)
3) Relationship (my flag) Flags: User
4) Argument (Flag user) User: UID (of the currently logged in user)

This is how I do it via traditional database views, which works well and should roughly line up with the recommendations in the handbook pages for flag. In this case, I run into this:

PHP Fatal error:  Call to undefined method apachesolr_views_query::add_where() in /mysite/sites/all/modules/views/handlers/views_handler_argument_numeric.inc on line 93

... which seems to be unimplemented, similar to #849650: Global: Random sort bug and add_orderby method missing in class apachesolr_views_query. So next I'm going to fix that and see how we go from there. Adding the function directly from views corrects that php error, but also still does not return correct results.

I dumped the query out of apachesolr/SolrPhpClient/Apache/Solr/Service.php, Apache_Solr_Service, search function and can't really see anything at work here. Not sure where to go next. Hm....

Scott Reynolds’s picture

That error is on purpose, to prevent people from trying todo what you tried todo.

It is a very very bad idea to try and filter across Solr AND database. You should instead look to find a way to index who flagged the node into solr.

See my very first comment

The problem is we can't allow the arguments and filters to be brought in. As they won't work, you cant mesh two filtering schemes (Solr and DB) together.

rjbrown99’s picture

Well thank you Scott, that's actually quite helpful. I did toss the flag into the Solr index and I could search on it. I'm not sure how to relate that back to a relationship based on the user. IE, I think it would be relatively easy to query on any node that is flagged (since the flag is in the Solr index now), but the question is filtering on any node flagged by a specific user. That's where I went back to the relationship and how I got here :)

How I index flags: #1177210: How to make a facet for a per-user flag, with an added hook_flag() to run apachesolr_mark_node when the node is flagged or unflagged.

Scott Reynolds’s picture

Right you have to index it as a multivalue field. Think of it as an associative array.

rjbrown99’s picture

Thanks again all, working now using direct Solr with no relationships. The rough outline of what I did:

1) Add update_index function to index new field in Solr, including any custom functions (see #1177210: How to make a facet for a per-user flag)
2) Add a trigger for updating the index. In my case, hook_flag() to mark the node for indexing when it is flagged/unflagged (see #546980: Flag module: Index flags and update Apache Solr index every time a node is flagged.)
3) Add a facet via hook_apachesolr_facets() to enable the ability to facet on this content
4) In the Apachesolr UI, visit admin/settings/apachesolr/enabled-filters and turn on the new facet
5) Add very basic Flag UID argument support to Apachesolr tied to flag, using apachesolr_views_handler_argument so no special handlers.
6) Set up my view with the Flag UID argument, in my case defaulting to the logged in user

I have no patch to submit as this is already tied to my modified apachesolr_views module that has rolled in bugfixes from other issues as well as decimal support and a few other things from the issue queue here.

Thanks for the help, and sorry if I changed the point of this thread a bit from relationships to flag.

tbenice’s picture

so is the opinion that adding the node relationship to the indexed nid somehow dangerous? It seems important and I could use a little clarification on that point.

thanks in advance!
-Ted

Scott Reynolds’s picture

It is dangerous because filtering on the result set will be impossible with a relationship.

No way to combine filter on SQL with filter from Solr. Any relationships that get working need to ONLY allow the user to add new Fields NOT new filters / arguments or to create ADDITIONAL relationships.

My comments at the top of the issue talk about specifically Merlinofchaos here: http://drupal.org/node/443422#comment-1644378

It is just not possible to add filters / arguments from two different databases. It will not work.

rjbrown99’s picture

Yea I gave up on the db+solr relationships per the recommendation, and it definitely worked out better for me for flag filtering. My only issue now is 'realtime' indexing of content when users change flags. I need to trigger solr to index much more frequently as users expect to see content in there as soon as they take action on it. That's a different topic though.... thanks for the pointers.

rjbrown99’s picture

FWIW, my solution to 'realtime' indexing of content: #1234642: Rules Integration.

rjbrown99’s picture

Ok so I am back to this, with a need to link my apachesolr_views output to the content profile of the user. I only need CCK fields to display, no filtering or sorting or anything special. Basically what I want is "for node X returned by apachesolr_views, look up the author's content profile and display a few fields."

I would normally do this via a relationship with SQL views. Based on Scott's message I am not going to attempt to do it here, despite #12/#22 existing. I am taking the idea very literally that Solr+Database = broken/bad.

My new idea is to look at hook_views_pre_render() since the Solr output should already be available at that point, and I can go back and grab the relevant information from the profile. So it would work similar to the way a relationship would, just in code vs the UI. I'm mentioning it here since it does not appear elsewhere in this thread and may be an approach to help solve the relationship need.

tbenice’s picture

OK, I'll be watching this closely...I hope you won't mind posting progress...

thanks!

rjbrown99’s picture

That was pretty easy actually. Here's a helpful link: http://www.trellon.com/content/blog/view-views-api

function mymodule_views_pre_render(&$view) {
  switch ($view->name) {
    case 'view_my_view_name':
      foreach ($view->result as $index => $result) {
        $view->result[$index]->users_name = somefunction($result->uid);
      }
      break;
  }
}

Dump out the $view results using devel, flog, or some other way to see it and you should now see a $view->result array. It helpfully includes the node nid and owner uid, so you can take the uid and do whatever you want with it - load content profiles, etc. I have a custom profile cache function so I can avoid node_load of every content profile on a page load, so I call that function to come back with the user's name.

There IS a trick here. http://drupal.org/node/1130548#comment-4362428

Changing stuff in the $view->result will not be used in your view output if the field wasn't part of the original query. So for example, if you add a CCK field from your content profile and then try to modify the name via $view->result, it's not going to work per that comment above.

In my case, I just wanted to modify the user's name, so I added the User: Name field to the view (which would come back with the returned nodes, since every node has an owner), and then I modified that in the view result. Works for me now.

One last link that was helpful: http://groups.drupal.org/node/131029

rjbrown99’s picture

One more hint about my previous tip of replacing extra fields: watch your input settings. I don't allow user signatures so I was using that field for output, but Drupal is smart enough to run that through the HTML filter on input type. So it was stripping out my Only local images are allowed. tag. The point here is make sure that whatever field you are hijacking via pre_render is not filtered or your output may not be what you want.

kenorb’s picture

Priority: Normal » Minor
Issue summary: View changes