Apache Solr Views for D7?

Tagging.

+1

Here's a first pass at a patch to update the DRUPAL-7--1 branch of apachesolr_views to the last version of Drupal 7 and Apache Solr. It now depends on apachesolr_search instead of apachesolr because it looks like the code it was relying on got refactored out of the core module.

I haven't been able to test it fully, but hope to in the next few days.

Second pass. This gets Apache Solr views to display. Facets based on taxonomy terms still don't work; I hope to tackle that next. Looks like a pretty significant refactoring might be warranted.

Perhaps efforts would be better directed at Search Api (specifically Search Api Solr) for D7?
It already provides views integration with solr.

It doesn't yet have all the features of the Apache Solr Search Integration module but it could certainly be built upon (or Apache Solr Search Integration could have integration with Search API added).

#7, that was my suspicion: I wrote the patch above before hearing about Search API, and I haven't had a chance to play around with it yet. I'm certainly not working on trying to patch to provide Drupal 7 for Apache Solr Views anymore.

I've not yet started to test it myself either.
That said, everything I've heard/read/watched is very promising. Exciting, even.
Plus its being actively developed and making a lot of progress.

I'll be starting to try it out this week (tonight, hopefully!) after which I may give a bit more feedback here for the benefit of others looking for D7 Views & Solr integration.
Or you can see this short review by robertDouglass.

After talking to one of the primary maintainers of apachesolr, he's told me (while skirting the issues of search api) that any future views integration for apachesolr would happen in apachesolr module itself. That still however leaves a lot of work to be done there.

I'm also in the process of testing SearchAPI -- and have a few open issues there to see if all the functional from this module can be duplicated in that one. I'd personally rather leave apachesolr in the dust entirely and move to SearhAPI for its flexibility. But, time will tell if the large agencies putting resources into the apachesolr modules are willing to shift focus to SearchAPI and risk loosing past investments to the sands of time.

/Robin

Keeping track of this thread

I am planning to add Views integration to the main apachesolr module for 7.x, so perhaps we can converge on that work and abandon this module for 7.x?

subscribe

I've finally gotten around to testing Search API and it really is brilliant.
It clearly doesn't quite match up to Apache Solr Integration module for Solr-specific functionality but in general it is incredibly powerful and more-or-less does exactly what you want it to.

Now that I've got my head around how to use it I can whip up a custom solr-powered search page (with facets, of course) in in barely any time at all. I can make a more complex view page based on a solr index (again, with facets!) in just slightly longer than the time it'd take me to create the same view page normally (i.e. not based on solr and therefore without facets).

I'm still very much exploring what it can do but it seems like there really aren't many weaknesses. The views integration is constantly being improved so whilst its not 100% yet I am confident it will get there in time (and it really is pretty close already).

So if you wanted to invest time on a D7 views integration solution for solr, you could do a lot worse than contributing Solr-specific functionality in patches to search_api_solr (or in further contrib modules, where appropriate).
Indeed, I wouldn't be suprised if a large proportion of use cases will find everything they require is already there.

The only downside to search API seems to be its reliance on entities and it performing node loads of search results resulting in a higher db load (this from another developer and not from first-hand testing). Which means it might be slower than apache_solr, and wouldn't work as well in a multi-site environment.

I do, however, prefer the implementation to apache_solr currently.

With regards to the reliance on entities, I understand that this is something that drunken monkey is planning on fixing for Search API 2, which he wants to work on for GSoC this year (so from May, if his proposal is accepted). See his proposal for details.
It might also be being worked on by Palantir (see #1064884: Add support for indexing non-entities) already, but I must confess to having only skimmed the issue.

subscribing

Search API has some of it's own issues. The views integration, while it works, is not great. The faceted_search module in D6, so far did the best of of integrating with views.

With that said, I'm currently shopping around for a faceted search module for D7. I need my version to work with views. I'm currently using Search API, but it's got issues. I'd love to be able to test out Views integration for ApacheSolr, but currently it doesn't appear to exist.

Indeed - currently doesn't exist. Need someone to help dive in an determine if that should be implemented via facet API or via apachesolr directly.

I've never had a direct use case for Views integration, so it would be helpful to have input (and patches!) form someone who does.

I personally believe the facets in ApacheSolr are useless.

FacetAPI seem to be the only 1/2 decent facet available for Faceted Search in D7.

For my sites, I don't use faceted search to search all the content. Usually it's just for E-Commerce related product catalog filters & sorts. So being able to use view to limit which products get shown is helpful. Additionally I get the added benefit of being able to style views, instead of custom theme hooks.

@j0rd - have you used the current D7 code? There are no facets at all in apachesolr - they are only provided via Facet API.

If you are not satisfied, please help improve the code.

Yes. I am quite happy with how those facets work via the FacetAPI. I assume a views integration with ApacheSolr would use the same facets. As for where to implement views integration (ApacheSolr or FacetAPI) I'm not exactly sure where you would do this.

I believe I miss understood comment #20

It seems like at the moment for D7 the 'state of the art' for Solr searching + facets + views is SearchAPI plus FacetAPI. I haven't personally tried either one, just giving an impression based on reading issues, blog posts, etc.

One of my challenges before testing anything is how to move from D6 apachesolr_views, since I have significant site functionality rolled into it and growing. For better or worse this has required implementing different things like numeric argument support and lots of customized indexing and sorting. My biggest use case over core Solr search for views integration is to add lots of customized 'search' pages on the site, each either taking arguments from the user or starting with default arguments.

For example, in D6 land I have an apachesolr_views page showing node content, where the view has a default argument of the logged in user (so it only shows their content), a view filter that excludes nodes not tagged with specific taxonomy terms (IE, only show content that has been tagged), and 3 different sorts applied so the results are sorted by taxonomy weight and two different taxonomy vocabularies. I have a fair amount of these views sprinkled in various places as custom windows into the node world.

pwolanin how do you see this proceeding? Would you want to start with the apachesolr_views code included here and manage it as a sandbox issue for the core apachesolr module? Or start from scratch? If you want to start from scratch, how do you see it evolving? Start with an issue that just talks requirements and approach? I'm going to have to figure this out eventually, so it's worth at least starting to talk about it.

Things may have changed in D7 branch in the past few months since I last worked on a D7 port to this (but it doesn't look like it), but the major stumbling block I encountered was Apache Solr Views' assumption that taxonomy terms were in the same format they were in Drupal 6.

So presumably the biggest hurdle to getting a working port would be ensuring it plays nice with the Field API, particularly taxonomy. Everything else was straightforward renaming (see patch in #5). Any major architectural issues could always be tackled after the straight port.

Subscribing.. Mostly noting that the last patch is from December and I'm not sure from this thread if this module should just be marked obsolete in D7.

I wouldn't call it obsolete, as there seem to be a few people using it. I think the broader issue is that the community at large has not selected a particular direction for solr views in D7 yet. It's probably because nobody with enough time or enough of an itch decided to scratch it yet.

any updates? Can I use view to customize results?

my gut feeling is that ApacheSolr as a search engine, and a paid-service, will become just one option in a nebulae of search

with ElasticSearch, CouchDB and more exotic search technologies like InfiniteGraph coming onto the scene, SearchAPI is really the only smart long-term investment

ApacheSolr should be a commodity. Although at the moment it seems to be an easy sell as a premium add-on for hosts, networks and search service providers, but with an easier installation method and a more robust and stable version in 3.x, in combination with Nutch, its just going to become a very low value necessity.

SearchAPI is a diamond in the rough. Why commit to a single search technology? that's simply not Drupal

You know it makes sense.

After applying patch #5 on current 7.x-1.0 branch and trying to create a view I got error:

Fatal error: Class apachesolr_views_query contains 26 abstract methods and must therefore be declared abstract or implement the remaining methods (DrupalSolrQueryInterface::getFilters, DrupalSolrQueryInterface::hasFilter, DrupalSolrQueryInterface::addFilter, ...) in /srv/www/apachesolr_views/sites/all/modules/apachesolr_views/apachesolr_views_query.inc on line 990

PHP 5.2.10-2ubuntu6 with Suhosin-Patch 0.9.7

subscribing

SearchAPI has all sorts of nuances. I'd personally like to see a simple solution of views integration with solely Apache Solr.

subscribing

subscribing

subscribing.

Here is first version of the port to D7. Architecturally it is more like rewritten version of the apachesolr_views module.

Biggest change is that apachesolr_views_query class doesn't implements Drupal_Solr_Query_Interface but call solr query during execute method.

At the moment we can use only fields in solr index (I mean we can only add solr fields to the view).

What should be tested is:
- adding / removing fields to the fiew
- create filter, exposed filter
- add sorts to the view

This is far from full port of the D6 version but it already does something that can be used by everyone :)

Btw please remember to index your text fields if you expect them to appear in the list of the available fields. The full list of the apachesolr fields you can view on your solr admin pages http://localhost:8983/solr/admin/schema.jsp (in case of single core).

To index your text fields you should implement hook_apachesolr_field_mappings_alter in separate module:

function hook_apachesolr_field_mappings_alter(&$mappings) {
$mappings['text'] = array(
'indexing_callback' => 'apachesolr_fields_default_indexing_callback',
'index_type' => 'string',
'facets' => FALSE,
'multiple' => FALSE,
'name_callback' => '',
);
}


I will very appreciate any feedback on the patch and lets start developing functionality of this module for d7.

+1 for not extending Solr query anymore. That caused much code to be rewritten and opening a can of worms by putting in one class a Views Query and a Solr Query.

Also liked that only indexed fields show up, this saves some time in debugging for people not used to manually index fields to Solr.

Subscribing. Will help test the port as soon as I can.

@ygerasimov - great to see something working. Given that this module is unmaintained, I'm still thinking we should roll the code into the main module. What do you think?

+1 roll-it.

This module is unmaintained? Rats. Guess the status should be changed from "Actively maintained" then.

My use cases are the same as in #24 - being able to show a View of Solr objects with various contextual arguments is something I'd love to be able to have in D7, as that's how a lot of our D6 site is currently running. If this module's no longer maintained, then I'll be seeking a viable alternative.

@pwolanin, I completely agree. I see a lot of places where the code should be improved but more people will try it, faster we will get to stable code. So big + for pushing the code to 7.x branch and creating dev release.

dstuart is the active maintainer for the module at the moment. Latest commits as of 5 weeks ago and he has been active in the issue queue. I would not call this abandoned by any stretch - it's a critical module for me and if help is needed I'd pitch in and contribute to maintenance as needed.

+1 to create the dev release

I review patch and cut a 7.x.dev release early next week

Regards,

Dave

Sounds great. +1

ok, if dstuart is taking it over, I'll take that characterization back. The prior maintainer disappeared for a long time.

someone opened an issue pointing to this module on github: https://github.com/markbirbeck/apachesolr_passthru

Looks like it's implementing some Views integration, so might be worth seeing if there is any useful code to take.

Module in repo https://github.com/markbirbeck/apachesolr_passthru has apachesolr_views as dependency and extends apachesolr_views_query query class. So this module doesn't provide integration by its own.

Hey,

Reviewing the patch and on line 91 of the apachesolr_views_query.inc you are implementing ApachesolrViewsSolrBaseQuery yet I can't find that class anywhere in the patch. Also you reference handlers/views_handler_area_apachesolr_views_results.inc in the handlers which is also not included.
Can you provide this code or is it yet to be implemented

Regards,

Dave

@dstuart Thank you for the review. Good catch. That is code that is my project specific. I just haven't cleaned it up properly.

Please find attached patch.

I have removed reference to handlers/views_handler_area_apachesolr_views_results.inc and ApachesolrViewsSolrBaseQuery should be SolrBaseQuery that is part of apachesolr module.

Ok, so I have reviewed and commited the patch at #36, I haven't removed all of the handlers and plugins as they will be a good starting point for all of these features. I have also added the D7 dev verison the to the project page.

This is a good starting point to make this really good D7 module thanks to @ygerasimov for the patch.

More to come soon

I am not sure whether keeping old handlers is good idea as we integrate apachesolr with views in different way and now we do not use any of old plugins in hook_views_data(). Also we do not have any relations to any other fields from any other tables as well.

I would recommend to remove those handlers in order not to mess around other developers who will look into the code.

Basically now we allow views to get only indexed fields and nothing more then that.

For the todo list I would have following tasks:
1. Handling multivalue fields
2. Test creating different views, getting different fields, sortings etc.
3. Write documentation (specially about hook_apachesolr_views_field_names_alter)

@dstuart you can also give me commit access for this module. I will keep developing only 7.x branch.

OK point taken, I'll remove the extra handlers, although things like MLT, I think we should keep. I would love to give you commit rights but unfortunately Scott Reynolds is the only one I know of with maintainers access.

added ygerasimov to committers and gave dstuart admin maintainers permissions.

Thanks Scott

Thank you guys. Lets close this issue then. All new tasks/bugs for 7.x branch should be opened as new issues.

Would you please give a link to download the latest version(for drupal 7). Urgent need. Thanks.

@x_tra06 please use dev version of the module for drupal 7: http://ftp.drupal.org/files/projects/apachesolr_views-7.x-1.x-dev.tar.gz

new version please?

Great work @ygerasimov

It's a shame that it's not working in Drupal 7.