Add support for facet queries

This functionality would be really nice to have, you're right. The current way date facets are handled might be portable across backends, but is just unusable for larger sites. Having this functionality available for your module would therefore definitely make sense.

However, I'm not sure this needs to be this Solr-specific. Why should your module need to be responsible for creating a Solr filter query? I think with just a few changes we could easily make this compatible with other backends as well (unless I'm missing something). There's no real reason why we shouldn't implement range facets/facet queries in the database backend, too.

The way this could/would work in my eyes (and, again, I might be missing something, as I don't completely understand the Facet API side of things):

• Define a Search API "feature" ID for your module so backends can properly advertise their being compatible with it. Then use $index->server->supportsFeature($your_feature_id) to check for it – if the backend is not compatible, you can just fall back to the old, sucky "sum up individual timestamps in PHP code" way. In your module's documentation, properly document what is needed to support this feature (which is detailed below). (Also add your feature to the handbook.)
• Backends supporting the feature would be required to recognize a few more options for incoming facet options. Instead of solr_facet_query, this should just contain the minimum, maximum, step size and whatever more is required to create the correct range queries. (By the way, why aren't you using Solr's range (or date) facets functionality instead of raw facet queries?) It's probably best to just make this one additional option, date_facets, which is an array containing this information. (Good practice, to avoid collisions.)
• Pass the additional option as you do now, just defer creating the actual Solr query to the Solr service class. Use PHP strtotime() date math to create your start and end points. Not quite as powerful as Solr's, sadly, but it should be enough. You can also make the new facet option's settings recognize some special syntax/values, if that makes it easier. Just don't make it too hard to parse in PHP.

I think with that we should retain almost the same functionality, at least as far as ranges are concerned, but make it easy (or at least as easy as possible) for the database backend to support this later, too.

could not apply to search_api_solr module.
checking patch service.inc..
error:service.inc:No such file or directory.

Apply the patch manually, it works perfect.
here the the patch apply to dev version.

Re-rolled the patch in #5 to work with more recent versions of Search API Solr (7.x-1.11)..

Cross-posting this patch to Date Facets issue #2128517: Properly determine facet count when using search_api_solr

#6 works great for me in combination with #2128517-14: Properly determine facet count when using search_api_solr

Still "Needs work" because of #2.

drunken monkey wrote in #2:

However, I'm not sure this needs to be this Solr-specific. Why should your module need to be responsible for creating a Solr filter query? I think with just a few changes we could easily make this compatible with other backends as well (unless I'm missing something). There's no real reason why we shouldn't implement range facets/facet queries in the database backend, too.

Backends supporting the feature would be required to recognize a few more options for incoming facet options. Instead of solr_facet_query, this should just contain the minimum, maximum, step size and whatever more is required to create the correct range queries. (By the way, why aren't you using Solr's range (or date) facets functionality instead of raw facet queries?) It's probably best to just make this one additional option, date_facets, which is an array containing this information. (Good practice, to avoid collisions.)

You are right, Solr provides built-in range facet support via facet.range. But those ranges can only be of a fixed gap size (arbitrarily sized buckets were discussed, but not implemented as far as I can tell, see https://issues.apache.org/jira/browse/SOLR-2366). The same holds true for facet.date.

On the other hand, facet.query is not just for range facets, the queries can be any filter, e.g. facet.query=color:("blue" OR "green"). And as the date_facets module lets one configure arbitrary ranges, I think facet.query is the (only?) way to go.

As I need multiselect functionality for my custom range facets, I enhanced patch #6 to support the OR operator and make facet filters inclusive.

OK, that makes sense. Passing arbitrary ranges would still be possible in a unified way, I think, but I guess having a general mechanism for any kind of facet query also makes sense.
Just a few corrections in my attached patch, please test/review! Then I guess we can really commit this.

However, is it really necessary for you to manually set the facet tags? This should actually now work automatically, since all facet-based filters should have an appropriate 'facet:[FIELD]' tag (which automatically gets included in the Solr filter).
Anyways, I've left that code there, but added a check to make sure we don't try to add two tags (which Solr rejects).

Hi drunken monkey!

Thanks for the revised patch, it works for me.

However, is it really necessary for you to manually set the facet tags? This should actually now work automatically, since all facet-based filters should have an appropriate 'facet:[FIELD]' tag (which automatically gets included in the Solr filter).
Anyways, I've left that code there, but added a check to make sure we don't try to add two tags (which Solr rejects).

As far as I can see, the tag is only automatically set for the "term" query type in SearchApiFacetapiTerm::execute(). search_api's SearchApiFacetapiDate::execute() for example doesn't do this for the "date" query type and Drupal_SearchApi_Facetapi_QueryType_DateRangeQueryType from the date_facets module uses SearchApiFacetapiDate as the base class. I'm not sure about the right place to add the tags...

Is there a place to document the "solr_facet_query" facet query option?

As far as I can see, the tag is only automatically set for the "term" query type in SearchApiFacetapiTerm::execute(). search_api's SearchApiFacetapiDate::execute() for example doesn't do this for the "date" query type and Drupal_SearchApi_Facetapi_QueryType_DateRangeQueryType from the date_facets module uses SearchApiFacetapiDate as the base class. I'm not sure about the right place to add the tags...

Ah, yes, since OR doesn't really work with our normal date facets, we didn't add the tags there. (Though I guess we can easily change that – see attached patch.)
For your own code, instead of doing this:

$this->addFacetFilter($query, $field, $filter);

do this:

$facet_filter = $query->createFilter($conjunction, $tags);
$this->addFacetFilter($facet_filter, $field, $filter);

Just like in SearchApiFacetapiTerm.

Is there a place to document the "solr_facet_query" facet query option?

Good catch, thanks! Added documentation to README.txt.

I also tried to process facet filters a bit when extracting the facets, to make sure they are valid for Search API Facets. However, this might mess up what you're doing with them, so if you already have code for, e.g., correctly converting ranges, maybe we can use that instead? (Or, I guess, we can just be a bit more clever than what I'm currently doing – at the very least, we should remove the 'TO ' from range filters.

Hi drunken monkey, unfortunately 2128537-13--support_facet_queries.patch from #13 doesn't work for me anymore as somehow the filter-tagging code got lost:

-          // Tag filters that contain the current facet field so we can exclude
-          // them when creating facet.query expressions below.
-          foreach ($fq as &$filter) {
-            if (strpos($filter, $field . ':') !== FALSE
-                && strpos($filter, '{!tag=') === FALSE) {
-              $filter = "{!tag=$tag}$filter";
-            }
-          }

Without that, only the selected range facet item is displayed instead of all items as the now untagged filters aren't excluded for the facet query expressions built in the subsequent lines:

           foreach ($info['solr_facet_query'] as $index => $expression) {
-            // Build a local "ex" param to exclude filters regarding the current
-            // field, tagged above. This is necessary to continue displaying the
-            // other (unselected) facet items as if no item had been selected at
-            // all.
             $facet_params['facet.query'][] = "{!ex=$tag}$field:$expression";
           }

For your own code, instead of doing this:

$this->addFacetFilter($query, $field, $filter);

do this:

$facet_filter = $query->createFilter($conjunction, $tags);
$this->addFacetFilter($facet_filter, $field, $filter);

Just like in SearchApiFacetapiTerm

Thanks for the tip!

I also tried to process facet filters a bit when extracting the facets, to make sure they are valid for Search API Facets. However, this might mess up what you're doing with them, so if you already have code for, e.g., correctly converting ranges, maybe we can use that instead? (Or, I guess, we can just be a bit more clever than what I'm currently doing – at the very least, we should remove the 'TO ' from range filters.

Could you elaborate a bit on what code snippets or functions you are referring to, please?

Without that, only the selected range facet item is displayed instead of all items as the now untagged filters aren't excluded for the facet query expressions built in the subsequent lines:

That's what the Search API patch is for. With that, the tags should be appropriately placed for facet filters, too.

Could you elaborate a bit on what code snippets or functions you are referring to, please?

I'm referring to this code, which attempts to transform the Solr query into a value suitable for use as a Search API Facets filter (see the documentation for \SearchApiFacetapiExampleService::search()):

+++ b/includes/service.inc
@@ -1480,88 +1480,112 @@ protected function extractFacets(SearchApiQueryInterface $query, $response) {
+              $term = substr($term, strlen($field) + 1);
+              if (!preg_match('/^(?:[(\[][^ ]+ .*[)\]]|".*"|!)$/', $term)) {
+                $term = "\"$term\"";
               }

So, did you manage to make it work with the patches in #13?

Hi drunken monkey! I finally got around to taking care of this. As there were incompatible changes in the meantime, I rerolled #13 so it now cleanly applies to the latest dev. And after adapting my code according to your suggestions, it works fine for me. Thanks a lot!

Thanks for getting back to this!
Good to hear you could make it work. Then, let’s just commit this now. It’s not in the UI, so few people will use this, and they can just come and complain if something isn’t working for them. (Unless we broke something existing, but I don’t think that’s the case.)

Committed both patches. Thanks again, everyone!

Awesome, thanks!

I have further enhanced this in #3057813: Add support for field-independent facet queries.