Random sort in Views

I'd call this a bug in Views, who incorrectly label some backend-specific components as "global". There is no way in the Search API (and, therefore, also in it's Views integration) to retrieve random results.

You could write an extension to the query class which allows this, though, by getting all results and then randomly loading the data of only some of them.

So, this should most probably fixed in Views, by removing the "Global: Random" sort for non-DB backends.

I'm having this issue in 7.x-3.x - i'd like to randomize my search results - but I am getting this error.

I found a solution to have a random sort order for results with the solr module.

In schema.xml, there is already the neccesary field type (name="rand").

I added the line

<dynamicField name="random*" type="rand" stored="true" />

as proposed in http://realize.be/random-results-apache-solr-and-drupal

then I implemented hook_search_api_solr_query_alter and added

    $seed = rand(1, 200);
    $call_args['params']['sort'] = 'random_' . $seed . ' asc';

works great, I limit the number of displayed nodes, on every page reload I get a random set of nodes.

Wouldn't it be somehow possible to add this feature to the solr-module, so that you can select it directly in views as a sort criterion? If you point me how to do it, i would write the patch.

Here are two patches to make Global:Random work for sorting in a random order.

The first patch is for the module search_api_solr, the second for search_api.

it works fine for me! - Thanks k4v - This solution seem to be currently the single chance to get a random ordering on result side.

Should we really be storing the dynamic value? I'm going to go dig up some docs but it seems like that would give you the same random sort every time right?

Ah okay so the random call in service.inc handles keeping the results unique for each call but this bit of documentation does stored="false": http://lucene.apache.org/solr/api/org/apache/solr/schema/RandomSortField...

hmm you are right "storing" is not necessary

so heres a new patch without the url-comment and stored="false"

I'm getting: '400' Status: Bad Request after applying the patches. Cleared and re-indexed. Ideas?

M

Hi,

it sound like your query is to long. See #1276970: Large queries break solr search

@MHilliker: are you using Sarnia? I found this too. Adding the same 'search_api_random' field to the Sarnia service's getFieldNames() fixed it--I've committed that to Sarnia, so if you apply the Search API patch from this issue, you should be able to get random sorts.

Here are re-rolled versions (updated comments, patch name format) of the other two patches from earlier in this thread, to Search API and Search API Solr. I find that these allow me to do random sorting with Search API, Search API Views, and Solr.

I applied the patches in #17 but get the same result as #15. Using only SearchAPI and SearchAPISolr.

Ah, the random_* field hasn't been added to search_api_solr's schema.xml file.

Here's an updated patch for search_api_solr that includes the schema.xml snippet from comment #5 above. You'll need to put this updated file in your Solr setup, too. I've included the search_api patch again for the sake of completeness.

Thanks. Will give it a try.

#19 Worked for me!

Will this be committed?

#19: 1197538-19-search_api_solr-random_sort.patch queued for re-testing.

#19: 1197538-19-search_api_solr-random_sort.patch queued for re-testing.

i think the first patch fails testing because it has to be applied to the module search_api_solr, not search_api. Should I open another issue for the separate patch?

None of these patches worked for me, I get FAILED in ssh.

These patches worked great for me. As becw mentioned, the first patch is for search_api_solr and that's why it fails testing in this issue queue.

I admit, the current patches are very appealing in their simplicity. However, we cannot just introduce a new special sort field that service classes will have to know. Especially since the module is now stable and should therefore keep not backwards-compatible API changes to the absolutely necessary minimum.
The current approach will break all backends except Solr. A better approach would probably be to define a new "search_api_sort_random" feature, or the like, which allows random sorting via an option. Though I admit that the current approach is cleaner and more functional – we sadly can't just add features anymore.

Also, this is definitely a feature request. (And as I'm currently busy solving accumulated bugs of over half a year, it'll probably take a while till I get to it again. Sorry!)

Maybe you could add it in the next version? Or do you have a good example how to do the "feature" the right way? Thanks for the fantastic modules ;).

#19: 1197538-19-search_api_solr-random_sort.patch queued for re-testing.

I have solved the problem.

the patch for solr_api_solr not work.

i have add this line:

'search_api_random' => 'random_' . rand(1, 200),

at line 480 of this file:

search_api/solr/includes/service.inc

In this way the services views can create random sort order.

it would be to create the patch for the search_api_solr module.

pippopeppe: see #27

Maybe you could add it in the next version? Or do you have a good example how to do the "feature" the right way?

I don't think we should generally add this as a built-in feature, since some backends might have problems implementing it and it's by no means a must-have for searches. So in my opinion, it should stay an add-on feature in later versions (D8, for example), too.

I don't really have a good example for doing exactly this kind of feature, but it would work like other features – e.g., search_api_mlt, also defined in the Search API Views module. In the add_orderby() method you check whether the index's server supports the feature and, if it does, add the random sort. (Otherwise, you'll probably have to throw an exception or at least display a warning.) Documentation for the feature has to be added in the README.txt file, and also a short description to the handbook page (after the patch was committed, of course). As the format, since the query class doesn't allow arbitrary fields for sorting, I guess you'll have to use an option you pass with the query to override the normal sorting. In my opinion, this should also support having the random sort not as the primary one, so a bit of trickery will probably be necessary there.
(On second thought, since an option would really be complicated, you could also set the sort directly by using the reference returned by &getSort().)

For Solr, just adapt the patch code to the new way of passing the random sort, and add the feature's ID to supportsFeature().

When I try to apply the patch to search API, I get

"unknown field search_api_random." within views.

19: 1197538-19-search_api_solr-random_sort.patch queued for re-testing.

I had not time to implement the proposed solution. Attached you find a rerolled patch against the latest version of search:api and search_api_solr

Since this is a feature request and drunkenmonkey made it clear that 7.x-1.x is stable and should not introduce API changes I'm moving this up to the D8 chain.
If someone would be able to change the summary so it represents the ask and why, that'd be awesome.

As long as it doesn't change any APIs but only introduces a new feature, I'd be fine with it. I'm just waiting for a patch that does that. I also don't think it should become a fixed part of the framework in D8 – this isn't useful enough in my opinion to force all backends to (at least try to) support it.

Rerolled the first patch from # 37. Posted a rerolled version the 2nd patch to a new issue in the search_api_solr queue. #2313591-1: Random sort in views

This should still use a feature, not a hard-coded field.
Also, #2313591: Random sort in views already references this issue as its parent, we don't need to add a reference back.

NOTE FOR ORDINARY PEOPLE WHO JUST WANT TO ADD RANDROM SORT ORDER:

I have solved the problem and successfuly added random sort order for search api on my site.
My steps:

1. I have applied #37 patch to search_api_solr module.
2. I have applied #40 patch to search_api module.

Hi,

I have the same issue with Elastic Search ; I'm not using a community module as a connector.
How can I solve this ?

Thank you.

also, that about search_Api_db?

#45 working for me

I have the same issue with Elastic Search ; I'm not using a community module as a connector.
How can I solve this ?

Apply the patch from #40 to the Search API, then adapt your service class code to look for sorts on the search_api_random field and add a random sort in that case. As to how to do that with Elastic Search: no idea, search on the web.

I rerolled #40's patch against develop.

Also i rerolled #37 search_api_solr patch against 1.6

Hi!

I patched the search api module with the #40 patch, but it didnt work for me :(
any suggestion? I not use the apache sorl module to patch it

I not use the apache sorl module to patch it

Sorry, I don't understand that.
If you aren't using the Search API Solr Search module, this won't work for you, you'd have to implement it again for the backend you're using.
If you are using the Search API Solr Search module, then you need to patch it for this to work.

What about search api random sorting with pager?
In case of a simple view, views_random_seed is used, but in this case it doesn't work.
So, if i have let's say 100 results in 5 pages, there are duplicates.

Thanx!

Hm, you're right, I don't think this will work with a pager. The question would then be whether this is a valid/usual requirement, or randomly sorted views usually don't use a pager anyways. Solving the use case with pager would of course be even more difficult, as you would have to pass the random seed value along to the other pages.

Yeah, it looks like the random seed is changing on a per page basis... one approach would be to add a new handler that is similar to the one created for https://www.drupal.org/project/views_random_seed

The question then is would this be something that should be added to the module itself or to the views random seed views data module. If the random is supported by the searchapi modules, the random seed module could potentially take care of the paging issue.

I was able to get paging working by slightly adjusting the handling of random and utilizing views random seed module with a patch found here: https://www.drupal.org/node/2461099

Essentially, the difference is where the random is passed. If it is passes as the order we can check for its existence and then if it isn't there add it. This allows for the random seed module to also utilize the orderby function and have the search follow. These patches do not depend on views random seed, but allow for it to be utilized.

Also, I saw that random was in the schema.xml already, so I added the false flag instead of duping the dynamicfieldname.

Steps to get working:

1) Apply both patches to appropriate modules
2) Download and enable views_random_seed
3) Patch views random seed with patch found here: https://www.drupal.org/node/2461099

The previous patch is slightly problematic because of the assumption that order isn't used... The thought is that ASC/DESC do not matter because it is random. It is similar to paging in the sense where if you have an exposed filter and hit the asc/desc filter, then you end up possible duplicates and wouldn't get the same results if you went back to the original filter. I do not think that is an issue, however, let me know.

Soo... what's the status on this ?
I have a view showing results from search api + database search (important: not solr).
I want to randomize the view results.

At this point I cannot get this working. If I use patch from here (not the solr patch because I use db search) and apply global: random to my view, I get: Trying to sort on unknown field search_api_random

If I just use the module views random seed (current or dev version) and sort on 'random seed' I do not get random results (tried setting to random for each user, and change it every 5 minutes. No matter which user or how long I wait, I always get same order instead of random). If I try to apply the patch for views random seed to latest dev version of that module, the patch does not apply correctly.

If anyone has gotten this working (again, i'm not asking about search api with SOLR, i'm asking about search api using database search): please be so kind to tell which modules, versions and patches that are working for you.

#50 working fine. Thanks ..:)

Hi,

#57 with the latest dev version (7.x-1.14+6-dev) does not work for me. PHP error is invoked:
"PHP message: PHP Parse error: syntax error, unexpected '*', expecting function (T_FUNCTION) in ../search_api_views/includes/query.inc on line 146"

#50 with latest stable release or dev one does not work as well. No php error, but same on flyke, the "Trying to sort on unknown field search_api_random" error show up.

bisw, which version of api_search (and api_search_db) do you use to make it work ?

same here...

Rerolled patches are attached.

Main changes (as compared to @timodwhit):

• Updated comments for search_api_random field
• Moved the setup of SOLR random field usage (random_" . rand(1,200)) into search_api_solr module, since it does not make sense to use it in the generic search_api module
• Removed store="false" edits from schema.xml, the field is never used to STORE data since it is used as a "virtual field" just at query time.

Other points:

• The addidtion of the search_api_random field needs to be ported also to DB-based engines (i.e.: search_api_db).
• The integration with views_random_seed module is on the go, but it will requires some "hacks" in the sorting procedure

Updated patches, tested with SearchAPI + SearchAPISOLR + ViewsRandomSeed.

Changes:

• Using add_orderby() $params to handle seed value (allow ViewsRandomSeed integration)
• Allowed the integration with Views Random Seed module (see patch here: #2461099-2: Search API View Query Random Seed )

How to use:

1. Patch search_api module (7.x-dev release)
2. Patch search_api_solr module (7.x-dev release)
3. In your SearchAPI view (based on Solr index) add "Global: Random" sorting

My patch for views_random_seed allows to use the "Global: Random seed" sorting, that, in turn, uses the current random sorting provided by SearchAPI by providing a pre-computed seed.

Removed unrelated patch, for search_api_solr patch, see: #2313591-4: Random sort in views.

Here is an updated patch from #57 without the fatal error:

@ timodwhit: There are already several newer patches, posting updates of old ones is just confusing.

@ thePanz: Good progress, thanks! However, once again, you can't just require all service classes to now support an additional field for sorting. Instead, please specify a feature for this functionality and mark the Solr service class as supporting it (supportsFeature()).

Other than that:

1. +++ b/contrib/search_api_views/includes/query.inc
   @@ -136,6 +138,22 @@ class SearchApiViewsQuery extends views_plugin_query {
   +   * Implement the same add_orderby() method as views_plugin_query_default so
   +   * that Views' "Global: Random" sort can be used.
   +   *
   +   * @param array $params The set of parameters for the search_api_random:
   +   *  - 'seed': a predefined seed for the random generator
   

   This doesn't follow Drupal's documentation coding standards.

2. +++ b/contrib/search_api_views/includes/query.inc
   @@ -136,6 +138,22 @@ class SearchApiViewsQuery extends views_plugin_query {
   +  function add_orderby($table, $field = NULL, $order = 'ASC', $alias = '', $params = array()) {
   

   Should have an explicit public modifier.

@drunken monkey, sorry about that. Just didn't get around to seeing the issues until today. If people are interested, those patches, should apply and work. We're currently using them in production and sorting random. My apologies for the confusion.

Updated (and rerolled) patch.

Adding the "Random sort" as a SearchAPI feature.

Looks a lot better, thanks!
I still had some corrections (some of them just taste, though), they are contained in the attached patch.
Also, the feature needs to be properly documented in README.txt – I've added a @todo there to that effect. It would be great if you could then also add the feature to the list/matrix of existing features, then I think this would be RTBC.

(Whether we should or shouldn't abort the whole search when random sort is used but not available is of course up for debate. I'd say, though, that this is hardly ever that critical. If you want to abort, though, you need to pass the error message through t()!)

In any case, thanks again a lot for your work on this, seems this can soon finally be resolved!

Oh, also, just realized this: why did you change the "magic" field name from search_api_random to search_api_random_sort? I'd say the former is fine, too, and more concise.

Attached a re-rolled patch with your edits.
Regarding the field name changes: I wanted it to be more specific just in case in the future a new "random" feature needs to be implemented.
Feel free to rename the field if you like, just take into account that the search_api_solr patch must be updated accordingly.

Cheers

Thanks for providing the feature definition!
I've just polished it a bit and changed the field name back to search_api_random.
The patch is now RTBC from my point of view, so if you are also OK with it, I can commit. (As you say, we'll have to adapt the Solr patch before committing it, though.)

Tested and working Ok for me.
SearchAPI Solr module patch available here: #2313591-14: Random sort in views

Excellent, great to hear.
Committed.
Thanks a lot again for finally fixing this issue!

I guess this should also be ported to D8?
(Most of the Views integration is missing anyways, but sorts are already working, I think.)

How does this apply to random sorting not working with Search API DB?

I get the following Watchdog message instead:

Tried to sort results randomly on server ... Server which does not support random sorting.

Yes, you can't test it with the DB backend, and I don't know whether the D8 version of Solr already supports this (probably not). But the code should still be ported.

And here it is!

Instead of hard-coding random sorting support in the query class, I opted to take the more elegant and extensible route in D8: #2604874: Move query validation logic to backend plugins. (That patch is therefore needed to test this, too.)

Oops, sorry, the one posted in #89 was based on #2601224: Add Views filters.
Here is the correct one for current HEAD.

I still cant get this to work, can someone please provide necessary info ?

I want a view based on search api, db search, to show me 5 random products from my webshop.

important note: i'm using NO SOLR search but database search (search_api_db).

- using latest dev of search api (7.x-1.x-dev)
- using latest dev of views random seed (7.x-1.2+0-dev)

-> If I sort by 'general: random seed': no errors, but always same nodes, so random is not working
-> If I sort by 'general: random': I get the error: 'Tried to sort results randomly on server xxxx which does not support random sorting.'

Please tell me what I have to do to get this working.

Do I need a patch for either search_api and/or views random seed ? If so, which is the working patch I should apply ?

Many thanks to those who can fix this !

@flyke: as the message says: your server (Database) is not supported.
Only SearchAPI Solr has been integrated to work with random sorting.
Please open a new issue for that.

Committed.

This is not supported when using Search Api DB. I use the following code to shuffle my results:

/*
 * Implements hook_views_post_execute().
 */
function YOUR_MODULE_views_post_execute(&$view) {

  // For this view shuffle our $view->result.
  if ($view->name == 'my_view_name') {
    shuffle($view->result);
  }
}

You can also add extra logic if you only want the shuffling for a page of that view for example. Hope this helps someone out.

@Fons Vandamme I did the same in order to get a random sort.

But my prob comes when I try to sort it by title/date or whatever.
I get the same messasge Tried to sort results randomly on server DB Drupal Server which does not support random sorting. and what's worse, I get no results.

function mymod_search_api_db_query_alter(SelectQueryInterface &$db_query, SearchApiQueryInterface $query) {
  $index = $query->getIndex();
  if ($index->machine_name == 'node_index') {
    $query->sort('title', 'DESC');
  }
}

I have the latest dev versions of search_api, search_api_solr, and views_random_seed.
Regular random sort works perfect.
But when it comes to random seed sorting, nothing happens. No errors, and no random sorting.
I tried this patch for random seed: https://www.drupal.org/node/2461099#comment-10303439
Nothing.
Any ideas why it's not working for me?

Thanks!