The pluggability of both indices and servers is really fantastic with this module, and being able to switch which index uses which server is quite easy from the UI. The area where there may be an opportunity for some improvement is in the handling of index-server pairings during deployment and general CI. In this regard I'm thinking that it would be very handy to have some search server support in the drush commands, specifically the ability to change which server an index uses.

We have a common search index used across dev/stage/prod, and because it's a fairly complex index, we manage it with features. This works great. The trouble is, we also use different server back-ends across dev/stage/prod (stage and prod use separate solr cores and dev sites use the DB back-end). Because the server value is saved in the index configuration, it's also part of the feature, so when we deploy that feature across dev/stage/prod we also must manually change (override) the server setting on an instance-by-instance basis each time we deploy changes. This can be cumbersome via the UI and can even lead to problems if we accidentally load something that uses search API (like a view) in the precious window between deploying and toggling the right server value on an updated instance.

Maybe there's a better strategy to all this, and we've tried a few different things, but we always come back to this need to manually override the index feature on an instance-by-instance basis. I also assume this must be a situation other users encounter (e.g. a staging site can't use the same index as prod unless that index is actually in the database).

So, if we could change the index's server value with a drush command, during deployment, that would be gravy. I've been experimenting with this, and hope to have a patch soon. I'd also be interested to know if others facing this situation have other ways to deal with all this.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

rjacobs’s picture

There may also be some common ground between this issue and #1762146: Ability to switch between indexes / or have the same index on multiple backends.

rjacobs’s picture

Status: Active » Needs review
FileSize
3.71 KB

Attached is a patch that adds the described feature. With this it's possible to set a server on an index with:

drush search-api-set-index-server index_name server_name

or:

drush sapi-sis index_name server_name

It would be interesting to know what people think... both about the patch and about the general need for this kind of feature.

rjacobs’s picture

Yeah, so the idea of maintaining different search servers across instances, but sharing a common index (that's in a feature and version control) has proven to be a very tricky thing.

We've now abandoned the idea of toggling the server setting with drush during deployments, as it's proven too hard to do cleanly (especially when swapping from solr to DB servers). The alternative that seems to work (and that I think we'll adopt) is setting different search servers on different instances but using the same machine name for each. It looks like the only server detail stored on the index conf is the server machine name, so once we have appropriate index/server pairings setup on a instance-by-instance basis, we can keep all the details of the index constant across all environments (and maintain it in a common feature). We just have to be careful to selectively bypass certain tables (like search_api_server) if we ever sync data upstream and to keep all server settings out of our features.

So this means that drush feature added in the patch is not so important, at least to us. Regardless, perhaps this would be of value to others in some other context? I guess if I don't hear anything back on this in a while I'll just close it under the assumption that it's not a useful feature addition.

drunken monkey’s picture

Using the same server machine name for all of those seems like the proper way to handle this, I think. But I can understand that it's problematic. If you are only using Solr servers then Search API Override might be a good solution – but if there's also a DB server involved, it gets really tricky.

Regarding the patch, I can't really say how useful it would be to people in general. I guess if one or two others show interest here, I'll commit it.
In any case, thanks for your contribution!

rjacobs’s picture

Thanks for confirming that the technique we arrived at would be best practice given the circumstances.

Search API Override is an interesting approach but, as you noted, the fact that we have DB servers in the mix makes that an incomplete solution. Because the table mappings (field > search_api_db table) are stored as part of the server configuration for a DB server, but are still dependent on the index settings, things can get pretty tricky whenever swapping servers in our case.

For now we'll see how things go with the common machine name technique. No matter what I think the work needed to figure this out was still easier than setting up new solr servers/cores for each and every dev space that may need to be spun-up (hence the reliance on search_api_db).

13rac1’s picture

Status: Needs review » Reviewed & tested by the community

This patch works perfectly (with minor offsets), although I may end up using Search API Override. I'm still looking into the ideal process.

I have a dev, staging and two live environments. The weekly data migration is updated (174k nodes) on staging then synced to the disabled live environment. The single live Solr index is cleared to removed deleted nodes, re-indexed, caches (Drupal and Varnish) cleared, then the live environments are flipped (Apache points to a symbolic link.) The removed nodes are deleted from the staging Solr index by Migrate, but not the live index hence the required index clearing. I wish it were less complicated, but I'm not seeing a better way. The problem is the shared Solr server means clearing the index results in two hours of broken search results. Both of these solutions will fix the problem. I just must decide which is easier to maintain. Thanks!

drunken monkey’s picture

Status: Reviewed & tested by the community » Fixed

OK, great, if it helps others, too …
Committed. (Only applied a few style changes before.)
Thans again for your patch, rjacobs! And thanks for reviewing, eosrei!

  • Commit 0859f9d on 7.x-1.x authored by rjacobs, committed by drunken monkey:
    Issue #2233749 by rjacobs, drunken monkey: Added drush support to change...
13rac1’s picture

Great thanks!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

drunken monkey’s picture

Version: 7.x-1.x-dev » 8.x-1.x-dev
Status: Closed (fixed) » Patch (to be ported)

Needs to be ported to D8.

drunken monkey’s picture

Issue tags: +Novice
cristiroma’s picture

Assigned: Unassigned » cristiroma
Status: Patch (to be ported) » Needs review
FileSize
2.49 KB

Added patch for D8

drunken monkey’s picture

Status: Needs review » Fixed

Looks good, thanks!
Committed.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.