The pluggability of both indices and servers is really fantastic with this module, and being able to switch which index uses which server is quite easy from the UI. The area where there may be an opportunity for some improvement is in the handling of index-server pairings during deployment and general CI. In this regard I'm thinking that it would be very handy to have some search server support in the drush commands, specifically the ability to change which server an index uses.
We have a common search index used across dev/stage/prod, and because it's a fairly complex index, we manage it with features. This works great. The trouble is, we also use different server back-ends across dev/stage/prod (stage and prod use separate solr cores and dev sites use the DB back-end). Because the server value is saved in the index configuration, it's also part of the feature, so when we deploy that feature across dev/stage/prod we also must manually change (override) the server setting on an instance-by-instance basis each time we deploy changes. This can be cumbersome via the UI and can even lead to problems if we accidentally load something that uses search API (like a view) in the precious window between deploying and toggling the right server value on an updated instance.
Maybe there's a better strategy to all this, and we've tried a few different things, but we always come back to this need to manually override the index feature on an instance-by-instance basis. I also assume this must be a situation other users encounter (e.g. a staging site can't use the same index as prod unless that index is actually in the database).
So, if we could change the index's server value with a drush command, during deployment, that would be gravy. I've been experimenting with this, and hope to have a patch soon. I'd also be interested to know if others facing this situation have other ways to deal with all this.
Comment | File | Size | Author |
---|---|---|---|
#13 | search_api-add_drush_change_index_server-2233749-13.patch | 2.49 KB | cristiroma |
| |||
#2 | search_api-add_drush_change_index_server-2233749-2.patch | 3.71 KB | rjacobs |
Comments
Comment #1
rjacobs CreditAttribution: rjacobs commentedThere may also be some common ground between this issue and #1762146: Ability to switch between indexes / or have the same index on multiple backends.
Comment #2
rjacobs CreditAttribution: rjacobs commentedAttached is a patch that adds the described feature. With this it's possible to set a server on an index with:
or:
It would be interesting to know what people think... both about the patch and about the general need for this kind of feature.
Comment #3
rjacobs CreditAttribution: rjacobs commentedYeah, so the idea of maintaining different search servers across instances, but sharing a common index (that's in a feature and version control) has proven to be a very tricky thing.
We've now abandoned the idea of toggling the server setting with drush during deployments, as it's proven too hard to do cleanly (especially when swapping from solr to DB servers). The alternative that seems to work (and that I think we'll adopt) is setting different search servers on different instances but using the same machine name for each. It looks like the only server detail stored on the index conf is the server machine name, so once we have appropriate index/server pairings setup on a instance-by-instance basis, we can keep all the details of the index constant across all environments (and maintain it in a common feature). We just have to be careful to selectively bypass certain tables (like search_api_server) if we ever sync data upstream and to keep all server settings out of our features.
So this means that drush feature added in the patch is not so important, at least to us. Regardless, perhaps this would be of value to others in some other context? I guess if I don't hear anything back on this in a while I'll just close it under the assumption that it's not a useful feature addition.
Comment #4
drunken monkeyUsing the same server machine name for all of those seems like the proper way to handle this, I think. But I can understand that it's problematic. If you are only using Solr servers then Search API Override might be a good solution – but if there's also a DB server involved, it gets really tricky.
Regarding the patch, I can't really say how useful it would be to people in general. I guess if one or two others show interest here, I'll commit it.
In any case, thanks for your contribution!
Comment #5
rjacobs CreditAttribution: rjacobs commentedThanks for confirming that the technique we arrived at would be best practice given the circumstances.
Search API Override is an interesting approach but, as you noted, the fact that we have DB servers in the mix makes that an incomplete solution. Because the table mappings (field > search_api_db table) are stored as part of the server configuration for a DB server, but are still dependent on the index settings, things can get pretty tricky whenever swapping servers in our case.
For now we'll see how things go with the common machine name technique. No matter what I think the work needed to figure this out was still easier than setting up new solr servers/cores for each and every dev space that may need to be spun-up (hence the reliance on search_api_db).
Comment #6
13rac1 CreditAttribution: 13rac1 commentedThis patch works perfectly (with minor offsets), although I may end up using Search API Override. I'm still looking into the ideal process.
I have a dev, staging and two live environments. The weekly data migration is updated (174k nodes) on staging then synced to the disabled live environment. The single live Solr index is cleared to removed deleted nodes, re-indexed, caches (Drupal and Varnish) cleared, then the live environments are flipped (Apache points to a symbolic link.) The removed nodes are deleted from the staging Solr index by Migrate, but not the live index hence the required index clearing. I wish it were less complicated, but I'm not seeing a better way. The problem is the shared Solr server means clearing the index results in two hours of broken search results. Both of these solutions will fix the problem. I just must decide which is easier to maintain. Thanks!
Comment #7
drunken monkeyOK, great, if it helps others, too …
Committed. (Only applied a few style changes before.)
Thans again for your patch, rjacobs! And thanks for reviewing, eosrei!
Comment #9
13rac1 CreditAttribution: 13rac1 commentedGreat thanks!
Comment #11
drunken monkeyNeeds to be ported to D8.
Comment #12
drunken monkeyComment #13
cristiroma CreditAttribution: cristiroma commentedAdded patch for D8
Comment #14
drunken monkeyLooks good, thanks!
Committed.