Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.
Currently the Search API stores a lot of unnecessary stuff in the options settings (making #1228726 necessary). This makes it painful when working with Features and introducing some small changes to the index that then might lead to large differences in the export code.
Ideally, only the fields that get indexed are stored in the options settings.
Comment | File | Size | Author |
---|---|---|---|
#26 | 1308638--mlt-follow-up-26.patch | 748 bytes | drunken monkey |
#23 | 1308638--follow-up-23.patch | 6.5 KB | drunken monkey |
#21 | 1308638--follow-up-21.patch | 4.83 KB | pbuyle |
#16 | 1308638--follow-up-16.patch | 4.43 KB | drunken monkey |
#14 | 1308638--follow-up-14.patch | 3.52 KB | drunken monkey |
Comments
Comment #1
drunken monkey:(
Comment #2
drunken monkey(Clarification: We won't touch options besides the fields settings here, I think.)
Current plan:
- Remove "name" and "indexed" key from field settings.
- Only store indexed fields in settings.
- Provide an API method (
SearchApiIndex::getFields()
) to access the previously available information (with unindexed fields, names and additionally descriptions).- Related fields will be added via a special
$index->options['additional fields']
entry. (This also means unused related entities will now automatically vanish again when the fields form is saved.)Patch is under construction, it's not too trivial.
Comment #3
drunken monkey- Store "boost" only for fulltext fields, and if it is different from "1.0".
Comment #4
drunken monkeyDisregard that – just checked, and for me about 2/3 of memory size (after the above changes to the fields settings) come from the „Aggregated fields“ data alteration alone. While, for testing purposes, I do have several aggregated fields defined, this is of course a tremendous waste of space.
So I'm gonna apply the above general rules also to those settings. This shouldn't have any impact on other modules, however, as I guess none of them manually checks the internal settings of that data alteration.
Comment #5
drunken monkeyMost processors also have a "fields" setting, this was also shrunk down to the essentials.
The attached patch works for me (with patches to several other modules, which I'll post in a minute to those) and reduces the stored size of the index options by a factor of about 10 (again, for me).
Comment #6
drunken monkeyChanged other modules:
- Database search
- Multi-index searches
- Saved searches
- Solr search
Comment #7
mh86 CreditAttribution: mh86 commentedTested your patch (in combination with the Solr Search) and the Features export code for Search API index fields was reduced from around 1000 to 25 lines, yeah :)
One small problem I've found so far:
--> the indexed field names in the search index view page are missing.
Comment #8
drunken monkeyAh, you're right, missed that one.
Comment #9
drunken monkeyCommitted.
Comment #10
dimarick CreditAttribution: dimarick commentedI am get a problem with this commit, becouse field names are deleted from index config, but search_api_views is not fixed.
See attach for details
Comment #11
dimarick CreditAttribution: dimarick commentedComment #12
Damien Tournoud CreditAttribution: Damien Tournoud commentedI had a lot of issues with that too. It seems that this commit partly removed the fields settings from the index configuration (?!?).
Comment #13
maxmic CreditAttribution: maxmic commentedI had a lot of issues with that too. Also with search_pages modules and the extend_searche_page.
Comment #14
drunken monkeyAh, yes, sorry, missed that spot. Patch attached.
The fix is already contained in #1231512: Use real Relationships instead of level magic in Views integration, too.
You'll have to be a bit more specific there, I fear.
However, I've already spotted two bugs myself, which should be fixed by the attached patch.
As previously listed:
Comment #15
maximpodorov CreditAttribution: maximpodorov commentedField selection (in "Processor settings" section) is lost after saving settings in index workflow page. Is it possible to fix this?
Comment #16
drunken monkeyAh, yes, thanks for catching that! I wholly blame Form API's many pitfalls.
Extended patch attached.
Comment #17
maximpodorov CreditAttribution: maximpodorov commentedThank you. Th problem #15 is fixed.
Comment #18
mh86 CreditAttribution: mh86 commentedJust realized that for one index some of my facet blocks are gone (I guess it's related to this issue). This only affects those from related entities (e.g. Profile -> Field collection -> Taxonomy terms), the others ones (fields on the base entity) are still there.
Comment #19
mh86 CreditAttribution: mh86 commentedFurthermore, all my views field with '(indexed)' (in combination with #1231512: Use real Relationships instead of level magic in Views integration) disappeared.
Comment #20
drunken monkeyDefine „gone“.
If they still appear on the „Facets“ tab, but (when enabled) nowhere on the „Blocks“ page, this also happens to me from time to time and seems to be a weird bug in the Facet API. Something that should never be
NULL
is suddenly set toNULL
for some facet blocks and chaos ensues.However, I can't reproduce either of these two, so I'll need a bit more details. Let's just discuss this this afternoon.
Comment #21
pbuyle CreditAttribution: pbuyle commentedI had several "Undefined index: value" and "Undefined index: original_value" error preventing correct indexing when using an Features exported index in SimpleTest that I traced to usage of
$this->options['fields']
inSearchApiIndex::index
. Replacing it with$this->getFields()
fixed the issue.Here is an updated version of the patch in #16 with this additional fix.
Comment #22
drunken monkeyIf you executed update.php properly, this shouldn't happen. Using
$this->options['fields']
should be completely correct in this case, as we only need the indexed fields''type'
keys.How exactly did you trace this? And what do you mean with „using an Features exported index in SimpleTest” – did you try to import an index that was exported before this change? This of course won't work, like with all changes that require DB updates.
Also, I'm pretty sure you mean
original_type
, notoriginal_value
.Comment #23
drunken monkeySpotted the bug in
getFields()
that caused the whole trouble for Matthias. And once again it was the type of bug where you wonder afterwards that anything at all worked … Quite a miracle all worked for me – maybe I should rethink my testing setup.Anyways, patch with the fix attached, please test!
Comment #24
mh86 CreditAttribution: mh86 commentedThanks Thomas for taking a look at it. My views fields and facets are back now. And if you need some complex configs for testing, let me know ;)
Comment #25
drunken monkeyGreat, committed.
Hopefully that's all fixed now.
Comment #26
drunken monkeyNo, not yet. :-/
See #1111852-105: Add a "More like this" feature – the „More like this“ config form has still some wrong usage. Patch attached.
Comment #27
drunken monkeyNeither API change nor release blocker now, though.
Comment #28
modstore CreditAttribution: modstore commentedFixed the problem for me. Cheers.
Comment #29
drunken monkeyGreat, committed.
Thanks for testing!
Comment #30
BeaPower CreditAttribution: BeaPower commentedIs this in any available releases such as stable or dev - or not yet?
Comment #31
drunken monkeyIt's in RC 1 and dev.