Reverting Features Drops Search API DB tables

I'm also having this issue.

Notice: Undefined index: search_api_viewed in SearchApiDbService->indexItem() (line 332 of sites/all/modules/search_api_db/service.inc).

...So it's a Search DB module issue it seems.

Are you using the latest version of the Database search module?

Using the Beta 2 of the DB search module, and RC1 of Search API.

Try saving the index's „Fields“ form, or disabling and re-enabling indexing of the „Complete entity view“ field. If all else fails, moving the index from the server and then adding it back again would also be an option. I don't know why that field shouldn't be stored in the server …

Is there a table corresponding to the field in the database, like search_api_db_INDEX_search_api_viewed?

I get the same issue. Here's the list of generated tables from the Search API.

I removed the Complete Entity View stuff as well, no affect.

I also get the same issue and the table search_api_db_INDEX_search_api_viewed was generated.
Saving the index's „Fields“ form, or disabling and re-enabling indexing of the „Complete entity view“ field does not change anything.

Have you two tried the latest devs of this module, Search API and Entity API?

I have the exact same problem. I am running the latest devs for search api and entity and it still persists.

Notice: Undefined index: search_api_viewed in SearchApiDbService->indexItem() (Zeile 332 von /xxxxxx/sites/all/modules/search_api_db/service.inc).

Search API -> 7.x-1.x-dev (2011-Dec-16)
Entity API -> 7.x-1.x-dev (2011-Dec-07)

The strange thing is I have other indexes that index similar fields, and they do not have this issue.
I tried removing the index, re-adding it again and so on ...

From the report:

SQLSTATE[42S02]: Base table or view not found: 1146 Table 'db_drupal_xxxxxx.d7_xxxxxx_' doesn't exist

Okay, I managed to fix these errors.

I tried to index Commerce prices as mentioned in http://drupal.org/node/1310970#comment-5320656.
After adding aggregated fields and resaving fields I get many Notice: Undefined index: xxx in SearchApiDbService->indexItem() (line 332 in .../sites/all/modules/search_api_db/service.inc)..

Then I noticed on the fields tab that the aggregated field is of type 'Text'. When I changed it to 'Decimal' all errors were gone. Maybe it will help someone.

Also experiencing problems.

Notice: Undefined index: search_api_fulltext in SearchApiDbService->indexItem() (line 332 of ...public_html/sites/all/modules/search_api_db/service.inc).
Notice: Undefined index: search_api_fulltext in SearchApiDbService->indexItem() (line 333 of ...public_html/sites/all/modules/search_api_db/service.inc).

Had this issue, but it resolved when I removed diaeresis, acute accents and ñ from the content.

Cheers.
www.drop.cl

Advice from comment #4 by drunken monkey helped me. Thanks!

I have done:
* Disabled Entity HTML output field in the field to index list
* Ran cron
* Enabled Entity HTML output field from the field to index list
* Ran cron
* Ran cron once again

No notices and views filter is working properly.

#4 Helps me, thank you. Just re-save fields to index page admin/config/search/search_api/index/default_node_index/status.

Thank you for great module!

comment #4 also helped me to solve the indexing problem, thank you.

Good job!

Please test the patch in #1414138-10: Notice: Undefined index: search_api_access_node for the Search API, which should solve the issue properly. (Re-enable the data alteration after applying the patch.)

When reverting a search feature, all tables are dropped and thus no content can be indexed.

We have to manually create the tables each time a feature is reverted, the workaround as proposed in #4 does not work for us.

I can confirm that reverting an index feature drops all of the tables. Here is the full behavior I've encountered. This is an installation profile.

Upon initial installation of the profile, only non-field based tables are created. This happens in part because the fields in the database haven't been created yet at that point during installation. I've tried to change the order of dependencies of the modules in the info file but this doesn't fix that.

After installation, when only some of the index fields tables are created, if I revert the feature all of the index tables are dropped. I can see that this is because preDelete() is fired which drops any tables for the index. I imagine that this is the preferred behavior. It just seems like the update isn't run after the tables are dropped.

After installation, if I disable and then enable the search_api_db based feature then all of the tables for the index fields are created. Reverting the feature does not delete the tables in this state.

I also notice that upon initial installation, the entry for the search_api_db_server for the index in question has the incomplete number of fields. If I feature revert that index entry has the correct number of fields, but as noted above all of the tables are dropped. If I trick features to think that the database state is out of sync with the code and feature revert again then the correct tables are loaded.

I will try and troubleshoot this further if I can.

I found that enabling the feature that contains the server and index in the installation *.install file loads the correct tables, but they are still removed during the initial features revert. It looks like the server is getting updated, which removes the index tables, but the index is not getting updated, which repopulates them.

Updating title. Don't have a solution for this but will try next time I get a chance.

Any chance this is fixed with the patch in #1414078-21: Revert of exportables not working?

I'm in the same situation that achton is and I had to save the structure of my index tables in .sql files so I can drop them and re-create them again.

Not a nice solution at all, but it is better than the millions of hours I was spending in disable, enable, re-saving, reverting, etc etc etc.

I seem to be encountering this problem as well, at least with version 7.x-1.0-rc3 (wasn't happening with 7.x-1.0-rc2). I haven't looked into it too far, but just wanted to post another data point here.

After installing with an install profile (with exported Search API/Search API DB settings), and reverting all features via drush, the field-specific tables don't exist.

After installing with an install profile (with exported Search API/Search API DB settings), and reverting all features via drush, the field-specific tables don't exist.

That's good, that just means that #2083079: Store single-valued fields in a single, denormalized table is working properly. Or does the search not work after re-indexing as well?
In that case: What version of Search API are you using, the latest one?

The reindexing didn't work, since I got a PDOException with the missing table; I was using the latest stable releases of Search API (7.x-1.8) and Search API DB (7.x-1.0-rc3).

You might need to recreate the DB server feature after the update. With Features, it's always hard to tell …
Move all indexes from the server, then on it again and then save the new state of the server feature.

Additional info

I can confirm the issue.
I am using the latest development versions of search_api and search_api_db.
Field specific tables are not created on feature install/revert (Feature code contains these tables in config).

I have this problem.

Unfortunately in my instance it comes down to a limitation of MySQL. There is a maximum limit of 64 characters length on a table name (see http://dev.mysql.com/doc/refman/5.0/en/identifiers.html).

I am using the field_collections module https://drupal.org/project/Field_Collection - this puts a pile of junk onto the front of every field name such that with the index I get search_api_db_node_index_{collection_field_name}_field_{name of field within the collection} or worse if I have a collection within a collection then I get search_api_db_node_index_{collection_field_name}_{secondary_collection_field_name}_field_{name of field within the collection}

-huge tables names are a bit of a problem here!

Unfortunately in my instance it comes down to a limitation of MySQL. There is a maximum limit of 64 characters length on a table name (see http://dev.mysql.com/doc/refman/5.0/en/identifiers.html).

There shouldn't be any problem with that, we take care of this limitation in the code. Table names in this module never exceed 62 bytes in length.

Finally having the time to look at this, I find that I cannot reproduce this problem. In my case, when creating features for a server and index, changing and reverting either works without any problems.
If you are sure this problem still occurs with the newest version of all related modules (even after recreating the features), please provide a step-by-step instruction on how to reproduce this.

I encountered this problem once on a drupal.org test-site.
Wasn't able to reproduce locally, and wasn't able to reproduce on a second drupal.org-testsite.
I assume the problem occurred when there where some other changes in the search_api settings, which were not in the feature, and had the feature revert go strange.

comments above helped me figure out why were getting base table not found errors for a search index view and broken handlers eg
Field handler search_api_index_publication.nid
base_table = 'search_api_index_publication'

various comments above suggested disabling fields etc - i skipped to the easy step of 'run cron' and magic - all back. might help others

I am finding this is still a problem but it is a "chicken-egg" thing for us. In our install profile Features, the Search API DB and Index are created before the content types/field instances.

I wrote a hook that enables our Search API feature after the content type Feature is enabled.

We have a migration that runs during installation and there are pre-import and post-import hooks so I used one of those. However, others could probably also do this using hook_post_features_enable_feature(). One would just need to run module_enable() for their Search API Feature after components were finished being created and then do drupal_cron_run() to have all the items indexed.

I wrote a hook that enables our Search API feature after the content type Feature is enabled.

Isn't that just what module dependencies are for?
Unfortunately, while we can automatically define some dependencies, this is one we probably have little to no chance of automating, that's something you'll have to do for yourself. (But, as always, all of this should be much better in Drupal 8.)

However, I don't really see the relation to this issue. It's not about reverting in your case, but the initial installation not working, right?

Isn't that just what module dependencies are for?

Not exactly. Here's how a Features module with the Search API settings and content types versioned would work on installation:

1. Enable the dependencies from the module.info (this is the step where you'd declare a Search API Feature that looks for content types)
2. Install components. (this is the step where the content type would get created)
3. Everything gets reverted as a part of installation,

Hence the chicken-egg problem where you would rather have two features. One to create the content type first and then another to enable the Search API settings that is best enabled via a hook after one is certain the content types that should be indexed are persisted.

So this is a problem for us because Features auto-reverts on install.

Hence the chicken-egg problem where you would rather have two features.

Er, yes, I'd definitely recommend that! Then make the content types feature a dependency of the search feature and the issue should be solved – that's what I meant. I didn't think you'd have both of those in the same feature.

Also, again, I don't see the relation to this issue. Your problem is about installation, not reverting, and you already have figured out what's going wrong. If there is more to discuss for you, please open a new issue, and only set this one back to "Active" if you can explain how the problem is the same.

I can't reproduce this reliably, but I just experienced this issue with the dev version of panopoly_search on a test site, using the latest stable versions of search_api and search_api_db. Even disabling the Feature, deleting all the indexes/servers and then re-enabling it didn't fix it.

Here's the process that fixed it for me in the end:

1. Edit the index and set the Server to "< No Server >" and Save
2. Edit it again and set the Server back to "Database Server" and Save
3. Edit it AGAIN and enable it
4. Now, everything appears to be working fine

@drunken monkey, thank you for the comment in #35 about making the search feature dependent on the content types.

I ran into a very similar issue with Solr, where reverting features caused my search pages to break (dead response from web server, nothing loaded in the browser). For anyone facing a similar problem, here are the errors I was seeing in the logs:

Undefined index: target in /sites/all/modules/search_api_ranges/search_api_ranges.module on line 602
Undefined variable: remove_facet_url in /sites/all/modules/search_api_ranges/plugins/facetapi/widget_select.inc on line 63
Undefined variable: remove_facet_url in /sites/all/modules/search_api_ranges/plugins/facetapi/widget_select.inc on line 96
Undefined index: #active in /sites/all/modules/search_api_ranges/plugins/facetapi/widget_select.inc on line 209
Undefined index: target in /sites/all/modules/search_api_ranges/search_api_ranges.module on line 602
Undefined index: target in /sites/all/modules/search_api_ranges/search_api_ranges.module on line 602
Undefined variable: remove_facet_url in /sites/all/modules/search_api_ranges/plugins/facetapi/widget_select.inc on line 63
Undefined variable: remove_facet_url in /sites/all/modules/search_api_ranges/plugins/facetapi/widget_select.inc on line 96
Undefined index: #active in /sites/all/modules/search_api_ranges/plugins/facetapi/widget_select.inc on line 209

It certainly makes sense to have search items in a separate feature. However, I hadn't really thought about it since we've often had content types and Views in the same feature, and have never seen this type of break.

Is there maybe an opportunity to add some code to Search API that would detect for the existence of content types, entities, fields, etc. in a feature, and build the order of the feature accordingly? Or at least add some notes to the documentation that provides Search API features best practices?

Thanks again!

Still facing this issue and have found a perfectly working workaround.

Add install file for the feature and lock the server component

/**
 * Implements hook_install()
 */
function FEATURE_NAME_install() {
  // locking search_api_server to stop running into base table or view not found issue
  features_feature_lock('FEATURE_NAME', 'search_api_server');
}

It is definitely a workaround and not a solution, but a working workaround.

^^

Can we get some movement on this issue, please? Happy to provide any information I can.

Dropping the tables during revert causes installation profiles to fail to install.

I just tried moving the "local DB" search server definition into its own dedicated feature that the feature containing the indexes depends on, and it helped a little but, but we're still getting a SearchApiException during install. Curiously, when running install through Drush CLI, this exception is not fatal, and when the site is done installing, the index is working as expected.

Perhaps the fix is to check if the site is being installed, and to soften this exception?

WD search_api: SearchApiException while adding index Artifact search [error]
index to server Local database: SQLSTATE[42S02]: Base table or view not found: 1146 Table 'search_api_db_search_index_artifacts_text' doesn't exist in SearchApiDbService->fieldsUpdated() (line 684 of search/search_api_db/service.inc

Okay, found out I was actually not running the latest version of search_api_db. I was running 1.4 instead of 1.5. Updating addresses the SearchAPIException, but trades it for this error:

Artifact search index: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'title2' in 'field list':
INSERT INTO {search_api_db_search_index_artifacts} (author, created, search_api_language, status, sticky, title2, item_id)
VALUES
(:db_insert_placeholder_0, :db_insert_placeholder_1,
:db_insert_placeholder_2, :db_insert_placeholder_3,
:db_insert_placeholder_4, :db_insert_placeholder_5,
:db_insert_placeholder_6); Array
(
[:db_insert_placeholder_0] => 1
[:db_insert_placeholder_1] => 1486348185
[:db_insert_placeholder_2] => und
[:db_insert_placeholder_3] => 1
[:db_insert_placeholder_4] => 0
[:db_insert_placeholder_5] => Product title
[:db_insert_placeholder_6] => 1
)
in SearchApiDbService->indexItem() (line 938 of modules/contrib/search/search_api_db/service.inc).

In some ways, this is actually worse because search on the site will not index anything or function without manually disabling & reenabling the server + index.

I have tried re-exporting the feature after fixing the index but the exported feature ends up with the same content as before the re-export.

A potential fix you could try would be to make (kind of) the same switch as we made in D8: Instead of storing the table structure in the server options, store it in a variable (in D8, we use the state). That could potentially resolve this. It would be a good idea to do in any case – having this information in the server options was a design flaw from the beginning.

Instead of storing the table structure in the server options, store it in a variable (in D8, we use the state).

Is this change in D8 discussed a issue? If so, can you point to it?

Yes, here it is: #2564847: Move information about database backend's tables into the site's state.
I guess, though, that a variable also wouldn't be a good idea. Probably we need to have a separate table for this information. (The data structure is a bit large for the variable table.)

Thank you. I'll have a look. Any change that someone that fully understand the problem to update this issue?

I can't really say I understand this problem fully, i.e., the one discussed in this issue. Apparently I even tried to reproduce it once but couldn't.
For the larger issue of making the DB backend and Features work better together, I guess we should use a separate issue – maybe there is already one (it feels like I've replied to a dozen already), otherwise we could create it.

We currently have this issue on a site with 15M nodes in an index. It causes horrific load when the index has to be moved to a new ID.

We have temporarily locked the feature.