Fix tokenizing of string fields

Try to go to the „Fields“ tab and re-save the form. If that doesn't help, remove the index from the server and then re-add it. Somehow it seems a table was created for a field with the wrong data type.

one error record per not indexed node?

Yep.

I had the same problem. To solve, I disabled all fields, then had to add the main body text as a related field and set content type to Fulltext instead of String.

I'm using search_api_db and latest search_api dev
In my case happens changing a field indexing type from text to string. (The field itself is a string)

In SearchApiDbService::indexItems() the field is indexed as 'tokens' instead of 'string' and the code try to save it in a wrong table structure.

This was caused by the processor 'Filter Html', removingthe field from there solved the indexing error.
Hope this help.

I had the same error with a node index created as seen in the screencast: http://vimeo.com/15556855.

I unchecked the 'Filter Html' processor and it worked.

Thanks.

Same issue here. The solution in #7 worked for me as well. Maybe there needs to be something in the code that automatically removes a field that has been changed to a string from the processor?

Here I had "title" indexed as fulltext and filtehtml AND tokenizer filters enabled on it.

Once I moved title to text (in order to be able to use it for sorting) I had to remove it from both filters to get rid of this blocking error.

I guess this should be marked "fixed" only when a change from full text to title doesn't throw this error. And maybe the tokenizer and the filterhtml shouldn't show fields for which they are not able to work, at all.

I'm reopening this as bug report beucase it is easy to be reproduced and it breaks the index. At most it should be closed as "Won't fix", not as "fixed".

I confirm #10.
Had basically same issue.

So when you index some field as Fulltext and on 'Workflow->Processors' add that field under Tokenizer(or HTML) filter and then change that field from Fulltext to String, make sure to remove checkboxes under these filters.

It's not possible to run Tokenizer or HTML filter for String type fields.

Cross-referencing #1893790: WD search api: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'word_id' in 'field list'.

Likewise, can confirm #10 and #11 - thanks much!

I got this issue in a special situation:
I am indexing a node with an entityreference field to another node using search_api_db. The node title of the referenced node can be indexed as full text successfully, but does not serve the goal in facets. If I try to index it as a string, I get this error. If I don't index it, indexing works fine.

I tried removing and adding multiple times. I believe this issue is related to an underlying bug.

Thanks for reporting this problem! The problem here is that several processors assume that they're always run on fulltext fields (see #1346276: Tokenizer should only run on fulltext fields for example), though that might not be the case. For example, the HTML filter could also be useful for string fields, if the data contains HTML tags – e.g., when using the field as a facet, you usually don't want the tags to be displayed there.

The attached patch should fix this centrally for all processors at once – if they try to tokenize a string field (or another non-fulltext field), the processor will automatically revert it back to a string by concatenating the tokens.
Please test/review!

(However, if the string field doesn't contain tags, you should probably still just deactivate the filter for that field.)

Wrong version …

#15: 1311260-15--tokenizing_of_string_fields.patch queued for re-testing.

Committed.