Add a data alteration for indexing complete hierarchies

This one works for me, please test / review.

I've also sneaked in a few other minor fixes, you can just ignore those. It would just be good to know if search_api_extract_fields() still works the same with the change (should now just need much less entity loads).

Hm, I can't reproduce this. I take it you use the latest version of all modules?
Then please add debug($item); return FALSE; at the beginning of SearchApiDbService::indexItem(), index a single item manually and post the result.
Also: Does this happen for all items, or just for some? (Kind of complicated to tell that, I guess. You can index a single item manually and see whether the error occurs. Then you could also mark items indexed by adding return TRUE; as the first line in indexItem() and see if it also occurs when indexing other items manually. Or call search_api_index_specific_items() directly from code.)

Ah, I knew I messed something up in search_api_extract_fields(). Restored that to the original version, I should just fix it properly, in a different issue.
The SQL exception is really weird, this really should not hap— oh, wait. Please go to the Fields tab, save the form and see whether this fixes the error.

The error could also stem from an interaction between the "Aggregated fields" data alteration and this one. See if the error occurs when enabling only those two data alterations and no other fields.
Otherwise, yes, please find out what fields cause the errors.

Ah, of course you can't reproduce – as you just wrote, you changed the fields you index, so you already saved the Fields for mas I suggested. Sorry, didn't think of that.
Anyways, seems like that has to be fixed somehow. The issue here seems to be that when the type is changed, the server won't be notified about it.

OK, now I'm confusing myself completely. To summarize, the two bugs:

- SQL exception when indexing most items: I'd guess this is due to stale field type data in the index. When activating the alteration and saving the "Fields" tab, this should vanish. (But you say, they didn't, so maybe I need a new theory …) Do the entries that are reported as duplicates correspond to the node nids you try to index?
- Fatal error: This I think could come from the interaction between the two data alterations.

Thanks for investigating and explaining the problem! I think I now understand what might be wrong here.
The attached patch hopefully fix the bug. I've also included a note that all fields should be de-selected before disabling the data alteration – otherwise there's hardly a way to keep this from triggering in this case. (A flaw in the current framework which will have to be fixed separately.)

I heard about the Converter module from Chris already, but from the description it doesn't sound like those kinds of things would fall into its area of expertise. This is, after all, more about changing meta data than changing the data format.
Also, Converter isn't usable yet, so we're gonna need an interim solution anyway. However, once Converter is stable enough, it'd be definitely great to integrate it into the Search API (probably with a generic "Converter" data alteration). After all, I've had only positive experiences with Chris' generic frameworks so far. ;)

(By the way: Are you gonna be in London? We'll probably do a Search API BoF there.)

Go to the index's "Workflow" tab. There is now a new data alteration, "Index hierarchy", which lets you do that.

Ah, my apologies! I didn't remember to re-roll this after committing #1064884: Add support for indexing non-entities – a working patch should be attached.

And thanks for reviewing!

Do you have the latest module version, and did you clear the cache?

I am using the facet_api integration branch. Your patches applied cleanly. I assumed I needed to use that branch to get integration with Facet API in the first place. Maybe you can shed some light on that? Thanks.

You need the branch for Facet API integration – however, this patch is in principle independent of that development. You can index hierarchies without it – you just won't get the corresponding hierarchical facets.
And I just tested it, it works with both the normal Search API and the Facet API integration branch / fork. If you cleared the cache and it still doesn't appear in the workflow (even though you have taxonomy term references on the indexed entity), maybe check the logs what might go wrong. Or debug the search_api_admin_index_workflow() function to see what goes wrong there.

First there is no entry in search_api_search_api_alter_callback_info() for the new class: SearchApiAlterAddHierarchy.

This probably means you made some mistake while applying the patch. See the last-but-one change in the patch linked in #20.

Note the $type == 'field_collection_item' addition. Is there any reason that this doesn't simply filter only on taxonomy_vocabary?

Yes – as the functionality is not really specific to taxonomy terms (by the way, for those, filtering on "taxonomy_term" would be the right thing to do), I figured we could just as well allow all other hierarchical information, too.

What kind of errors did field collection items cause? Hard-coding a special case is definitely not the way to go here, so we should look for why this causes errors.

When does the error occur – during indexing, or already when configuring the data alteration?
And are you using the latest version of the Entity API? If I'm not mistaken, a similar bug was recently fixed there, and this does seem like an Entity API problem.

BTW, the original error occurred when configuring the data.

Ah, thought so.

Great to hear it now works. Thanks for reviewing!
Committed.