It's possible to get drush search-index in an infinite loop, for example if you have Solr search installed but set to read only mode. search-index just iterates through, invoking hook_update_index() until the number of remaining items reaches zero. But in the case described, the number remaining never changes, so it keeps going and never completes. (If you're asking why you would run search-index with Solr set to read only, remember that you might want core search to index. Besides, this is only an example.) Perhaps we should set a limit on how long search-index can iterate without making progress and then fail with an error. Patch to follow.

CommentFileSizeAuthor
#5 drush-infinite-search-index.patch855 bytesabautu
#1 drush-search-index-infinite-loop-1824510-1.patch1.21 KBtraviscarden

Comments

traviscarden’s picture

traviscarden commented
Status: Active » Needs review
StatusFileSize
new drush-search-index-infinite-loop-1824510-1.patch1.21 KB

Here's a patch to get things started. It just tallies consecutive unproductive iterations and fails after three with the message "Indexing stalled with @number items remaining." The test for unproductive iterations may be a little naive--it just assumes that if the number of unindexed items remaining hasn't changed from the last iteration that it didn't progress, but if nodes were added during that iteration equal in number to those indexed it would return a false positive. There's probably a slim chance of that happening, but I mention it anyway. Perhaps requiring three such instances in a row is adequate. Perhaps it needs to check that the difference between total nodes and number unindexed hasn't changed. In any case, this gets the discussion started.

kenorb’s picture

kenorb commented

Duplicates:
#985778: search-index does not process
#880554: search-index never completes, but admin/settings/search shows site as 100% indexed
#955736: drush search-index fails

kenorb’s picture

kenorb commented
Status: Needs review » Reviewed & tested by the community

Looks better for me:

$ drush search-index
Remaining items to be indexed: 41  [ok]
Remaining items to be indexed: 41  [ok]
Remaining items to be indexed: 41  [ok]
Indexing stalled with 41 items remaining.   [error]
The search index has been built.      [ok]

Apart of this, that the message is repeated 3 times.
But at least doesn't gets into an infinite loop

moshe weitzman’s picture

moshe weitzman commented

Looks fine to me. Any objections?

abautu’s picture

abautu commented
StatusFileSize
new drush-infinite-search-index.patch855 bytes

It's more or less related to your problem, but from my experience, indexing with ApacheSolr (branch 6.1) will always create an infinite loop, even for a read/write index. This happens because ApacheSolr uses variables to store current index status (compared to core search which queries the tables).

In short, drush calls a subprocess to index nodes. The subprocess will update variables, but this updates are not visible back in the main process (due to caching variables). In the main process, when drush asks again for the the indexing status, ApacheSolr uses the old variable values (instead of updated ones). The indexing runs normally, but drush always report the same number of remaining items. When indexing is completed, that's when your original problem kicks in, too.

I'm attaching a patch for this, which would complement the previous one.

Cheers,
Andrei

traviscarden’s picture

traviscarden commented

Thank you, @abautu. That should probably be a separate issue. Would you create one and link to it from here?

abautu’s picture

abautu commented

Sure. Done here: #1918922: Search-index gets into an infinite loop when using ApacheSolr due to variable caching

moshe weitzman’s picture

moshe weitzman commented
Status: Reviewed & tested by the community » Fixed

Committed to 5 and 6.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.