consolidate Solr queries on cron run
robertDouglass - August 27, 2009 - 13:44
| Project: | Apache Solr Search Integration |
| Version: | 5.x-2.x-dev |
| Component: | Code |
| Category: | feature request |
| Priority: | critical |
| Assigned: | claudiu.cristea |
| Status: | closed |
Description
In response to some solr server setups that throttle operations, there may be a need for some people to limit the number of delete operations that are sent to the server on cron runs.
| Attachment | Size |
|---|---|
| apachesolr.nodeDeletionFix.patch | 1.36 KB |
#1
I don't really like this implementation for the general public. Is it proven that anyone needs this protection on normal servers, or just servers that have throttling? Also seems like an odd thing to make a configuration for (not that this patch has configuration), so I wouldn't want to go down that route.
#2
Let's find a reasonable approach for this asap - it continues to be a problem.
I'm inclined to pt this patch in for the next RC if we don't have something better
#3
Here's a mostly untested patch that groups Solr deletes by 500 at a time rather than sending one delete query per node.
This should give better performance in almost all cases, and be more compatible with a limits on update queries.
#4
That patch isn't really working and doesn't provide a useful count of nodes to be deleted. Here's a better one.
#5
With update to README
#6
oops - missed README changes
#7
minor tweak - stop sending updates on cron if one fails.
#8
committed to 6.x-1.x
#9
And to DRUPAL-6--2
#10
The patch for DRUPAL-5--2.
#11
Fixed in #278744.
#12
Automatically closed -- issue fixed for 2 weeks with no activity.