In response to some solr server setups that throttle operations, there may be a need for some people to limit the number of delete operations that are sent to the server on cron runs.

CommentFileSizeAuthor
#10 throttle-561082-D5.patch16.99 KBclaudiu.cristea
#7 throttle-561082-7.patch8.76 KBpwolanin
#6 throttle-561082-6.patch8.62 KBpwolanin
#5 throttle-561082-5.patch7.99 KBpwolanin
#4 throttle-561082-4.patch7.99 KBpwolanin
#3 mass-delete-561082-3.patch6.18 KBpwolanin
apachesolr.nodeDeletionFix.patch1.36 KBrobertdouglass

Comments

robertdouglass’s picture

robertdouglass commented

I don't really like this implementation for the general public. Is it proven that anyone needs this protection on normal servers, or just servers that have throttling? Also seems like an odd thing to make a configuration for (not that this patch has configuration), so I wouldn't want to go down that route.

pwolanin’s picture

pwolanin commented
Version: 6.x-2.x-dev » 6.x-1.x-dev
Priority: Normal » Critical

Let's find a reasonable approach for this asap - it continues to be a problem.

I'm inclined to pt this patch in for the next RC if we don't have something better

pwolanin’s picture

pwolanin commented
StatusFileSize
new mass-delete-561082-3.patch6.18 KB

Here's a mostly untested patch that groups Solr deletes by 500 at a time rather than sending one delete query per node.

This should give better performance in almost all cases, and be more compatible with a limits on update queries.

pwolanin’s picture

pwolanin commented
StatusFileSize
new throttle-561082-4.patch7.99 KB

That patch isn't really working and doesn't provide a useful count of nodes to be deleted. Here's a better one.

pwolanin’s picture

pwolanin commented
StatusFileSize
new throttle-561082-5.patch7.99 KB

With update to README

pwolanin’s picture

pwolanin commented
StatusFileSize
new throttle-561082-6.patch8.62 KB

oops - missed README changes

pwolanin’s picture

pwolanin commented
Title: Throttle deletes on cron run » consolidate Solr queries on cron run
StatusFileSize
new throttle-561082-7.patch8.76 KB

minor tweak - stop sending updates on cron if one fails.

pwolanin’s picture

pwolanin commented
Version: 6.x-1.x-dev » 6.x-2.x-dev
Status: Needs review » Patch (to be ported)

committed to 6.x-1.x

robertdouglass’s picture

robertdouglass commented
Version: 6.x-2.x-dev » 5.x-2.x-dev

And to DRUPAL-6--2

claudiu.cristea’s picture

claudiu.cristea
Primary language Romanian
Location Arad 🇷🇴
commented
Assigned: Unassigned » claudiu.cristea
StatusFileSize
new throttle-561082-D5.patch16.99 KB

The patch for DRUPAL-5--2.

claudiu.cristea’s picture

claudiu.cristea
Primary language Romanian
Location Arad 🇷🇴
commented
Status: Patch (to be ported) » Fixed

Fixed in #278744.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.