At Dries' keynote in Boston, there were two modules tipped for removal. Ping, and Throttle. Ping has gone, here's a patch for throttle.

A few reasons I personally think it ought to go:

- Adds complexity to low level functions like module_list, in addition to being responsible for a bunch of module_exists calls.

- Complicates two of the more critical user interfaces - module administration and blocks, once it's been enabled.

- Doesn't play very nice with aggressive caching (a much, much bigger performance boost). This message might possibly discourage new users from using that setting:

The following enabled modules are incompatible with aggressive mode caching and will not function properly: throttle

- You can set the auto-throttle probability limiter (!) to 100% and get an extra database query on every page view, which is a small but nevertheless unnecessary performance hit.
Working patch including upgrade path is attached. I think I got everything but haven't yet done thorough testing.

Comments

catch’s picture

StatusFileSize
new39.87 KB

Freso found some whitespace issues in IRC and told me about cvsdo :)

catch’s picture

StatusFileSize
new29.3 KB
new39.87 KB

Let's try that again.

Freso’s picture

StatusFileSize
new40.19 KB

Okay, here's one properly showing in the diff that the file is removed. (Note to self: Use diff -N after having removed the file.)

Looks good otherwise. This one also fixes a few trailing whitespaces still remaining. Going to test and look more properly through now, but thought I'd throw this patch up for now, for anyone else wanting to give this a look. ;)

Freso’s picture

Status: Needs review » Reviewed & tested by the community
freso@nayru /s/h/l/h/drupal7> patch -p0 < ../patches/throttle.patch
patching file CHANGELOG.txt
patching file includes/install.inc
patching file includes/module.inc
patching file includes/session.inc
patching file modules/block/block-admin-display-form.tpl.php
patching file modules/block/block.admin.inc
patching file modules/block/block.install
patching file modules/block/block.module
patching file modules/statistics/statistics.module
patching file modules/system/system.admin.inc
patching file modules/system/system.install
patching file modules/system/system.module
patching file modules/throttle/throttle.admin.inc
patching file modules/throttle/throttle.info
patching file modules/throttle/throttle.module
freso@nayru /s/h/l/h/drupal7> grep -ir "throttl" *
CHANGELOG.txt:- Removed throttle module:
CHANGELOG.txt:    * The throttle module has been removed. Alternative methods to improve
CHANGELOG.txt:    * Impose a throttle on indexing of large sites.
CHANGELOG.txt:    * Added a mechanism to throttle certain operations.
CHANGELOG.txt:    * Refactored the throttle module configuration.
CHANGELOG.txt:- Added throttle.module:
CHANGELOG.txt:    * Auto-throttle congestion control mechanism: Drupal can adapt itself based on the server load.
CHANGELOG.txt:- Rewrote watchdog and submission throttle:
modules/CVS/Entries:D/throttle////
modules/search/search.admin.inc:  // Indexing throttle:
modules/search/search.admin.inc:  $form['indexing_throttle'] = array('#type' => 'fieldset', '#title' => t('Indexing throttle'));
modules/search/search.admin.inc:  $form['indexing_throttle']['search_cron_limit'] = array('#type' => 'select', '#title' => t('Number of items to index per cron run'), '#default_value' => variable_get('search_cron_limit', 100), '#options' => $items, '#description' => t('The maximum number of items indexed in each pass of a <a href="@cron">cron maintenance task</a>. If necessary, reduce the number of items to prevent timeouts and memory errors while indexing.', array('@cron' => url('admin/reports/status'))));
modules/system/system.install: * Remove throttle columns and variables.
modules/system/system.install:  db_drop_field($ret, 'blocks', 'throttle');
modules/system/system.install:  db_drop_field($ret, 'system', 'throttle');
modules/system/system.install:  variable_del('throttle_user');
modules/system/system.install:  variable_del('throttle_anonymous');
modules/system/system.install:  variable_del('throttle_level');
modules/system/system.install:  variable_del('throttle_probability_limiter');
modules/throttle/CVS/Repository:drupal/modules/throttle
modules/throttle/CVS/Entries:/throttle.admin.inc/-1.2/Tue Jan  8 10:35:43 2008//
modules/throttle/CVS/Entries:/throttle.info/-1.5/Wed Feb 20 19:37:22 2008//
modules/throttle/CVS/Entries:/throttle.module/-1.83/Fri Dec 14 18:08:49 2007//
sites/all/modules/coder/tests/coder_style.inc:<p>If you want certain blocks to disable themselves temporarily during high server loads, check the "Throttle" box. You can configure the auto-throttle on the <a href="@throttle">throttle configuration page</a> after having enabled the throttle module.</p>
sites/all/modules/coder/tests/coder_style.inc:<p>You can configure the behaviour of each block (for example, specifying on which pages and for what users it will appear) by clicking the "configure" link for each block.</p>', array('@throttle' => url('admin/settings/throttle')));
sites/all/modules/coder/coder.module:      'throttle' => 1,

As can be see from the above grep, the only remains of throttle are either expected on in contrib land. Site seems to operate fine without throttle as well, and the update runs smoothly. :)

catch’s picture

StatusFileSize
new29.89 KB

re-rolled for concat spacing.

hass’s picture

I wasn't in Boston... but why don't we need this module anymore to reduce server load? Should i see block caching as a replacement or is there anything other new I'm not aware about?

catch’s picture

hass: afaik the throttle module was originally designed to help sites survive 'slashdotting'. In my view that's best served by aggressive caching for anonymous users - which operates at a much more fundamental (and effective) level compared to throttle, and we currently tell people that throttle and aggressive caching are incompatible via the standard performance settings warnings.

For authenticated users block caching does a lot as well. There were some improvements mooted on the original block caching issue it'd be good to get into D7 -like setting custom cache expiry for individual blocks. That would further complicate the blocks interface, and is an argument for removing throttle by itself if there was a need to use that space for something more robust. Additionally, the registry patch will help sites running lots of modules to load less code per request, and I doubt many modules are using hook_throttle any more (it appears to be only statistics module in core that does so).

So essentially, we've got more generic, more effective solutions in core (and even more in contrib with pluggable caching) and we can do a lot to tidy up interfaces and disentangle some very low level core code at the same time.

David_Rothstein’s picture

Status: Reviewed & tested by the community » Needs review

This looks pretty good... although the actual removal of the throttle module seems to have gotten lost between #3 and #5 ;) Getting rid of all that entangled code is definitely very nice.

However, am I right in thinking that if this patch is applied, it won't be possible to have a throttle module at ALL (even in contrib)? If so, that might not be so good. I see what you're saying that in most cases aggressive caching is a better solution, but I don't think that's always true. (For example, you might be running other modules on your site which separately prevent you from using aggressive caching. Or you might have a very expensive module enabled that you want to throttle even for logged-in users.)

Just looking at your patch, it seems like there are only two real places in core where the throttling itself actually takes place (not counting that random thing in the statistics module, which seems unnecessary). These are module_list() and _block_render_blocks(). In both cases, the pattern is basically "generate a list of 'things' (modules or blocks), and skip over the throttled ones."

I am just wondering to what extent a new hook could be added that would let contrib modules do stuff here? Something like hook_module_list_alter() or hook_block_list_alter (or maybe combine them into a single more generic hook). Basically, something where Drupal says "I just made a list of things I'm about to do some processing on; does anyone want to remove things from that list before I do it?"

Hm, I guess hook_module_list_alter() might be tricky (given where it's called from it can't really be a traditional "hook"). But perhaps there's still a way. If so, this would let Throttle easily become a contributed module, and maybe such hooks could be useful in other scenarios too? They could definitely allow for a richer variety of throttling solutions in contrib; for example, a "throttle by role" option that throttles the site for some users but allows others (e.g., site admins) to still get the full functionality.

Anyway, just thinking out loud here ;)

dries’s picture

Status: Needs review » Needs work

Patch no longer applies, but otherwise looks good.

Freso’s picture

Status: Needs work » Needs review
StatusFileSize
new29.43 KB

Heh. Catch had gotten the patch for #244942: outdated function name in schema description mixed into this, and since that has already been checked in, that hunk obviously failed. Re-rolled patch without this hunk. :)

catch’s picture

@David. Yes there's currently no way to do this in contrib.

As far as I'm concerned, caching makes block throttling pretty much redundant (although we could do more to make block caching a bit smarter, but that's a different issue).
Statistics module is the only module which actually disables parts of itself 'intelligently', so all we have is 'switching arbitrary bits of your site off when it gets busy' which should come a very far third behind optimisation and caching (and shouldn't be an option at all on any site which has high traffic or a regular user base.

cwgordon's steps patch might allow for the kind of thing you're describing with hook_module_list_alter. The registry patch will also make things like this more possible. However I think these are good feature requests once those patches get in, and I don't think we lose anything by taking throttle out now except some very entangled code and complicated UI.

More importantly, there's a real risk for misconfiguration with throttle.module. On a 5.x site, I just played with it a bit - set the throttle settings to a very low level, throttled the search module, then disabled throttle module, and my search was still off. I haven't tried to reproduce this on HEAD yet, but still.

Rerolled again with the proper cvs removal (edit: and again).

Stefan Nagtegaal’s picture

Status: Needs review » Reviewed & tested by the community

This is ready... :-)

Freso’s picture

StatusFileSize
new39.71 KB

You forgot the "throttle" column in the "system" table (it was included in your earlier patches). Otherwise looks good, so I'll keep the RTBC. :)

(Oh, and I also found and removed a trailing white-space with the grep regex I showed in IRC yesterday. ;))

catch’s picture

Status: Reviewed & tested by the community » Needs work

Slow down people! There's typo in install.inc. Looks like no-one tested the install (including me).

Working patch coming up in about 5 minutes.

catch’s picture

Status: Needs work » Needs review
StatusFileSize
new39.71 KB

OK. I've just tested this one and it should all be fine. Thanks for the speedy reviews/RTBCs and sorry for silly typos.

Freso’s picture

A visual review says that all previously reported problems are fixed in this one (and my grep regex doesn't catch any trailing white-space ;)). Waiting for a second pair of eyes and hands to look it over before marking RTBC, but I think it's there.

David_Rothstein’s picture

Status: Needs review » Reviewed & tested by the community

@catch: OK, cool, thanks. You're right that block throttling has very limited use cases now (and besides, that hook would be easy to add if it's ever needed). And it sounds like something along the lines of hook_module_list_alter() will be possible too, although the best way to do it is still up in the air. All sounds good. I just wanted to check that there is some kind of foreseeable path whereby this might be able to eventually exist in contrib... seeing as how there is, I guess it's time to say R.I.P. to the throttle module ;)

I tested the patch in #16 both with a new installation and an upgrade from D6 (with throttle module enabled). I also reviewed the code and did a final "grep" to make sure the throttle stuff is all gone. Everything seems to work fine...

catch’s picture

Status: Reviewed & tested by the community » Fixed
Freso’s picture

Status: Fixed » Reviewed & tested by the community
StatusFileSize
new3.7 KB

And http://drupal.org/cvs?commit=111061 too ;) But throttle.admin.inc is still in the repos though...

dries’s picture

Status: Reviewed & tested by the community » Fixed

Finally removed them all now ...

Freso’s picture

There, there Dries. It's gone now. :)

( http://drupal.org/cvs?commit=111284 )

dries’s picture

Finally removed them all now ...

Anonymous’s picture

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.

yakupaltas’s picture

wwe

David_Rothstein’s picture

Status: Closed (fixed) » Needs review
StatusFileSize
new968 bytes

This is like a bad horror movie... the throttle module just won't die! I noticed that somehow, the throttle column is still hanging around the {system} table in the database! Not sure what happened since the patch that was committed sure looked like it got rid of it. Believe it or not, I think that somehow there were two copies of the 'throttle' definition in system_schema(), and the committed patch only got one of them and left the other behind.

Anyway, this should kill it for good :)

catch’s picture

Title: Remove throttle module » Remove throttle module again
Status: Needs review » Reviewed & tested by the community

Nice find David.

dries’s picture

Status: Reviewed & tested by the community » Fixed

Committed to CVS HEAD. For real, now. Thanks David.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for two weeks with no activity.