Hi,

I am fighting sustained attacks on 2 sites, ~900/hr at each, 24/7 for several weeks now. They are a mixture of login attempts using random usernames (the Login Security module blocks those hosts), attempts at posting (the Captcha module blocks those) and various PHP and other exploits, which Phpids is working very well at blocking.

Nothing has ever got through. There is a lot of crossover between IP's, the same IP's are often trying all 3 forms of attack, and some are attacking both sites. Some have made hundreds of dumb and pointless attempts.

Phpids has proved extremely useful and accurate, but most of the crap it blocks include long lists of fashion counterfeit, pharma and porn site links that gets logged to Watchdog. Such is the volume of bad activity that the log is exploding, quickly taking up all our webspace and bandwidth. Of 4GB/m only about 5% is legitimate use.

To minimise this impact I have been manually copying offending IP's to "deny from" directives in .htaccess. Although that has reduced logged intrusion events in Watchdog from 900/hr to 0-10, I have to spend time every day going through Watchdog and adding new "bad" IP's to .htaccess, usually around 20-30/day.

It would be wonderful if Phpids could either:

  • have an "extreme prejudice" setting which would write "deny from"'s direct to Drupal's root .htaccess.
  • or, an option to block repeat offenders with a terse "IP n blocked has been blocked x times" msg in Watchdog, without writing several k of put string and summary to Watchdog

Comments

patrickd’s picture

Status: Active » Closed (won't fix)

I'm afraid drupal 5 support was dropped
sorry

steeph’s picture

Version: 5.x-3.x-dev » 7.x-1.x-dev
Status: Closed (won't fix) » Active

I didn't even see that this issue was tagged 5.x. I think the feature would be very useful indeed.

halftone’s picture

Version: 7.x-1.x-dev » 6.x-2.x-dev
Status: Active » Closed (won't fix)

I'm now on D6 /Phpids 6.x.2.x-dev and the same issue of profligate bad behaviour persists, so can I reinstate a feature request? I've changed the Version accordingly.

The Troll module allows manual hard blocking of IP's and there was some talk of integration with Phpids module at http://drupal.org/node/425582 but Troll appears to be tumbleweed city now. Still most of the code to impose hard IP bans for excessive PHPIDS impact appears to exist. If only I wasn't a php remedial...

patrickd’s picture

Status: Closed (won't fix) » Active
patrickd’s picture

Status: Active » Closed (duplicate)

PHP IDS 7.x-2.x, 6.x-3.x will be integrating with core ip-blocking.

See pending sprint tasks.

halftone’s picture

StatusFileSize
new7.48 KB
new3.91 KB

Excellent. Thank you. Although I just saw http://drupal.org/node/1570102 which seems to place doubt over whether IP blocking will remain in core. I didn't use the core access blocking because Troll is (according to the module docs) much more efficient by blocking early in the bootstrap phase.
Meanwhile and because I need this functionality ages ago, I had a go at adding a few things to Phpids 6.x.2.x-dev.

  1. Ability to turn off logging for blocked IP's via checkbox in the admin. I have 3 sites which have been heavily assaulted with exploit attempts that often each run to 80 or more links several times a minute. They are on shared servers with limited disk and bandwidth allocation that quickly gets filled with logged attempts. If logging is turned off, Phpids only logs the IP address that has been blocked. Obviously if you are getting false positives do not turn logging off.
  2. If the Troll module is installed it's now possible to enable Troll IP blocking as a further defence, and set a Troll impact at which this happens. These options only appear in the Phpids admin UI if Troll is installed. A ban is written into the troll-ban-ip DB table, which means the entire site is forever "access denied" to the offending IP. At present the ban is permanent with no way of cancelling it except deleting the entry via the Troll UI. This is an "extreme prejudice" option:).

I'm sure this can be done much better by the maintainers, so this is just a temporary fix. But if anyone else needs the abilities right now, try the attached patches. It's hugely enjoyable watching attackers ban themselves. In less than a day 75% of them have wiped themselves out of my logs, which is nice.

One thing to watch is that I changed the 'level' textfields to select dropdowns with values of 1-30 but Drupal seems to be ignoring defaults. I spent a lot of time grappling with this and gave up; it seems to be a FAQ without a proper answer. Once set, levels are stored and retrieved OK so do make sure you set them. Personally I'm Troll banning anyone who hits level 3. Though most of my profligate offenders hit 9-40 I've not yet seen a 3 that was a false positive. And I'm really fed up with them.

I also changed the info mail subject line around from "Phpids detect a ... [domain]" to "[domain] Phpids detect a...". I will be running this on several sites and this way round makes the mails much easier to filter.

halftone’s picture

Just a follow up for anyone wondering if hard blocking is worthwhile. It appears so. On my test live site, which is the least attacked of three, 78 IP's were hard blocked at the rate of about 1/hr. Since then, absolutely nothing in 24hrs, my logs are clean for the first time in many months. It looks as if the botnet responsible simply ran out of hosts to use, since they were all over the world.

[edit: I spoke too soon. After a break they've resumed again. Someone must have taken the botnet down for maintenance :(

halftone’s picture

A further follow up after 8 weeks of use. Hard blocking (now a total of 904 IP's) is having no significant effect on the number of attacks, which has varied between 102-132 per week during this period. There is no downward trend in the number of attacks so I conclude hard blocking has no real advantage over disabling logging (to prevent Watchdog logs getting very large) and allowing Phpids to soft block ad hoc.

Obviously there is a risk of slowing the site as the pool of banned IP's grows, as every visitor IP is checked. This isn't noticeable with ~900, but at some point Phpids soft blocking is going to be more efficient.

I've spent some time looking at the blocked IP's. About 60% are static, 40% dynamic. Obviously there is little point and a risk of collateral damage by hard blocking dynamic IP's, but blocking around 500 static IP's has not slowed the attacks. From which I conclude that the originating botnet is so large that removing 500 IP's is a drop in the ocean. Unfortunately, using Troll's hard blocking doesn't log any IP's that are denied access, but my feeling is that if an IP is blocked, the attack is simply transferred to another botnet machine until it gets through.

Not a successful experiment then.

halftone’s picture

Just a small follow-up in case anyone is interested. Perhaps I spoke too soon. A year later, I now have ~13,000 IP's hard blocked by this patch, and logged Phpids attacks are now very few. In recent weeks no more than 5 a day, and some days nil, down from 40-60/day peak. On other sites where I haven't done this, the number of attacks has been maintained.