kill spam via mod rewrite
andre75 - November 20, 2007 - 05:38
I am trying to block spam via .htaccess, since I am getting hammered. Even though akismet finds them all, it is getting compute intensive, so I was considering adding yet another level on the web server itself.
The basic idea is that legitimate spam (also called regular comments), will be coming from my site.
However the code below doesn't seem to work
# Blocking Comment Spam
RewriteCond %{REQUEST_METHOD} POST
RewriteCond %{REQUEST_URI} ^.*comment/reply/.*$
RewriteCond %{HTTP_REFERER} !.*aguntherphotography.com.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^$
RewriteRule .* ^http://www.myspace.com [R=301,L]
...
Why do you have a '^' in your RewriteRule?
"doesn't seem to work" doesn't help us, because we don't know what _does_ work. There's a general method to solve a problem: try to solve a less complex problem first. In other words, remove things from your recipe till you arrive at something that works.
You shouldn't redirect your users to myspace. Redirect them to some page that tells them they don't have the correct "referer". I sometimes "surf" with the referer disabled. Also, use '[NC]', because domain are case-insensitive. Better start with one of the numerous recipes you can find on the web.
Not sure
Thanks for the tips!. I am not sure how the '^' got there. I am just using myspace for testing, usually I just deny access thats it.
Again thanks for the tips, I will start playing a little more with this.
I am not sure how you surf without the referer disabled (or did you mean user agent?). Either way, if you chose to do this, I chose not to let you comment. No big deal, as this will rarely ever hit a legit commenter but cut down a great deal on spam.
I will consider redirecting to a static page with a short message telling people what to do. This seems to be the sensible thing.
Andre
P.S.: Sometimes I feel all I ever do is fighting spam. I really enjoy interacting with my users and keeping the comments, but often I wonder if I should just disable them.
-------------------------------------------------
http://www.aguntherphotography.com
...
Have you tried other solutions? the captcha module for example? or the spam one?
http://drupal.org/project/captcha
http://drupal.org/project/spam
There are more. If nobody has written a page in the handbooks about fighting spam I suggest you start a new thread with this question (start afresh; don't mention the mod_rewrite thingy).
My browser has a menu option to toggle the 'referer'.
That's fine. I didn't complain about this but about the fact that I'll be directed to some other site without knowing the reason. I wouldn't even know whether that was intentional or maybe you had a bug in your site. I'd waste my time trying to figure out what had happened.
Resources
Thanks for the tips. Akismet is fully capable to deal with the spam, however I do like to fight it on every level including .htaccess in order to limit resource usage of my servers. Its taking on astronomical proportions.
-------------------------------------------------
http://www.aguntherphotography.com
Here
Actually the idea was outlined here:
http://codex.wordpress.org/Combating_Comment_Spam/Denying_Access
I have removed the user agent and it looks somewhat like this:
RewriteCond %{REQUEST_METHOD} POSTRewriteCond %{REQUEST_URI} http://.*aguntherphotography.com/comment/reply/.*
RewriteCond %{HTTP_REFERER} !.*aguntherphotography.com.* [NC]
RewriteRule (.*) ^http://%{REMOTE_ADDR}/$ [R=301,L]
but whenever I go directly to my-url.com/commen/reply/xxx (xxx = node number) I still get the page
so I tried:
RewriteCond %{REQUEST_METHOD} POSTRewriteCond %{REQUEST_URI} http://.*aguntherphotography.com/comment/reply/.*
RewriteRule (.*) ^http://%{REMOTE_ADDR}/$ [R=301,L]
Still no go.
Looks like I got the REQUEST_URI thing wrong.
I tried these combinations:
RewriteCond %{REQUEST_URI} http://.*aguntherphotography.com/comment/reply/.*RewriteCond %{REQUEST_URI} /comment/reply/.*
RewriteCond %{REQUEST_URI} .*/comment/reply/.*
RewriteCond %{REQUEST_URI} ^.*/comment/reply/.*$
I also tried this:
RewriteCond %{HTTP_REFERER} .RewriteCond %{HTTP_REFERER} !^http://(www\.)?aguntherphotography\.com [NC]
RewriteRule ^.*comment/reply/.*$ - [F]
Hope thats enough info to help, since I am all out of ideas (again)
-------------------------------------------------
http://www.aguntherphotography.com
...
That doesn't necessarily mean there aren't errors on that page.
(Sure. The first line in the recipe checks for a POST request. When you type something into the browser's address bar it's a GET request. "POST" is when you click the submit button of a form.)
(You forgot the '!' ahead of the 'http'.)
Doing mod_rewrite stuff is tricky. And its debugging is hard --surely for the non-guru. Let me know why you wouldn't use a ready-made solution. Hey, you aren't the only one attacked by spammers. You have a nice website, BTW.
Death to Spambots !!!
With some experimentation, I found the right mod rewrite code for catching spambots:
RewriteCond %{REQUEST_URI} ^/comment/reply.*RewriteCond %{REQUEST_METHOD} POST
RewriteCond %{HTTP_REFERER} !^http://(www\.)?domain1\.com [NC]
RewriteCond %{HTTP_REFERER} !^http://(www\.)?domain2\.com [NC]
RewriteRule ^(.*)$ - [F]
Above syntax allows you to specify multiple referral domains that will not be stopped.
Unfortunately returning "403 Forbidden" to spambot requests (as done above) is not very effective. Why? Because many spambots are really dumb and keep issuing the same request over and over. This could be as taxing to your server as no fix at all.
My attempt to solve this problem is to issue a redirect to download a huge file (think gigabyte-sized ISOs) hosted by large software companies that can afford the bandwidth and computing resources to battle spammers. This can quickly tie up zombie machines used by a botnet. Use your judgetment in picking your lucky redirect target. Last line would be like this:
RewriteRule .* http://tinyurl.com/lucky-host-of-a-gigabyte-ISO [L,R=302]Please report how well this fix is working for you.
...
I mentioned earlier that it's possible to browse the web with referrer logging turned off. So it's better to redirect to a "spam attack detected" page and to explain there this referrer thingy (or you may put this explanation in the template you serve for 403). Don't cause your users to download 1GB.
Works for 99.99% users
Sure this is an interesting corner case, but most users are not savvy enough to turn off referrers. If they indeed are hacking their HTTP requests, maybe banning them from posting comments is not a big price to pay.
Returning anything to these spambots is resource-prohibitive when site comes under heavy DoS attack by a large botnet. If you have better ideas for deterring DoS damage, I am all ears.
...
I didn't oppose to your scheme. It's just that I suggested displaying an explanation to the users. It's not the ban that I complained about. It's not being informed about it.
Use your code, but add the following:
ErrorDocument 403 "You were detected by our anti-spam mechanism as a robot blah blah blah blah. Please make sure you've enabled referer logging in your browser. blah blah blah blah"Apples vs. Oranges
We are talking about different things here. What I am trying to figure out is how to cope with a Denial-of-Service attack that is brining the site down, not how to devise the friendliest false positive explanation.
The problem is that returning any 403 message does nothing to prevent DoS as spambots keep hammering (hundreds of requests simultaneously). What I want to figure out is how to tie them up, so they cannot keep hitting the site.
Redirecting to a huge download is one idea, but I am interested in hearing more.
...
Not sending me to a 1GB download isn't about being friendly. It's about not being nasty.
I acknowledge the spam problem --I tried to help earlier in this thread. I don't have a solution. And if you read me carefuly you'll notice that I didn't reject your kind of solution.
I am looking for ways to
I am looking for ways to stall the spambots from sending repeat requests without tying up my server resources.
Redirecting to a huge download is the best I can come up with. Being friendlier to humans who browse without referrers turned on is nice but secondary importance. If you have any suggestions how to accomplish both, I am very interested.