By pavlos on

Many of you might have noticed hits in your referrals log from sites which are not linking to you. This is a known problem which affects search engine rankings severely and I will explain how. I didn't know where exactly to post this but it is crucial that the community is aware of this issue.

The trick is done by sites sending traffic to you with 302 redirects in the server header response. Now, when googlebot or msnbot visit the spammer's site and then there are 2 possibilities which I have detected.

The one is instantly sending the bots to your site with a 302 redirect

The other is by featuring a list of sponsored listings which have a few dynamically chaning links to various "sponsor" sites. They link to a special url in their site, which then sends a 302 server header response and the bot/visitor goes back to the site.

Now there are some networks which have thousands of domains on a few IP addresses, all running the same script. From what I can understand, they are aggregating their links from google results related to relevant keywords.

I have tried to block their IP addresses, but it does not seem to work for me. The following is what I have in my .htaccess (at the very top):

Order Deny,Allow

Deny from 216.34.131.

I am trying to block all hits from domains hosted in the 216.34.131.*** ip range.

It just does not work...

Then I tried adding this at the top:

RewriteEngine on
RewriteCond %{HTTP_REFERER} ^http(s)?://(www\.)?spammer.info.*$ [NC,OR]
RewriteRule .* http://www.google.co.uk [R,L]

which also does not block the referrals from the domain.

Any ideas on how to block this? If you have faced a similar attack from spammers, it would be very interesting to know what you have done to block them.

Additionally I think there should be a forum dedicated to SEO stuff specifically related to Drupal.

Regards,
Pavlos Skoufis

Comments

gdip’s picture

gdip commented

This one blocks "bad_bots" but should work equally well for your purpose. Name the variable eg. referral_spam.

SetEnvIf Remote_Addr "84\.19\.188\.27" bad_bot
 SetEnvIf Remote_Addr "195\.154\.174\.[0-9]+" bad_bot
 Order Allow,Deny
 Allow from all
 Deny from env=bad_bot

Use the example row with "195\.154..." for blocking an ip-range. You probably have to check whether SetEnvIf Remote_Addr is available for your Apache. Google for it.

pavlos’s picture

pavlos commented

Actually I do not need to block the visitors. Just need to redirect or deny access to visitors being referred from certain servers with specific IP addresses.

Your code is using the Remote_Addr variable, which is meant to be the visitor's IP address. I need to check the referring website if it belongs to a list of IP addresses which I do not want traffic from.

Thanks...

qasem’s picture

qasem commented

Here is some useful links about your problem; and try to read more about SPAM Module

http://drupal.org/node/24302
http://groups.drupal.org/node/4310
http://www.askapache.com/htaccess/fight-blog-spam-with-apache.html