Blocking referral traffic

Question

I am currently getting a massive amount of referral traffic to my site (Spam), that can sometime generate 1-2 GB worth of traffic per a day (which kinda sucks, when you have a 10GB limit from the hosting company).

I am seeing 4-5000 hits per day, generating 20-25000 page views... It seems that almost every day I'm having to add new sites to my .htaccess file to prevent my site being closed because I exceed my traffic limitations.

From what I can see in the logs, whenever I add a site to be blocked, the traffic just starts coming from a new site within a couple hours.

Here is something from my .htaccess file currently:

RewriteEngine On
RewriteCond %{HTTP_REFERER} ^http://www\.everyoneweb\.com [OR]
RewriteCond %{HTTP_REFERER} ^http://freefilearchive\.com [OR]
RewriteCond %{HTTP_REFERER} ^http://4runnerforex\.com [OR]
RewriteCond %{HTTP_REFERER} ^http://www\.julznakomua\.strefa\.pl [OR]
RewriteCond %{HTTP_REFERER} ^http://drugbuyersforum\.org [OR]
RewriteCond %{HTTP_REFERER} ^http://protopage\.com [OR]
RewriteCond %{HTTP_REFERER} ^http://stupidvideos\.com [OR]
RewriteCond %{HTTP_REFERER} ^https://sourceforge\.net [OR]
RewriteCond %{HTTP_REFERER} ^http://www\.sourceforge\.net [OR]
RewriteCond %{HTTP_REFERER} ^http://www\.thoughts\.com [OR]
RewriteCond %{HTTP_REFERER} ^http://www\.youfreeweb\.com [OR]
RewriteCond %{HTTP_REFERER} ^http://www\.facebook\.com\\notes\\phentermine [OR]
RewriteCond %{HTTP_REFERER} ^http://theresaraatt\.over-blog\.com [OR]
RewriteCond %{HTTP_REFERER} ^http://grou\.ps [OR]
RewriteCond %{HTTP_REFERER} ^http://www\.reverbnation\.com
RewriteRule (.*) http://%{REMOTE_ADDR}/$ [R=301,L]

order allow,deny
deny from 109.227.125.
deny from 109.227.124.
allow from all

Does anyone have an idea what could be done, about all that traffic coming my way ?

Answer 1

Well you can go here http://aaronlogan.com/downloads/htaccess.php and have a pretty big list of referrer spam blocks in your htaccess. Or you can use something like the following

# set the spam_ref variable - referrer site or a keyphrase
SetEnvIfNoCase Referer "^http://(www.)?some-spammer.com" spam_ref=1
SetEnvIfNoCase Referer "^http://(www.)?other-spammer.com" spam_ref=1
SetEnvIfNoCase Referer "^casino-poker" spam_ref=1

# block all referres that have spam_ref set
<FilesMatch "(.*)">
 Order Allow,Deny
 Allow from all
 Deny from env=spam_ref
</FilesMatch>

Or if you are using php.

<?php
$block = array("xxx.xxx.xxx.xxx", "yy.yy.y.yyy");

if (in_array ($_SERVER['REMOTE_ADDR'], $block)) {
  header("Location: http://google.com/");
  exit();
}
?>

Either way you will need to keep on top of it and update as necessary. And the list can get huge.

Answer 2

If this is an example of referrer spam, then the way to stop it in the long term is to make sure that your log files are not indexed by search engines. That makes this a valueless exercise for the spammer and eventually they will cotton on to this and stop doing it.

Answer 3

These kind of bandwidth stealing is common in web. you can deny all requests for your website files (images,videos,...) to except your website and its sub-domains and those domains you know.

and if requests are from special IPs you can block them!

otherwise if it's for all of your web pages what you're doing is only choice!

Answer 4

When dealing with a similar problem, I found that all requests came with a small variation of fake or unlikely User-Agent strings.

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; Media Center PC 6.0; InfoPath.2; MS-RTC LM 8)
Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 3.5.30729)
Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.221.7 Safari/532.2
Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; en) Opera 8.50
Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3
Opera/9.64(Windows NT 5.1; U; en) Presto/2.1.1
Mozilla/5.0 (X11; U; Linux i686; it-IT; rv:1.9.0.2) Gecko/2008092313 Ubuntu/9.25 (jaunty) Firefox/3.8
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.14) Gecko/2009082707 Firefox/3.0.14 (.NET CLR 3.5.30729)

If the same applies to your case, you could block requests that come from those fake UAs.