I do not need bots at all. I bought my traffic and I do not want my sites to show up on search engine or anything.

So I added this:

<IfModule mod_setenvif.c>
# remove most popular bot
SetEnvIfNoCase User-Agent ".*alexa.*" bad_bot
SetEnvIfNoCase User-Agent ".*google.*" bad_bot
SetEnvIfNoCase User-Agent ".*bing.*" bad_bot

Am I doing it right?

Basically any user agent containing alexa google or bing will not pass

  • Alexa is junk, but I would not block Google, Bing, or any other search engine that can help you. It is free traffic! Plus buying traffic has it's own issues. That can be a topic unto itself. – closetnoc Apr 25 '15 at 23:23

I may be somewhat overboard with my regex part, but these lines in your apache configuration will block them. Basically it scans the entire user agent identification string to see if it contains "googlebot", "bingbot" or "alexa" and if it does then direct users to the default error 403 page

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^(.*)googlebot(.*)$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)bingbot(.*)$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)alexa(.*)$ [NC]
RewriteRule .* - [F,L]

And if you want to set a custom error 403 page, use this line:

ErrorDocument 403 /path/to/errordocument.htm

  • I want to add that what you (OP) did was set an environment variable if the incoming user agent string matches one of the three that you're looking for. – Mike Apr 25 '15 at 16:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.