2

From http://www.allthingsdemocrat.com/block-bad-bots-in-htaccess.txt:

<IfModule mod_rewrite.c>
RewriteCond %{QUERY_STRING} http\:\/\/www\.google\.com\/humans\.txt\? [NC]
RewriteRule .* - [F,L]
</IfModule>

I want to put the whole .htaccess on my site.

What does it actually do?

  • 1
    Whenever you see slashes (and colons) that are backslash escaped in .htaccess you have to wonder whether they actually know what they are doing. – MrWhite Apr 25 '15 at 9:44
  • It is good practice to escape all literals that are not alphanumeric. While not all symbols currently have special meaning, that could change. Regular expressions reserve ALL symbols as potentially having special meaning in a later version. – Stephen Ostermiller Apr 25 '15 at 12:49
2

It looks like that code was developed to stop bots probing humans.txt via query strings.

  • Not Blocked: http://example.com/humans.txt
  • Blocked: http://example.com/?some_path=http://www.google.com/humans.txt?

There is plenty of online guides about blocking humans.txt additionally lots of websites explaining what does what, a lot of those rules in that block bad robots list you pasted is either old or just useless. Nasty bots don't announce themselves with there own unqine user agent. And increasing the size of your .htaccess can slow down your site slightly and even blocking bots uses bandwidth as its a request that can't be prevented.

I generally see a lot of people spending lots of time within .htaccess files while in a lot of cases its best spent doing security audits else where, such as file permissions etc.

0

This line:

<IfModule mod_rewrite.c>

Checks to see if mod_rewrite is installed in apache and if it isn't then anything between the above line and...

</IfModule>

is ignored.

This line:

RewriteCond %{QUERY_STRING} http\:\/\/www\.google\.com\/humans\.txt\? [NC]

checks to see if the URL contains http://www.google.com/humans.txt right after the question mark and the casing doesn't matter (because of NC). If there's a match, then the next line is executed...

RewriteRule .* - [F,L]

Which means if the URL is something like:

http://example.com/a/b/cde.asp?http://www.google.com/humans.txt

Then the user is denied access because of the F from the F,L flag and because the request matches the regex of .* which means match anything any number of times.

Apache doesn't even scan for files when it processes these rules that you show. That URL is probably something a hacker invented.

What I would do if I were you is to switch your links over to friendly URLs (urls that don't contain a query string attached to it) and redirect anyone trying to inject an HTTP:// anywhere in the URL to a failed page via these rules:

<IfModule mod_rewrite.c>
RewriteCond %{QUERY_STRING} (.*)http\:\/\/(.*) [NC]
RewriteRule .* - [F,L]
</IfModule>
  • Good advice! Often, the .htaccess is under used. I have a boat-load of these types of traps and I need to look for more and organize it into a page. I did one years ago, but it needs updating by now- still, there is good advice in the old code. Someday I will ping you and see what I am missing. – closetnoc Apr 25 '15 at 23:11
  • "friendly URLs" don't necessarily eliminate the need for a query string. – MrWhite Apr 27 '15 at 12:06
  • In the back-end of processing, you'll still have to deal with query strings, but in the front end you can make it look friendly (a url without the query string shown) – Mike Apr 27 '15 at 15:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.