Should an IP be automatically blacklisted for a given timeframe after so many bad requests within a minute?

Question

This seemed like the best site to ask this question based on the tour so here goes. I have a WordPress blog site that I've developed a custom theme for and when someone lands on the 404.php file from a bad request it will send an email with the request, remote address, remote identity and user agent (user agent is never a search engine/bot) that is sent to an old box which has a cron that builds a CSV file for review. Sometimes I can get 50-200 requests from the same IP at that time within a two minute window. Some of these requests are:

url/wp-content/uploads/bc.php
url/wp-content/themes/THEME/style.php?act=1
url/wp-content/themes/THEME/adodb.class.php
url/wp-content/themes/b374k-2.8.php
url/wp-content/plugins/onleinsfay.php
url/wp-content/plugins/bc.php
url/wp-content/plugins/cache/adodb.class.php
url/wp-content/languages/cache/adodb.class.php
url/templates/_hash.php
url/templates/adodb.class.php
url/SocketIontrol.php
url/wp-content/plugins/hungred-post-thumbnail/js/hpt_ini.js
url/wp-content/plugins/invit0r/lib/php-ofc-library/ofc_upload_image.php
url/wp-content/plugins/mailpress/mp-includes/action.php
url/septii.php
url/wp-content/login.php?login=cmd
url/wp-cache.php
url/wp-content/error-log.php
url/upgrade-network.php

I have no plugins on the live site, I've checked to see if an error_Log is being generated on the server and it isn't. Before every WP update I test locally my theme with Debug Bar to make sure no PHP errors exist. I've checked the IP address with arin and the location is sometimes different. I've heard there are some sites that can test to see what themes and plugins you're running but I've never used one nor do I know how to identify one so I'm unsure if that is what they are doing but should I address this as an issue?

One solution I had contemplated with PHP was to set the IP address to a variable, get the current time and count how many times that IP hits the 404.php within a min. If they hit the 404.php file 10 times add their IP to a txt file and let the .htaccess reference it. To prevent my site from having performance issues I would script a reoccurring event every 24 hours that uses file_get_content() to clear the .txt file.

However, I am unsure if this is an issue or if I am approaching the issue correctly. Should I do anything about these bad requests and is there a way I can do this that would be better then my solution?

Answer 1

These look like vulnerability profiling. Basically, they are looking for vulnerabilities to exploit.

If you can block these, you should.

These are likely coming from open proxies. Instead of blanking out your "hacker" log file, I suggest using a forgiveness model. This means that any IP address that behaves for a period, a long period (days), then you can delete it from the file. The rest stay as long as they remain active within the forgiveness period. Keep in mind, you are not blocking IP addresses in user blocks. These are compromised computers likely on host networks. So there is no danger in blocking users. Hackers do not use user blocks anymore. It is too easy to get caught and with automated scripts (script kiddie), it is too easy to fire the hack off and let it go on it's own without tracing back to the hacker.

I do something a bit more sophisticated, however, it sounds like you have a solid solution.

Answer 2

A while back, I noticed many hits to my websites (Drupal) involving Wordpress paths which obviously can be ignored (nothing happens other than a huge waste of time generating 404 pages that the hacker is going to throw away immediately...)

For those websites, I would run a tool that I wrote (still have it, obviously) called iplock The source is on github.com here.

The idea is to detect the wrong path with whatever means you have and then call iplock with that IP to block it. iplock can block all accesses or specific ports (i.e. 80 and 443 for HTTP[S], but bad guys can be turned off 100% anyway... why would you let them to your mail server, DNS, etc. if you already know they are hackers?!)

The Apache2 setup requires a PHP or some other script that can be executed on specific paths. For example, you could send those people to a "special page", but internally (i.e. no actual 301 redirect)

RewriteEngine On
RewriteRule "^/wp-" "/iplock.php" [passthrough]

Now the iplock.php can be a system call to the iplock binary with the proper parameters (please test to make sure it is correct, I am not testing myself here...)

<?php
system("iplock --scheme all --block " . $_SERVER["REMOTE_ADDR"]);

This will so quickly block that IP address that you are not likely to see more than 2 hits from the same IP before you see your iptables -nvx -L list show you a counter growing fast (at least that's my experience with those guys).

Note that the iplock tool is nothing more than a call to iptables to add a rule in there. You could just use iptables it's just a little more complicated and it does not manage as many possibilities and it requires you to be root to run it. Many things that can be complicated to go around depending on your level of competency.

The iptables command looks like this in the all.conf of iplock:

block=[command] -I [chain] [num] -i [interface] -p tcp -m tcp -s [ip] -j DROP

• [command] is iptables
• [chain] is INPUT or some chain you create just for that purpose
• [num] is the position at which the new rule will be inserted, in most cases we like to add new rules at the start, it is slower to add, but much faster to block that current user (it will appear first so it blocks the IP quickly)
• [interface] is something like eth0
• [ip] is that $_SERVER["REMOTE_ADDR"] address

Just remember that for security reasons, giving Apache the permission to run iptables directly is a really bad idea. The iplock tool hides that part and that's because it uses secure files to run the commands that it is safe.

Now... if your site is a Wordpress site, blocking all the /wp-* paths is not going to work for you, although if you have a static IP address, then you could allow your IP do do so and block anyone else as shown here. This means one extra rule in your Apache setup:

RewriteEngine On
RewriteCond %{REMOTE_ADDR} !^1\.2\.3\.4$
RewriteRule "^/wp-" "/iplock.php" [passthrough]

This RewriteCond rule says: execute the following Rewrite... command only if the REMOTE_ADDR does NOT match 1.2.3.4 so if you make 1.2.3.4 your static IP address, you will not get blocked, but everyone else will.

You can also do that for customers, but then that can become complicated since they may want to access their website from work, home, on travel... Although if you are a developer you could go around that by asking them to register the IP address in some automatic way first (i.e. go to page blah.com and that will open the door for your current IP address, you could even make that open door temporary, after like 8h, close it again...)

WARNING: people need to have access to /wp-content and /wp-plugins so the pattern above may need to be more restrictive such as ^/wp-admin.

Finally, there could be /wp-... paths that you do not use (i.e. would require a plugin you did not install). Those can be blocked even without the static IP address trick above. Thus you could have something like:

RewriteEngine On
RewriteRule "^/wp-(language|remote)" "/iplock.php" [passthrough]

which would block "/wp-language" and "/wp-remote". However, I do not think such happen very much. It could also be more complicated paths such as a specific content path or plugin path.

Answer 3

You can use or create a text file based on the IP addresses which you can fetch with the help of any backend language like PHP and then check if a request with a particular IP address comes in a very short time also you can set the time period around 300 milliseconds then you can just Block that site to be used by a particular IP address

You can do this by just giving a warning message to the user or the bot which is just sending you request in a very short time