4

This is my first question on StackExchange so bear with me, here is a brief breakdown of my current setup:

  • Two different domains, two different TLDs (.dk and .de) on the same hosting plan
  • for the sake of privacy, let's assume the domain names are "example website", i.e. https://www.example.dk and https://www.example.de
  • SSL set up on both domains
  • WordPress-driven, using WPML to serve the translated content accordingly

The problem is, I'm getting a crazy amount of hits (from countries like China, India, and Pakistan) for /not_found at the end of both domain names' URLs, in all possible formats:

  • secure, along with the www protocol (https://www.example....../not_found)
  • secure, without the www protocol (https://example....../not_found)
  • non-secure, along with the www protocol (http://www.example....../not_found)
  • non-secure, without the www protocol (http://eksemp....../not_found)

To me, this peculiar pattern looks most likely to be nothing more than vulnerability scan attempts, especially taking into account that the content on the sites is written in either Danish or German. Moreover, the websites are addressing to a specific Danish, or German target group, thus making it very unlikely for the websites to be worth a visit to the visitors from those Asian countries. Currently, the server's response is the 404-not found page provided by the WordPress theme that we're using, which in my opinion is not ideal because I would rather have them be denied access to that URL, even if it doesn't actually exist.

Long story cut short, I've been struggling over the past couple of days to figure out how to forcibly make the web server output the 403-forbidden page instead of the 404 one since the specified URL does not exist on the server.

I created a regular expression which I aim to use in the .htaccess file as a pattern that covers all the above-mentioned scenarios (4 for each domain name). The part of code that I placed in .htaccess goes as follows

# BEGIN /not_found blocking
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{REQUEST_URI} ^(https{0,1}:\/\/)(www\.){0,1}(example\.dk|example\.de){1}(\/not_found){1}$
RewriteRule . - [R=403,NC]
</IfModule>
# END /not_found blocking

but alas it doesn't work, as I'm still getting the 404-not found error page instead of 403 Access Forbidden. It must be my fault, but what's the reason?

Any help would be greatly appreciated. Thanks a lot!

  • 1
    Anything other than a 404 may encourage more and possibly different attempts. These are likely just kiddie scripts. If you want to block these, you may want to consider blocking by IP address. It could be a better solution in the long run. Cheers!! – closetnoc Feb 24 '18 at 21:17
  • You could just create the not found page and I'm php set the header status to 403 – Michael d Feb 24 '18 at 23:32
  • @closetnoc totally agreed, except for the fact that I'm already blocking a significant amount of IP addresses as well as IP classes in .htaccess. I'd rather prefer not to end up with a really big .htaccess file. I appreciate your input nevertheless! – GeekRider Feb 26 '18 at 9:13
  • @michaeld I was thinking of keeping this solution as a last resort, but I think I'll end up doing that anyway at some point. – GeekRider Feb 26 '18 at 9:15
  • @greekrider - The 403 status php page is an easy trick that I think works as well as well as htaccess. Either way, if these bots are really hitting your servers in high numbers I think you're going to have to block their ip. I've had bots attacking me, essentially causing a ddos and the only real option was to find their ips in the logs and block them in my htaccess files. – Michael d Feb 26 '18 at 9:23
1
RewriteCond %{REQUEST_URI} ^(https{0,1}:\/\/)(www\.){0,1}(eksempelhjemmeside\.dk|eksempelhjemmeside\.de){1}(\/not_found){1}$
RewriteRule . - [R=403,NC]

You would seem to be way overcomplicating things here. The REQUEST_URI server variable contains the URL-path only, ie. /not-found. And this is all you appear to be concerned about. You don't need to check for the domain, www/non-www, HTTP/HTTPS, since you seem to want to match all variations anyway. Unless there are some other subdomains etc. that you don't want to catch? (But in that case, it might be simpler to match the exceptions, rather than the target?)

However, as @closetnoc suggested in comments, I would still return a simple "404 Not Found" for these requests - since that is what they are. By "simple", I mean the default Apache 404, so you fail early, rather than passing the request through WordPress, which is what is currently happening. So, all you need is something like the following before the WordPress front-controller:

# BLOCK /not_found
RewriteRule ^not_found$ - [R=404]

No need for the <IfModule> wrapper, RewriteEngine directive (since that is already included in the WP block), or RewriteCond directive. When you specify a status code other than 3xx then you don't need the L (last) flag either (it is implied).

You might want to include the NC (nocase) flag, if you are getting requests for /NoT_FounD and/or /NOT_FOUND etc., but otherwise this should be omitted.

Note that in .htaccess, the URL-path matched with the RewriteRule pattern, does not include the slash prefix. So, the pattern ^not_found$ matches just the URL /not_found.

As an academic excercise, if you did want to return a "403 Forbidden" instead of a 404, then you would change the above directive to read:

RewriteRule ^not_found$ - [F]

Again, no need for the L flag when using the F flag. Alternatively, you can write R=403 instead, but F is the preferred shortcut.

  • 1
    Thank you @MrWhite for the crystal clear explanations, as well as for the bulletproof solution to my problem! – GeekRider Feb 26 '18 at 9:34
0

I use this htaccess code to remove certain words from URLs and redirect to the homepage. I modified it for your situation. I haven't tested this version but I think it should work. I'm not sure if something extra needs to be done for the underscore or not.

RewriteEngine On
RewriteBase /
RewriteRule ^(.*)not_found(.*)$ $1$2  [R=301,L]

You could just create the /not_found page and in php set the header status to 403

header('HTTP/1.0 403 Forbidden');

echo 'You are forbidden!';
  • 1
    That looks like a very useful snippet of code for situations where I'd want to handle other badly-intended requests, for instance when certain weird parameters are being sent as a part of the URL e.g. example.dk/ogShow.aspx?name=ogFoot&line=0&from=oGateeu which seem to be happening quite often according to the logged 404-not found entries. Bookmarked! – GeekRider Feb 26 '18 at 9:30
  • 1
    "modified it for your situation" - Although the OP is trying to block "vulnerability scan attempts", not fix the URL. "just create the /not_found page" - note that the URL is /not_found, not /not_found.php - so this would require an additional rewrite in order to serve the PHP page. However, creating a physical document for the sole purpose of serving an error like this is not recommended - it's not scalable and is slower than using Apache (an important factor when dealing with malicious requests). – MrWhite Feb 27 '18 at 11:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.