I have a Flask application I’m developing that relies heavily on external website interaction and is initiated by the end-user. If I leave the application without any sort of bandwidth control/rate limiting then this application may be abused by actors with nefarious intentions.
My goal is a fairly simple 2 stage approach:
Rate limit individual IP sources from performing more than
xnumber of connections a minute. This can be easily achieved with
iptables. Here’s a similar example to my goal:
iptables -A INPUT -p tcp --syn --dport 80 -m connlimit --connlimit-above 15 \ --connlimit-mask 32 -j REJECT --reject-with tcp-reset
Rate limit the applications ability to perform more than
xnumber of lookups per URL. Example:
APP ---- 10 pps ---> stackexchange.com PERMIT APP ---- 25 pps ---> google.com DENY / 15 SECOND BACKOFF
So far as I can tell,
iptables has no way of tracking separate URLs. It’s only able to rate limit these connections as a whole. That also doesn't seem like the only limitation to what I'm trying to achieve. If there were a way to setup
iptables in this way, it might provide some issues with my web application since these requests are user-initiated.
I’m using Flask, a viable option might be using a
before_request hook and manually tracking these destinations with a data store such as Redis. However, this is pretty high up in the stack to be dealing with connections in this manner. What I really need (or think I do) is an intelligent firewall application that can dissect requests in a custom way and close connections when certain breakpoints have been reached.
Is there any way to achieve what I’m trying to do?
If so, how?