Requesting advice on which reverse proxy solution to use [closed]

Question

I would like to clean slate some internet facing infrastructure, looking for recommendations on what reverse proxy solution would be easiest / most sensible to use to do this. I'm aware of CDN's like cloudflare/bunny that could help here but A; I don't think we need their scale, and B; don't want their cost when I already run servers in a co-lo. I have beginner level experience in Nginx, Traefik, and Cloudflare. Heard of many other solutions for this and I want to know what I should focus my efforts on.

Current Situation;

1. Users go straight to these systems external IP
2. ~100 back end web pages inside co-lo
3. ~100 back end web pages scattered through several US states
4. Servers are pretty much all the same Tomcat web servers / copies of a enterprise app with a handful of outliers
5. Two domains
6. Each server runs it's own SSL certificate, wildcards; I'm concerned with the internet switching to ~45 day certificates in coming years.
7. DNS is an "A" record, unique entries for each server, each server has it's own IPv4
8. Not using IPv6 at all

Goals for finished solution

1. Geo restrict origin; US only, with updates as IP space gets bought & sold (maxmind?)
2. Automate SSL certificates
3. Changes in resource mapping or rules, or the app updates itself require little down time. We have no SLA, a system wide outage for a handful of minutes non peak time will be acceptable
4. Consolidate to single, new domain. DNS will be wildcard and point mostly everything to proxy.
5. Users using old domain; display message "transferring you to new URL, please update your bookmark" and then forward them to the new URL for that exact server.
6. Real time monitoring, what's in use and how much etc, OK for just table of text
7. Some sort of long life internally trusted SSL cert between proxy and back end resources

Stretch goals

1. Proxy failover; the back end resources themselves are not capable of load balancing but the proxy itself might be capable of some form of redundancy
2. Historical monitoring with history of a month or two, with charts and graphs
3. Basic WAF features, something like OWASP mod security, SQL injection, block by threat score, bot mitigation, etc
4. Block based on IP reputation, with auto updates
5. Block bad bots, with auto updates
6. IPv6 availability, terminate at proxy, IPv4 to back end.

Answer 1

I'll try to comment on each section when appropriate.

Goals for finished solution

1. Geo restrict origin; US only, with updates as IP space gets bought & sold (maxmind?) - CloudFlare (free and paid versions) has a built-in solution for this in their WAF. You can also specify lots of additional rules (restrict to country, User Agents, internet provider, etc).
2. Automate SSL certificates - CloudFlare / Digital Ocean / Most hosting providers handle this automatically with no additional cost unless you require banking level encryption.
3. Changes in resource mapping or rules, or the app updates itself require little down time. We have no SLA, a system wide outage for a handful of minutes non peak time will be acceptable - You can handle this internally, depending on your tech stack. You could look into CI/CD pipelines to streamline changes to prod environment. If handling multiple servers, you can have almost zero downtime managing your resources efficiently (the pipeline updates the 1st server, brings down the others and declares the 1st one in the load balancer which handles traffic alone for a while while the other servers become updated. Once each one finishes, it's redeployed on the load balancer).
4. Consolidate to single, new domain. DNS will be wildcard and point mostly everything to proxy. That's a good approach. It will simplify your operation and reduce cost.
5. Users using old domain; display message "transferring you to new URL, please update your bookmark" and then forward them to the new URL for that exact server. That's not the correct way to handle it. You should redirect them automatically using a 301 header code.
6. Real time monitoring, what's in use and how much etc, OK for just table of text. CloudFlare (free with little info / paid version with lots of info) / Digital Ocean (included with basic/little info). You could also signup for other more sophisticated paid tools like New Relic to extensively monitor everything.
7. Some sort of long life internally trusted SSL cert between proxy and back end resources CloudFlare / Digital Ocean / Most hosting providers handle this automatically with no additional cost unless you require banking level encryption.

Stretch goals

8. Proxy failover; the back end resources themselves are not capable of load balancing but the proxy itself might be capable of some form of redundancy. Load balancing is done at the proxy level. You could handle it internally (using Nginx for example without additional cost), or externally (CloudFlare with paid version or using Digital Ocean built in Load Balancer).

9. Historical monitoring with history of a month or two, with charts and graphs - CloudFlare (free with little info / paid version with lots of info) / Digital Ocean (included with basic/little info) / New Relic (paid version with lots of info) can handle this

10. Basic WAF features, something like OWASP mod security, SQL injection, block by threat score, bot mitigation, etc CloudFlare (free and paid versions) has a built-in solution for this in their WAF.

11. Block based on IP reputation, with auto updates - CloudFlare integrates this into their security protection transparently

12. Block bad bots, with auto updates CloudFlare (free and paid versions) has a built-in solution for this in their Super Bot Fight mod. The paid mode allows a little more customization. I prefer to use a combination from their automatic detection and manually blocking bots based on static rules

13. IPv6 availability, terminate at proxy, IPv4 to back end. CloudFlare / Digital Ocean handle this automatically with no additional cost

A few extra notes:

• My answers heavily lean towards CloudFlare since I use their services and you mention to have some experience with them. There are other providers that have similar solutions.

• Note that most of the things you require can be handled by CloudFlare's free tier.

• Upgrading to Pro services allow you to customize your services, ability to filter analytics and to handle 3x more rules.

• The only other thing extra in cost with them would be if using their load balancing service.

• From your description, you don't really need to go to a Business Plan which is expensive IMHO. With some tweaking and customization, you can work with the Pro Plan and achieve most of your goals.

• Another provider I mention is Digital Ocean (again, there are others that provide similar functionality). You can setup Droplets which are basically virtual servers that you manage directly. They have a built-in Load Balancer that it's very easy to configure and use. You can even resize these Droplets, or create new ones in a matter of minutes.

• From your list, Monitoring is not part of what CloudFlare can handle. They can give you analytics for traffic, WAF and security events.

• To really monitor your infrastructure, you need New Relic or a similar solution, (unless it's not really a priority). Digital Ocean also provides some basic monitoring included in their plans (resource alerts, uptime checks and CPU/memory consumption with graphics for up-to 14 days)