SWAG - UFW / iptables inbound from Cloudflare CIDR's only

This similarly has been posted before but without an appropriate solution for me: (Swag only allow cloudflare IP)

I have (only) 443 port-forwarding for SWAG and am additionally proxying all my traffic through Cloudflare. Therefore, I want to just have an allowlist of CIDR’s inbound from Cloudflare’s ranges (https://www.cloudflare.com/en-gb/ips/). and deny all other 443 inbound.

By proxying my traffic through Cloudflare my IP isn’t exposed by DNS, and I’ve additionally added a block in my nginx server config to return 444 if not coming via the DNS. But, this won’t stop connection attempts (e.g. nc) direct to the IP on 443 and will then return a 400 error in my access logs.

I’ve attempted via UFW but, due to the common (and widely discussed) issue with Docker and UFW not behaving (and a solution being to turn off docker’s iptables config, clearly not going to work with fail2ban which I am using through the SWAG container)

I’m considering if anything needs to be done directly to the iptables (either on my host, raspberry pi, or in the container itself, where fail2ban updates the iptables with blocks).

It would appear the cause of the issue is the DOCKER owned chain in my raspberry pi’s iptables. So this takes precedence over my UFW rules which are allowing only to Cloudflare’s cidr’s…

-A DOCKER -d 172.17.0.3/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 443 -j ACCEPT

Basically I just want to stop port-scans and brute force attacks on my ip; which are frequent from reviewing nginx access logs. Has anybody had any success with the SWAG container?