Reverse proxy goes offline randomly after accessing certain domains?

Hi all, I’ve been experiencing some very weird behavior lately. I use swag and Authelia for my reverse proxy + authentication needs, and am generally very happy. However, there’s this weird thing where sometimes any remote access going through swag just doesn’t respond. As in, I get the browser message “This site can’t be reached”, not even a server error. Going local to the machine + port is fine, and I can also still SSH in remotely using the public IP.

I figure it’s a swag issue, but the logs show absolutely nothing. Restarting swag also doesn’t work. However after a while, usually 15/30 minutes, everything is fine again. I was having this issue on Ubuntu Server 22.04 a couple months ago, and last month I updated to Debian 12 but the issue persists.

I’m honestly clueless as to what could be causing this. It’s not really reproducible, but usually it happens after accessing Navidrome or Portainer, especially when logging in with a wrong password. It’s wild to me how that could nuke my entire reverse proxy though. Maybe some kind of DDOS protection I’m not aware of on a system level, or fail2ban doing something?