Problems getting non-containerised (VM) reverse proxy working

I posted this on the Unraid forums, but apparently that’s not looked at very much - https://forums.unraid.net/topic/51808-support-linuxserverio-swag-secure-web-application-gateway-nginxphpcertbotfail2ban/?do=findComment&comment=1009444

So as not to reinvent the wheel, I’m just copy/pasting.

Initial query

Hi,

I’ve been tinkering around with SWAG today to set up a couple of Docker instances and a VM.

After watching SpaceInvader One’s YouTube video I’ve changed my router to now point to the Unraid server instead of the VM and both the Docker instances work, but I’m really struggling with the VM.

I have, for a number of years, been using Mail-in-a-Box (https://mailinabox.email) as my personal mail server on a Ubuntu VM. It works really well and also has inbuilt letsencrypt to automate certificate renewal.

Obviously SWAG does this too, but I don’t want to mess around with the VM config and break things. I’ve been reading through this thread and trying to get it working, but I’m just stumped as nothing I do seems to work (which means I’m obviously not doing something right)!

For info, MiaB uses box DOT domain DOT com as its default and also manages the webserver at www DOT domain DOT com. It also has an inbuilt DNS server which you point to from your registrar.

The comments I keep seeing from everyone is to change the app to an IP instead of a server name, so this is what my current config file looks like that I’ve copied from the _template.subdomain.conf and named mail.subdomain.conf.

server {
    listen 443 ssl;
    listen [::]:443 ssl;

    server_name mail.*;

    include /config/nginx/ssl.conf;

    client_max_body_size 0;

    location / {
        include /config/nginx/proxy.conf;
        include /config/nginx/resolver.conf;
        set $upstream_app 192.168.1.210;
        set $upstream_port 443;
        set $upstream_proto https;
        proxy_pass $upstream_proto://$upstream_app:$upstream_port;
    }

I haven’t added anything to the SWAG Docker settings other than the initial settings to add the subdomains for the Docker instances, and I’m not sure what or where I should change there (if anything) if I don’t want SWAG to manage the letsencrypt certificates for the mail server.

Help, please :slight_smile:

Follow-up

OK, so I’ve managed to get slightly further with this, but it still doesn’t work properly. My VM has a capital M for Mail, so I’ve changed the server_name and .conf file name to Mail.* and can now get to domain DOT com or even box DOT domain DOT com, but if I try to visit any of the other pages on the website (domain DOT com/games.html for instance) I get the security error. The same goes for the webmail at box DOT domain DOT com/mail.

Additionally, Thunderbird keeps popping up with certificate errors asking me to add an exception.

I’m not sure what I’m doing wrong. Every place where I see people talk about this, they say all you have to change is the upstream_app to the IP address, and this clearly doesn’t seem to be the case.

Couple questions: can you reach https://192.168.1.210 and get whatever webpage you’re trying to reach on your mail server?
does your mail server also listen on http/80? if yes, it’s probably better to only have 1 layer of ssl rather than reverse proxying via ssl to another ssl

is thunderbird communicating to this mail server over http/https and not pop3/imap/smtp ?

Thanks for the reply @driz.

Without using SWAG, if I use the internal IP for the mail server it’s converted to the external address. With SWAG I get the certificate error.

Port 80 is open for MIAB (presumably to meet RFC specs) but redirects immediately to 443.

Thunderbird only uses IMAP & SMTP. Webmail is obviously on HTTPS.

MIAB is a complete mail server with webmail & DNS server and uses letsencrypt for the certificates, which is why I’m only looking for SWAG as a reverse proxy, and so that I can have a couple of docker containers internet facing. I can still get emails when I try redirecting through SWAG but I can’t get to the admin interface or anything but the main domain DOT com or box DOT domain DOT com, which obviously defeats the purpose.

what if you simply reverse proxy to the converted external address? I was thinking this was internally hosted, so im getting a bit confused. FWIW, i reverse proxy https://www.google.com without any issue, so it SHOULD work.

When you get the cert error, is it for the cert swag is providing or the cert miab is providing? are there more details you can provide on the cert error itself?

this may be better handled on our discord, fyi

OK, I’ve just been doing some more tinkering and realised Firefox has cached some information and the previous results aren’t right at all, so I’ve tested again on another browser.

If I redirect ports 80 & 443 to SWAG, I can get to my docker containers but, no matter what I put as the $upstream_app setting in the config file - whether it’s my internal or external IP, domain.com or box.domain.com, if I try to browse to my website, webmail or admin panel I’m redirected to the ‘Welcome to your SWAG instance’ page.

Router config for extra info:

External ports opened for the mail server and directed to that internal IP:

25, 53, 465, 587 & 993

External ports now pointed at SWAG using alternate internal ports configured in the docker container:

80, 443

As mentioned previously, I’m still receiving emails as all the relevant email ports are still pointed at the VM’s IP, but am intermittently getting certificate errors from Thunderbird. I haven’t noticed how often they’re popping up, and it’s certainly not on the arrival of every email. I can force the security exception popup to appear if I close/reopen Thunderbird, and it’s the SWAG certificate that it’s trying to authenticate with, which is obviously not correct as I’ve told swag to only get certificates for the two docker containers and set Only Subdomains as true.

[Edited to add]

Here’s my SWAG docker config:

Repository: linuxserver/swag

Network Type: Custom : customproxy

Fixed IP address (optional): blank

Console shell command: Shell

Privileged: Off

http: custom internal port

https: custom internal port

Email: my email address

Domain Name: domain DOT com

Subdomain(s): docker1,docker2,

Only Subdomains: true

Validation: http

DNS-Plugin: blank

AppData Config Path: /my/unraid/appdata/swag

Extra Domains: blank

Staging: blank

DuckDNS Token: blank

Propagation: blank

I think we need to see some more info then. If you’re seeing the swag instance page, it means it’s not setup to reverse proxy what you’re going to. You can also review your access and error logs (tail -f then try to access the site)

The compose you’ve linked doesnt really help since you obfuscated things like the ports. I would suggest visiting us on discord for further help, you’ll need to be ready to provide your compose (domain and email can be hidden, rest needs to show, the subdomain.conf file you’ve made, ping tests from swag to your mail system, docker logs, and nginx access/error logs.

Thanks.

I tried using tail -f at the Unraid terminal (both the main terminal and the SWAG one) but I’m guessing there’s more to it than that as it did nothing by itself, just sat there staring at me with a cursor no matter how many times I visited any of the dockerised instances.

My compose is listed at the start of this thread, copied from the _template.subdomain.conf and named Mail.subdomain.conf.

The obfuscated ports are, by and large, irrelevant as I know the dockers work when I point my router at them, it’s just that the VM stops working properly at that point. The blank sections are literally that - blank - I haven’t edited them and there’s nothing in them to start with. Discourse won’t let me add more than two entries containing a domain name as I’m a new user, not that I’d post my actual domain anyway :slight_smile:, and everything else is pretty much exactly as it above. As mentioned I used SpaceInvader One’s video so the settings he talked about is what I’m using.

As for Discord, I’ve already signed up to two sites to get an answer and, without wishing to sound ungrateful for the free help given, have no interest in signing up to a third. If I can’t get the answer here or on the Unraid forum I’ll just give it up as a failed experiment. I can already access the dockers I’m playing with over my internal VPN but thought it might be nice if I could get to them more easily or give access to a couple of friends without giving them access to my internal network.

specifically, you need to tail -f the 2 files i mentioned, access.log and error.log which are in your /path/to/swag/log/nginx/ folder. if you see no data in those files when you access things, then the traffic is not going through swag (or maybe you have multiple swag instances running?)

The proxy-config at the top is not compose, the swag thing you linked with everything obfuscated is and is too redacted to help, whether you think it matters or not, you have a problem and we need info to assist.

I have no clue who this person is, but if you’re following their guide it may be best to seek help from them? We’re here to help but you’re following some random person’s guidance, asking us to help you fix it, and then refusing to provide full info or join us on our preferred support platform…

While i would hate for you to give it up as a failed experiment, we’ve reached the end of what support can be provided based on what you’re willing to do.

As an example, when I try to visit the mail server admin page, the access.log shows:

MYEXTERNALIP - - [30/Jun/2021:21:34:58 +0100] “GET /admin HTTP/2.0” 200 583 “-” “Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36”
MYEXTERNALIP - - [30/Jun/2021:21:34:58 +0100] “GET /favicon.ico HTTP/2.0” 200 583 “https MYDOMAIN DOT COM/admin” “Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36”

Nothing new is happening in the error log, but there’s a couple of certificate errors from 3 days ago.

2021/06/27 10:34:02 [crit] 500#500: connect() to [2a02:26f0:e8::6856:6eea]:80 failed (99: Address not available) while requesting certificate status, responder: r3.o.lencr.org, peer: [2a02:26f0:e8::6856:6eea]:80, certificate: “/config/keys/letsencrypt/fullchain.pem”
2021/06/27 11:19:32 [crit] 501#501: *31 SSL_read_early_data() failed (SSL: error:141CF06C:SSL routines:tls_parse_ctos_key_share:bad key share) while SSL handshaking, client: 107.178.236.23, server: 0.0.0.0:443
2021/06/27 11:19:33 [crit] 500#500: *36 SSL_read_early_data() failed (SSL: error:141CF06C:SSL routines:tls_parse_ctos_key_share:bad key share) while SSL handshaking, client: 107.178.232.189, server: 0.0.0.0:443
2021/06/27 11:23:58 [crit] 500#500: *54 SSL_read_early_data() failed (SSL: error:141CF06C:SSL routines:tls_parse_ctos_key_share:bad key share) while SSL handshaking, client: 82.221.105.6, server: 0.0.0.0:443
2021/06/27 12:07:06 [crit] 500#500: connect() to [2a02:26f0:e8::6856:6eea]:80 failed (99: Address not available) while requesting certificate status, responder: r3 DOT o DOT lencr DOT org, peer: [2a02:26f0:e8::6856:6eea]:80, certificate: “/config/keys/letsencrypt/fullchain.pem”

Presumably those IP addresses all belong to letsencrypt as I don’t recognise them. I only have one instance of SWAG running.

Docker’s fairly new to me so I’ve obviously misread other posts and got the config and compose confused there though, as mentioned, there’s very little that I’ve obfuscated that can be of any use. Apart from the internal ports I’ve mentioned, the only things that are important are my email address and domain name, neither of which I have any plans on sharing. The subdomains are just the two subdomains for the dockers, both of which have valid A records and both of which work when I point my router at the Unraid server with SWAG running.

Sorry, SpaceInvader One is presumably more known on the Unraid forum as he seems to post a lot of ‘how to’s’ on YouTube. This is the tutorial I’m referencing for setting up a reverse proxy on Unraid - https://youtu.be/I0lhZc25Sro