Nginx `upstream_app` works when app contains underscores, but not with dashes

Hello,

This one is super weird for me.

My usecase: I already have my main app running and accessible from the internet:

  • server name: duplicacy-utils.tbp.land
  • container name: telegram-bot

Server:

server {
    server_name duplicacy-utils.tbp.land;
  
    listen 443 ssl http2;
    listen [::]:443 ssl http2;

    include /config/nginx/ssl.conf;

    location / {
        include /config/nginx/proxy.conf;
        include /config/nginx/resolver.conf;

       # set $upstream_app telegram_bot;  # ✅ 
        set $upstream_app telegram-bot; # ❌ 
        set $upstream_port 13337;
        set $upstream_proto http;

        proxy_pass $upstream_proto://$upstream_app:$upstream_port;
    }
}

Docker compose of app:

version: "2.4"
services:
  telegram-bot:
    # container_name: telegram_bot  # ✅  
    container_name: telegram-bot  # ❌ 

    image: thebestpessimist/duplicacy-utils-telegram-bot:latest
    restart: unless-stopped
    mem_limit: 80m
    environment:
      WEBSERVER_ADDRESS:
      WEBHOOK_ROUTE:
      TELEGRAM_API_TOKEN:
      CERTIFICATE_PATH:

networks:
  default:
    external: true
    name: nginx-common

Now i want to add a test env. This should be a clone of the existing conf, the only different things are:

  • server name: d.tbp.land
  • container name: telegram-bot-test
  • the env variables of the container, fed via an .env properties file

Server of test:

server {
    server_name d.tbp.land;
  
    listen 443 ssl http2;
    listen [::]:443 ssl http2;

    include /config/nginx/ssl.conf;

    location / {
        include /config/nginx/proxy.conf;
        include /config/nginx/resolver.conf;

        #set $upstream_app telegram_bot_test; # ✅ 
        set $upstream_app telegram-bot-test; # ❌
        set $upstream_port 13337;
        set $upstream_proto http;

        proxy_pass $upstream_proto://$upstream_app:$upstream_port;
    }
}

Docker compose of test app:

version: "2.4"
services:
  telegram-bot:
    # container_name: telegram_bot_test # ✅ 
    container_name: telegram-bot-test # ❌ 
    image: thebestpessimist/duplicacy-utils-telegram-bot:latest
    restart: unless-stopped
    mem_limit: 80m
    environment:
      WEBSERVER_ADDRESS:
      WEBHOOK_ROUTE:
      TELEGRAM_API_TOKEN:
      CERTIFICATE_PATH:

networks:
  default:
    external: true
    name: nginx-common

The problem is the following: using dashes in hostname does not work (see the pairs of “:x:” and “:white_check_mark:” above) .

When accessing the apps from the internet (my laptop), the first request passes successfully to each of the apps, however starting with the second request, most of the times (but not 100%) the app telegram-bot-test receives all the requests (so telegram-bot receives none).

I have tried and tried again, and this always happens .

Now the quirk: if i rename the 2 apps (and update the server configs) to use underscores (_) instead of dashes (-), everything is OK (see the pairs of “:x:” and “:white_check_mark:” above) .

I have checked both nginx’s error.log and access.log and things are as expected in both cases: entries are added in access.log successfully for both apps in the “working” usecase and the “not working” usecase.

I have also tried pinging the 2 apps at the same time. the pings are ok:

In the screenshot u can see “test” is on the right and “main” is on the left

Here’s a visual representation of all the edited resources:

Another thing which i tried is to create other apps like t-t and t-t-t, and these seem to not have issues. Or at least i could not reproduce them in the testing i did.

This leads me to think that something else might be wrong. Could it be something with nginx and the proxy_pass clause?

Do you have clues as to why hostnames with _ work, but with - have issues? Nothing else is changes except that.

Underscore is not supported as part of dns host names (at least with nginx)

but underscore fixes my issue, while dash does not.

After more reading (including but not limited to dns - Can (domain name) subdomains have an underscore "_" in it? - Stack Overflow)

I understand that you’re saying i should not use

set $upstream_app telegram_bot_test;

but instead i should use

set $upstream_app telegram-bot-test;

(dashes, and not underscores).

I have checked the hostname of my app. there are no dashes or underscores:

root@racknerd-c53fac:/var/zz/duplicacy-utils-telegram-bot-kt-test# docker exec telegram_bot_test hostname
d0ac4e49980b
root@racknerd-c53fac:/var/zz/duplicacy-utils-telegram-bot-kt-test# docker exec telegram_bot_test env
PATH=/opt/openjdk-15/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=d0ac4e49980b
WEBSERVER_ADDRESS=xxx
WEBHOOK_ROUTE=xxx
TELEGRAM_API_TOKEN=xxx
CERTIFICATE_PATH=xxx
JAVA_HOME=/opt/openjdk-15
JAVA_VERSION=15-ea+31
HOME=/root
root@racknerd-c53fac:/var/zz/duplicacy-utils-telegram-bot-kt-test#

So i believe the docker resolver deals with this somehow.

However my question still stands, what’s going on with the _ vs -, and who is breaking my stuff?

I come with more updates:
nginx does proxy pass to the correct $proxy_host, however the IP of that is wrong ($upstream_addr) (i checked the logs):

image

91.108.6.103 - - [14/Nov/2021:17:55:00 +0000] "POST https://duplicacy-utils.tbp.land/telegram/TELEGRAM_UPDATE_WEBHOOK_ROUTE HTTP/2.0" 200 0 "-" "-" 0.248 PROXY: "telegram-bot:13337 172.21.0.4:13337"
91.108.6.103 - - [14/Nov/2021:17:55:08 +0000] "POST https://duplicacy-utils.tbp.land/telegram/TELEGRAM_UPDATE_WEBHOOK_ROUTE HTTP/2.0" 200 0 "-" "-" 0.255 PROXY: "telegram-bot:13337 172.21.0.4:13337"
91.108.6.103 - - [14/Nov/2021:17:55:12 +0000] "POST https://duplicacy-utils.tbp.land/telegram/TELEGRAM_UPDATE_WEBHOOK_ROUTE HTTP/2.0" 200 0 "-" "-" 0.181 PROXY: "telegram-bot:13337 172.21.0.3:13337" # this one has the wrong IP for the correct host

Looking at the pings (from swag to both of the containers), sometimes those return a different value as well:
image

Ping commands:

docker exec swag ping telegram-bot
docker exec swag ping telegram-bot-test

this looks to me like a docker bug, so i opened an issue on github: Same IP address for 2 containers from resolver · Issue #43016 · moby/moby · GitHub

With docker compose, containers are accessible via container name as well as service name. When you set one of those with dashes, and one of those with underscores, compose is trying to make the container reachable by both.

Please excuse my stupidity. I read service name, but always thought of the container name. I apologize for wasting your time, albeit i learned something new today :).

I can confirm this works:

Service 1: telegram-bot

services:
  telegram-bot:
    container_name: telegram-bot
    image: thebestpessimist/duplicacy-utils-telegram-bot:latest

networks:
  default:
    external: true
    name: nginx-common

Service 2: telegram-bot-test

version: "2.4"
services:
  telegram-bot-test: # this is the only change: old: telegram-bot, new: telegram-bot-test
    container_name: telegram-bot-test
    image: thebestpessimist/duplicacy-utils-telegram-bot:latest

networks:
  default:
    external: true
    name: nginx-common

Thank you for all your time, and for keeping up with a dummy.

No worries, glad to hear it works

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.