Wireguard(vpn), docker swarm and script to update routes of other containers with health check

General
1

Hey All,

As you’re aware routing can be difficult if you setup a Wireguard VPN and want to route other containers through it, the 2 guides below are awesome until you have docker swarm (at least for me it was!)

The challenge here is I struggled to get swarm to set a static IP for the container(my research and tests showed it was’t possible and potential solutions didn’t work for me) - so I let docker give whatever it wants, and have startup scripts to manage it and a health check that runs every 15 mins to see if the file has been updated - if it has update the routes.

My /custom-cont-init.d/ is common and shared to all containers
The wireguard container has an additional ENV var set of

USAGE=ROUTER

You’ll see how this is used in the startup. Hopefully it helps out someone or gives them some ideas if they run into the same problems as me.

There is some setup required as well, make sure to read these 2 guides

This one is the one I modified

I like that if wireguard container IP changes, then traffic just doesn’t work to the internet - meaning no “leakage”

DNS for me is using my internal resolvers (pihole) - but if you want no DNS leakage as well make sure you set them to static upstream - or setup a pihole DNS server within the stack to manage this if you want more control

#!/bin/bash
# This will setup the routing correctly on all containers and do a check 15m to check correct routes are in place
# This file needs to be put into /custom-cont-init.d/
#set -x
CONTAINER_NETWORK=`ip r | grep default | awk '{print $3}'`
ROUTES_TO_ADD=("192.168.1.0/24" "192.168.2.0/24")  # These are my private networks I still wanna be able to route to - for accessing containers in the vpn stack.
ROUTER_IP_FILE=/custom-cont-init.d/WG_IP.txt
HEALTH_CHECK_FILE=/etc/periodic/15min/health_check.sh

# This checks if the ROUTER_IP_FILE's timestamp has changed - if it has it updates routes
check_timestamp() {
    file_to_check=${ROUTER_IP_FILE}
    timestamp_file=/tmp/last_update

    if [ -f "${timestamp_file}" ]; then
        previous_timestamp=$(cat "${timestamp_file}")
    else
        previous_timestamp=0
    fi

    current_timestamp=$(stat -c "%Y" "${file_to_check}")

    if [ "${current_timestamp}" != "${previous_timestamp}" ]; then
        echo "The timestamp of ${file_to_check} has changed from ${previous_timestamp} to ${current_timestamp}"
        update_routes  # Update the routes if it has changed
        echo "${current_timestamp}" > "${timestamp_file}"
    fi


}

update_routes(){
        source /custom-cont-init.d/WG_IP.txt
        ip route del default
        ip route add default via ${ROUTER_IP}
        for route in "${ROUTES_TO_ADD[@]}"; do
                if ! ip route | grep -q "$route"; then
                    # Add the route
                    ip route add $route via ${CONTAINER_NETWORK}
                    echo "Route added: $route"
                fi
        done
}

create_health_check(){
    if [ ! -f "${HEALTH_CHECK_FILE}" ]; then
        # Create the health check file
        cat > ${HEALTH_CHECK_FILE} << EOF
#!/bin/bash
source /custom-cont-init.d/routes.sh
EOF
        chmod u+x ${HEALTH_CHECK_FILE}
        echo "Created file ${HEALTH_CHECK_FILE}"
    fi
}

startup(){
    if [[ "${USAGE}" == "ROUTER" ]]; then
            echo "export ROUTER_IP=`ip -4 -o addr show eth0 | awk '{print $4}' | cut -d/ -f1`" > /custom-cont-init.d/WG_IP.txt
    else
            while [ ! -e "${ROUTER_IP_FILE}" ]
            do
                echo "${ROUTER_IP_FILE} does not exist sleeping for 10s"
                sleep 10
            done
            check_timestamp
            create_health_check
    fi
}

startup
1 Like
2

Thanks for sharing this, none of us on the team use swarm, so we don’t test or support it, but info like this will be very helpful to those who do :slight_smile:

3

Happy to help - the linuxserver.io images have saved me so much time! They are great

A small update to the above, there was an edge case where on startup an old IP was present before wireguard updated it with the new VPN address, so it will ping an IP on first startup and keep looping until that works - Just a new function called “first_start”

#!/bin/bash
# This will setup the routing correctly on all containers and do a check 15m to check correct routes are in place
# This file needs to be put into /custom-cont-init.d/
#set -x
CONTAINER_NETWORK=`ip r | grep default | awk '{print $3}'`
ROUTES_TO_ADD=("192.168.1.0/24" "192.168.2.0/24")  # These are my private networks I still wanna be able to route to - for accessing containers in the vpn stack.
ROUTER_IP_FILE=/custom-cont-init.d/WG_IP.txt
HEALTH_CHECK_FILE=/etc/periodic/15min/health_check.sh
HOST_TO_PING=8.8.8.8

# This checks if the ROUTER_IP_FILE's timestamp has changed - if it has it updates routes
check_timestamp() {
    file_to_check=${ROUTER_IP_FILE}
    timestamp_file=/tmp/last_update

    if [ -f "${timestamp_file}" ]; then
        previous_timestamp=$(cat "${timestamp_file}")
    else
        previous_timestamp=0
    fi

    current_timestamp=$(stat -c "%Y" "${file_to_check}")

    if [ "${current_timestamp}" != "${previous_timestamp}" ]; then
        echo "The timestamp of ${file_to_check} has changed from ${previous_timestamp} to ${current_timestamp}"
        update_routes  # Update the routes if it has changed
        echo "${current_timestamp}" > "${timestamp_file}"
    fi

}

update_routes(){
        source /custom-cont-init.d/WG_IP.txt
        ip route del default
        ip route add default via ${ROUTER_IP}
        for route in "${ROUTES_TO_ADD[@]}"; do
                if ! ip route | grep -q "$route"; then
                    # Add the route
                    ip route add $route via ${CONTAINER_NETWORK}
                    echo "Route added: $route"
                fi
        done
}

create_health_check(){
    if [ ! -f "${HEALTH_CHECK_FILE}" ]; then
        # Create the health check file
        cat > ${HEALTH_CHECK_FILE} << EOF
#!/bin/bash
source /custom-cont-init.d/routes.sh
EOF
        chmod u+x ${HEALTH_CHECK_FILE}
        echo "Created file ${HEALTH_CHECK_FILE}"
    fi
}

startup(){
    if [[ "${USAGE}" == "ROUTER" ]]; then
            echo "export ROUTER_IP=`ip -4 -o addr show eth0 | awk '{print $4}' | cut -d/ -f1`" > /custom-cont-init.d/WG_IP.txt
    else
            while [ ! -e "${ROUTER_IP_FILE}" ]
            do
                echo "${ROUTER_IP_FILE} does not exist sleeping for 10s"
                sleep 10
            done
            check_timestamp
            first_start
            create_health_check
    fi
}

# Sometimes the $ROUTER_IP_FILE exists, and its got an older IP in it.. so on first start lets do some more checks
first_start(){
    if [ ! -f "/tmp/first_start" ]; then
        echo "This is a first start, lets check we can ping upstream if not.. setup the routes again"
        if ping -c 1 "${HOST_TO_PING}" &> /dev/null; then
            echo "You are online - Congrats!"
            touch "/tmp/first_start"
        else
            echo "You are offline"
            rm /tmp/last_update # Remove the last update to force the next one
            sleep 10
            update_routes
            first_start
        fi
    fi
}

startup