To answer my own question, first, there’s an official docker image for mumble-server, also known as Murmur.
https://hub.docker.com/r/mumblevoip/mumble-server
From the docs on the GitHub site, you can supply most configuration details as environment variables, including the TLS certificates to use.
https://wiki.mumble.info/wiki/Murmur.ini#sslCert_and_sslKey
And LinuxServer’s docs explain setting up the directory with certs from SWAG as a volume attached to another service.
So the work I was requesting is basically already done. (The official Murmur image isn’t built for ARM CPUs currently, just for amd64, but they should be available again soon.)