Piwigo: Frequent Errors while Uploading Images: upstream timed out (110: Operation timed out) while reading response header from upstream, client

Hello! I’m so thankful for your creation of the piwigo container. It’s MOSTLY working for me however when uploading multiple images, I am almost always running into “upstream timed out (110: Operation timed out) while reading response header from upstream, client: ip-here” errors. Everything else appears to work fantastic so I’m just trying to knock this issue out.

The described install and configuration was performed the same for both of the below environments. In both cases, the same core issue remains.

RESOURCES SUMMARY

Setup #1: Test / Staging in a local virtual machine to emulate installing before moving onto Setup #2

  • Virtual Machine:
  • Host: Windows 10 pro (Intel CoreX 8(16) core CPU | 128GB RAM | Pile of fast SSDs)
  • Virtual Machine Software: VMWare Workstation Pro 16.1.1
  • Guest: Alpine Linux 3.13.4 (two CPUS/cores | 4GB RAM | 80GB Disk)

Setup #2: Deployment to a Linode VPS

  • VPS: Alpine Linux 3.13.4 (two CPUS/cores | 4GB RAM | 80GB Disk)

INSTALLATION AND CONFIGURATION

  • Images pulled:

  • docker [ . ] io/linuxserver/piwigo:latest

  • MariaDB (not yours): docker [ . ] io/mariadb/server:latest

  • Network creation for the above two eventual containers:
    podman network create mdb-pwg

  • Verification it was created:
    podman network ls
    NETWORK ID NAME VERSION PLUGINS
    514459879fa3 mdb-pwg 0.4.0 bridge,portmap,firewall,tuning,dnsname

  • Pod creation for the above two eventual containers:
    podman pod create --name mdb-pwg --network mdb-pwg -p 8080:80/tcp -p 8443:443/tcp

  • Verification it was created:

  • Note: “Exited” is due to encountering the timeouts and i stopped the entire pod
    podman pod ls
    POD ID NAME STATUS CREATED INFRA ID # OF CONTAINERS
    cda52a83ed9e mdb-pwg Exited 16 hours ago cba050938651 3

  • Create and start the mariadb container (not yours, just mentioning this for completion) inside of the created mdb-pwg pod

  • TL;DR: It works
    podman run -dt --name mariadb --pod mdb-pwg mariadb/server:latest

  • Piwigo container:

  • Make a directory for app data / config
    mkdir -pv ~myusername/piwigo-config

  • Deploy:
    podman run -dt --name piwigo --pod mdb-pwg -e PUID=myuid -e PGID=mygid -e TZ=UTC -v /home/myusername/piwigo-config:/config docker [ . ] io/linuxserver/piwigo:latest

  • I take care of the needed MariaDB tasks, start the containers, and I am able to visit https://[ip]:8443/ and everything goes as planned.

ISSUE DETAILS

  • When uploading more than one image the displayed web upload page generally gets “stuck” at one of the uploads

  • The more images involved, the more problematic the issue is

  • I have tried up to 13

  • Result: Varies between finishing without issue ONLY IF THE CONTAINER WAS JUST STARTED to POST requests that time out move onto the next image, which themselves may complete or may time out.

  • Inspecting in chromiums dev mode (F12) shows that these HTTP POST requests are timing out after 60 seconds

  • Inside the piwigo container errors such as these are observed:

==> /config/log/nginx/error.log <==
2021/04/10 17:26:23 [error] 329#329: *2 upstream timed out (110: Operation timed out) while reading response header from upstream, client: 10.88.2.2, server: _, request: “POST /ws.php?method=pwg.images.upload&format=json HTTP/1.1”, upstream: “fastcgi://127.0.0.1:9000”, host: “ip:8443”, referrer: “https://[ip]:8443/admin.php?page=photos_add&album=23”

  • The VPS is not running out of resources and in fact is barely affected. There are no errors observed in dmesg.

  • Alpine Linux 3.13.4 (two CPUS/cores | 4GB RAM | 80GB Disk)

  • No other errors have been observed other than the one above

SOLUTIONS ATTEMPTED

None of these actions changed the situation.

  • Different browsers

  • Chromium and Firefox, using both my standard profiles in each, as well as default profiles

  • Different client operating systems for each of the above

  • Linux Mint 20, Alpine Linux 3.13.4

  • Attempted to get more logging information from php / php-fpm however this has proven problematic

  • log_level = debug and also tried log_level debug as an apparently erroneous source mentioned

  • And then restarted the container

  • Result .ini file(s) being unable to load

  • pm.status_path = /fpmstatus

  • And then restarted the container

  • Result: 404 when trying to visit

  • These lines were removed when this did not improve the situation

  • And then restarted the container

  • Increased memory usage for PHP within the container
    cd /config/php && echo -e “\nmemory_limit = 512M\npost_max_size = 200M\nupload_max_filesize = 200M\n” >> php-local.ini

  • And then restarted the container

  • These lines were removed when this did not improve the situation

  • And then restarted the container

** This DID change the situation… temporarily **

  • Restart the pod containing mariadb and piwigo
  • However after the first upload of numerous images, the unwanted behavior returns

WHAT WORKS SO FAR

  • Restarting the pod containing mariadb and piwigo before uploading more than one image at a time

FINAL THOUGHTS

Thank you for reading all of this! I’m very excited to be able to use piwigo so easily and I know together we’ll resolve this. Please don’t take this as me trying to blame your work or questioning your quality. Something somewhere is causing this and I believe we can figure it out together.

Please let me know if there is something else I can do to provide more information.

1 Like