502 through nginx proxy manager after reboot
Running on unRAID using the docker compose method. I updated the stack today and now i cannot access immich through my reverse proxy, i get a 502 now. If i use ip:port it works fine. The proxy worked fine an hour ago before updating.
33 Replies
If ip:port works fine, then I'd expect the issue is in either your proxy config or the networking between it and immich
its strange, i had no advanced configuration in nginx proxy manager before and it worked without issue. While waiting for a response here i did some digging and came across this post on reddit which said to dumb this into the advanced section but that didnt help either, tried both immich-server:3001 and the ip:port
location /api {
# Compression
gzip_static on;
gzip_min_length 1000;
gzip_comp_level 2;
proxy_buffering off;
proxy_buffer_size 16k;
proxy_busy_buffers_size 24k;
proxy_buffers 64 4k;
proxy_force_ranges on;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
rewrite /api/(.*) /$1 break;
proxy_pass http://immich-server:3001;
}
Are you using the immich_proxy container?
You don't need that extra config from reddit then. Just point a bog standard reverse proxy at immich_proxy and it should work
yeah, thats what i had and after updating the stack today i get the 502, removed the additional config, restarted npm and immich and still the same
What's the reverse proxy target you are using?
Updating the stack will restart it and your target might be invalid now.
Using the ip:port of the immich_proxy from the screenshot above
Can you just tell me the port?
2283
Is the proxy container on the same network as your reverse proxy or a different one?
Like, is it the same docker network or a different one? How do you have the networking configured so they can communicate?
I run npm in it's own network, and I've added immich-proxy to that network. I then use
immich-proxy:8080 as the target. If they are on the same network you can use the service name and you need to use the internal port (8080) not the external port (2283).
If they are not on the same network and you're using port 2283 then you can't use the docker service name and so you have to use ip and it's possible the ip has changed.not super familiar with how docker networks function on unraid. NPM is on the br0 network so i can assign an ip address to it, immich shows its on immich_default but is running on the host ip
You should be able to create another network called
proxy (or whatever else you like) and add both containers to that as wellAre you running the reverse proxy on the same machine that's running immich? It sounds like that is what you are saying.
yeah, everything is running in containers on unraid
my question is that nothing has changed network wise in months, but after updating the stack yesterday its doing this i guess
You had host ip:2283 as the target? Has the host ip changed recently?
nope, no changes to any ips or ports
Since ip:port still works it's not an issue in immich, which is why we're trying to troubleshoot your network/proxy
Right, so nothing an immich release could be causing this problem as it looks to be purely a networking issue.
More than likely you have something misconfigured and it didn't become an issue until you restarted immich during the update.
What version of immich were you on before? The only thing we've changed recently has been how forwarded http headers are used, which shouldn't effect reachability.
i want to say 1.52.1
you guys might not know the answer to this but is there a way to change immich to run on a different docker network vs me moving my proxy to a different network to see if that fixes things?
looking through the logs on NPM i think you guys are on the right path with the network stuff, granted, im not sure what im reading here but i see this before i updated:
882#882: *138658 an upstream response is buffered to a temporary file /var/cache/nginx/proxy_temp/1/79/0000012791 while reading upstream, client: 192.168.0.110, server: immich.miakbox.com, request: "GET /api/asset HTTP/1.1", upstream: "http://192.168.0.50:2283/api/asset", host: "immich.miakbox.com"
then after updating i see tons of these:
[error] 6208#6208: *174125 connect() failed (113: No route to host) while connecting to upstream, client: 69.167.12.35, server: immich.miakbox.com, request: "HEAD /home HTTP/1.1", upstream: "http://192.168.0.50:2283/home", host: "immich.miakbox.com"
You don't need to move anything off its current network, a container can be attached to multiple networks
i attached NPM to the same network, still getting the 502
And you're using immich_proxy:8080 as the address?
no, npm is using the ip:port of the immich_proxy - ip:2283 - http://immich_proxy:8080 does not resolve on my network
It should resolve inside the docker network
Thats the idea behind attaching them together like that
sorry, im not sure i understand how to test it inside the docker network :/
Just set it as the upstream for NPM to test
interesting, that works
now the real question is, there is no way to force a container in unraid to connect to two networks, its apaprently a manual cli process each time the container restarts
if i take it off of the immich_default network i assume that it wont resolve then, correct?
Correct
You should be able to config multiple networks in the docker-compose.yml
unraid doesnt use docker compose normally unfortunately
unrelated question, i know you guys dont support the unraid community version that is not using docker compose, but if i were to switch to that i would likely have to redo everythign correct?
If you got (and restored) a backup of the database & library folder then it should theoretically be a seamless switch, but no guarantees
i assume there probably isnt an easy way to backup/restore the db is there
Take a look at https://github.com/immich-app/immich/discussions/1101
GitHub
Server Backup/Restore · immich-app immich · Discussion #1101
It would be nice to be able to backup the server with the database and being able to restore it incase of a disk crash or server migration.