Immich Server Microservices Error/Restarting Caused by Non-existent psql?
Machine Setup: Proxmox Machine with a debian lxc running docker as root. (4 cores, 6gb ram, 30gb SSD for the LXC)
Immich Version: V1.134.0
Installing using the docker-compose files from https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml and corresponding .env file
ISSUE: I'm trying to create a new immich install but am continuously running into an issue with the immich_server container constantly restarting, and the immich_postgres container never leaving the starting state.
The compose and env file successfully create the library and postgres directories when run, but the postgres directory stays empty, with no logs folder showing.
Seeing a similar previous issue with the microservice not connecting in docker compose logs, I tried entering the immich_postgres container and trying to interact with the database results in a network error.
I have tried multiple different isntallations with new LXCs, different OSs, using the custom media location, without using the custom media location, changing the postgres docker image to match a previous post with a similar issue, but nothing seems to fix the issue of the database not being created.
Previous install on version 1.121.0 was working until update in mid march
I wanted to ask if there was any insight on what I might be doing wrong.
Thank you so much!
59 Replies
:wave: Hey @Nallaka,
Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.
References
- Container Logs:
docker compose logs docs
- Container Status: docker ps -a docs
- Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
- Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA
Checklist
I have...
1. :ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time).
2. :ballot_box_with_check: read applicable release notes.
3. :ballot_box_with_check: reviewed the FAQs for known issues.
4. :ballot_box_with_check: reviewed Github for known issues.
5. :ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy).
6. :ballot_box_with_check: uploaded the relevant information (see below).
7. :ballot_box_with_check: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable
(an item can be marked as "complete" by reacting with the appropriate number)
Information
In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:
- Your docker-compose.yml and .env files.
- Logs from all the containers and their status (see above).
- All the troubleshooting steps you've tried so far.
- Any recent changes you've made to Immich or your system.
- Details about your system (both software/OS and hardware).
- Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h).
- The version of the Immich server, mobile app, and other relevant pieces.
- Any other information that you think might be relevant.
Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)
If this ticket can be closed you can use the /close command, and re-open it later if needed.just so it's been said, docker in an LXC is not supported
Was there a recent change that broke that functionality
I had a previous instance running on v1.12something until ~march
it's docker that states that it's not supported
I can quickly retry a VM
it's possible that it can work, but there are quirks that come with it - some that are hard to troubleshoot
Would it be ok to keep the post up while I retry a vm
absolutely!
back to your original question however, it seems that your database isn't completing startup. I'd check the logs of that container to see what's going on
only container logs that show for immich_postgres are:
is there a deeper inspect than just docker compose logs
were you upgrading versions when this became an issue?
I was and after three days I gave up and just tried installing fresh, still wasn't working
what version were you upgrading from?
i'm not trying to reconnect any of my old database or files, just trying to figure out why a fresh install isn't running
1.121.0 I believe
oh, so this is a new installation?
Yes this current post and all the logs are from a brand new installation on a new lxc
only thing that isn't new is the server it's on XD
gotcha. Would be interesting to know if the vm solves the issue
yeah working on that now
database logs can be found in
postgres/log
empty
I tried even making the log folder and just chmod 777 the entire immich-app directory incase it was perms
that log folder only exists because I made it manually, before /postgres would get made by the container but not populated with anything
I'm not acutally sure if the latest image logs there - my last entry was on the 24th
/postgres should definetly be getting populated
yup it's just the lxc
I guess i'm cursed?
the VM works, i'd just prefer to bind mount my existing images like I did before.
I know there are others running an lxc as well, I'm not sure what everyone's work arounds are though
is it poor practice to NFS my library, I don't beleive I can assign part of my zfs pool to the vm without assigning all of it
With the vchord migration and our postgres image we're logging to stdout again
So should be in the regular
docker compose logsLibrary is fine, however you'd want to keep the database local. Thumbs might also serve faster from local storage
Wait that's actually something I never tried. I never remade the directory I was using in my zfs pool
ok, so then we're not getting anything in the logs
but that doesn't explain why it wouldn't work when I did not bind it to there
this might be a complete shot in the dark. Would my router firewall settings at all affect the ability to connect or bind to the postgres container. That's literally the only thing that changed from when it was working to not working post upgrade.
To clarify, it was working v1.121.0 -> I changed my router -> I upgraded to v1.130.0 at that time -> stopped working
I assume the router is physical hardware that is not part of the server running immich?
yes it's seperate
then it shouldn't make a difference. The routing is done on the server between the docker network (at least in the default compose file)
How did you install docker on the LXC btw @Nallaka ?
couple of different ways:
none of those worked
only thing that's worked so far is a fresh debian vm. I'm still messing around. I haven't tried a non-debian lxc
curious
I'm running multiple docker lxc installed with helper scripts and have no issues
yeah this is the only issue I've ever had with the helper scripts
I meant for immich too
i'm going through and updating everything and i'm gonna try an ubuntu lxc
Here's my container.conf:
yeah before I had a docker lxc I got from helper scripts running my install
was working for like 8 months i think
Is your storage local or NFS or ... ?
local
Really strange then
I also tried that new immich script that got uploaded literally last night to proxmox helper scripts
I only know of issues with directories that have ACLs
yeah that's what I thought at some point so I tried just 777 everything the containers would touch and still no dice
i've tried both with binding and not binding my old storage
I wish I knew more about actually debugging the lxc, that's a bit past my current knowledge
You don't happen to run any other containers on the same LXC?
none, complete fresh
I even tried with and without portainer and the agent
I've been at this for a couple days lol
it's going to be one of those weird quirk things that we warn everyone about 🙃
what's your proxmox version. i just realized mine was stuck on 8.1.4 so I'm trying updating that
8.4.1
Literally update day today for me 😄
i'll see if that fixes much
haha
So all containers are latest
docker too
yeah it should be
i'm updating the proxmox host itself rn and I'll try recreating a new lxc
thanks for sticking around btw. Ik we technically got a solution with the vm, but now I'm determined to get this working 😠
oh my fricking god
that was it
something about the old pve-container breaks postgres
😭
3 days and all I needed to do was update my host
so, some shared host and lxc resource on proxmox <=8.1.2 doesn't work with the new postgres library
since it worked on a vm
i have no clue what it is, but everything should be good now 🙏🏿
Well either it was that or updating/rebooting fixed something that was broken