Broken Immich install with PGSQL exiting
Hello there,
I've been running into issues with my Immich install running on a Proxmox VM with Ubuntu 22.04 LTS in Docker.
It used to work as non-root but stopped working, complaining about missing pg_notify folder.
It seems the errors are related to permissions for the postgresql folder (mounted with bind mounts in a folder on another local drive), with missing permissions, even though I switched to running the PG container without UID/GID options to unblock the situation.
At first I was getting error in the docker logs about missing pg_notify folder. So I manually created said folder under the PG folder, set immich to run as root to avoid permission issues, but it's still not starting properly.
In the PG logs, I can now find errors, still related to missing folders. Everytime I create one, there's a new one popping up. Ex : pg_tblspc and pg_replslot then pg_twophase
Could you please help me understand what I'm doing wrong ? I feel like using bind mounts was a mistake and I should have used docker volumes :/
Thanks in advance for your help !
42 Replies
:wave: Hey @StanRex,
Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.
References
- Container Logs:
docker compose logs docs
- Container Status: docker ps -a docs
- Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
- Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA
Checklist
I have...
1. :blue_square: verified I'm on the latest release(note that mobile app releases may take some time).
2. :blue_square: read applicable release notes.
3. :blue_square: reviewed the FAQs for known issues.
4. :blue_square: reviewed Github for known issues.
5. :blue_square: tried accessing Immich via local ip (without a custom reverse proxy).
6. :blue_square: uploaded the relevant information (see below).
7. :blue_square: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable
(an item can be marked as "complete" by reacting with the appropriate number)
Information
In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:
- Your docker-compose.yml and .env files.
- Logs from all the containers and their status (see above).
- All the troubleshooting steps you've tried so far.
- Any recent changes you've made to Immich or your system.
- Details about your system (both software/OS and hardware).
- Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h).
- The version of the Immich server, mobile app, and other relevant pieces.
- Any other information that you think might be relevant.
Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)
If this ticket can be closed you can use the /close command, and re-open it later if needed.Ex :
What do you mean by a bind mount to a local drive? Local how
It's a bind mount pointing to a folder that is on a secondary drive, not the system drive, mounted using fstab
but it's not a network drive, it's still in the same computer
Can I see the fstab line in the VM?
sure : /dev/disk/by-uuid/84d18d66-fc9f-430f-a267-c0550c39f567 /mnt/data ext4 defaults,noatime,nodiratime 0 2
Its not clear to me why you would ever have permission issues to begin with
Do you have some weird script or something that’s changing disk permissions?
Or something that would delete empty folders? Any kind of space saver?
not that I know of, I used to run with different permissions though, with UID/GID set on all containers (and it's been running like this for month)s, but since I started debugging this, I removed all settings of UID/GID to run as root, and avoid permissions problems
I’m not sure if that’s a good thing in this case. Because I think Postgres might drop to a different user regardless
So by changing to root you’re really retuning to the default PG user - 999?
ah
so I should try switching back to forcing UID/GID ?
Based on some quick googling seems all these folders are often empty
So I suspect something is deleting empty folders on your system.
Probably, and you can chown -R the whole folder to the UID
How are you setting UID in docker?
' user: ${UID}:${GID}' line for each container
and UID and GID are set in a .env
let me try that
Should work unless UID is like a docker special variable or something
I’d try using a more unique variable name
770 for permissions ?
Shouldn’t have to change that, I would leave those as is
:/ still getting 2025-04-15 12:05:15.253 UTC [15] FATAL: could not open directory "pg_twophase": No such file or directory in PG logs and seeing the PG container exiting 1
I'll try creating it
Yeah if the folder is absent on disk the owner/group won’t fix that
It might be better to restore from a DB backup
You have to figure out why your filesystem is losing folders though
I'm not sure tbh, I've cleaned some docker volumes, but it shouldn't be affecting the bind mounts
or might have been power loss at unexpected times
I'll try following the db restore process, thanks !
I’d be surprised if power loss only affected unused empty folders
true
quick question, when restoring with
I'm assuming dbname needs to match my db name from my .env. Shouldn't I also set --password=<DB_PASSWORD> ?
No
Dbname doesn’t need to change either
ok !
thanks !
err, running line by line the restore script
at this point, I check the logs
and see
etc...
so I'm basically still getting permissions errors even though I wiped the DB folder :/, and have the DB container set to use the immich user
oh well, looks like the PG folder is created by root :/
even though I have user: ${UID}:${GID} set on all containers in my compose file
If you’re using a bind mount in docker with a non root user you usually need to crate the folder and chown it before first startup.
That’s just a docker thing. Maybe root would be easier for you. Non root is trickier
damn it, I had forgotten I had done that the first time I set it up
some progress, looks like PG is restored, thanks for that !
but now it's Immich machine learning's time to crash
with what looks like more permission issues
😬
would I have avoided all this pain if I had gone with docker volumes instead of bind mounts ?
Not the empty folders being gone, no. That’s just an issue with your system.
yes, but the permission issues ?
I think non root is the bigger issue for you Than the docker volumes
Non root may be easier in them but still has caveats
yes, I agree non-root isn't that fun, but I work in cybersecurity, and have great fears of running as root imprinted in my bones due to this ^^
Haha I mean that’s fair
Docker volumes probably would be easier but IMO this kind of stuff needs to be second nature if you run non root because you run into these issues often in my experience
All it saves you is the initial chown
By chance, would you happen to know how to fix this ? I've wiped the bind mounts for the ML containers, and replaced them with docker volumes; from scratch, I started running the stack with non-root uid/gid, but I'm still getting that error when starting the stack
for my sanity s sake I'm going to try running the immich-ml as root to see if it solves this
Looks like more permissions issues
Can’t create tmp folders etc
yup, but what I can't understand is that I'm mounting the volumes listed here : https://immich.app/docs/FAQ/#how-can-i-run-immich-as-a-non-root-user.
Extract from my compose :
I'm trying with no UID/GID defined for root
it's working with no UID/GID defined
yay for database backups I guess
I'd love to understand why immich-ml is not playing nice with defined UID/GID and those volumes config
What OS are you on
Ubuntu 22.04 LTS
This isn’t a snap docker install is it?
no, it's installed using apt
Sorry I think your system is just cursed lol
it's super weird, I only use this vm as a docker host, so it's not like I've messed with tons of stuff on it
At some point I'll set up some iac config so that I can tear it down and build it back up, and will see if I'm encountering the same issue on the new box
thx a lot for the help !