I
Immich•2w ago
nagyviktor

"Failed to write upload/encoded-video/.immich" - permissions seem to be ok

I'm trying to migrate from a systemctl + external mount setup to a Kubernetes + NFS setup. Immich fails to start at read/write/create check of the upload dir. Based on the following, I think permissions should be ok.
# The external mount
nagyv@server-2:~$ ls -lna /mnt/raid1/immich/
total 32
drwxr-xr-x 8 1000 1000 4096 Jan 21 2025 .
drwxr-xr-x 10 0 0 4096 Sep 21 22:44 ..
drwxr-xr-x 2 1000 1000 4096 Sep 20 02:00 backups
drwxr-xr-x 3 1000 1000 4096 Jan 24 2025 encoded-video
drwxr-xr-x 2 1000 1000 4096 Jan 21 2025 library
drwxr-xr-x 2 1000 1000 4096 Jan 21 2025 profile
drwxr-xr-x 3 1000 1000 4096 Jan 21 2025 thumbs
drwxr-xr-x 3 1000 1000 4096 Jan 21 2025 upload
# The same NFS mount
nagyv@server-2:~$ kubectl exec -n immich immich-server-75cd9c4954-98h27 -- ls -lna /usr/src/app/upload
total 32
drwxr-xr-x 8 1000 1000 4096 Jan 21 2025 .
drwxr-xr-x 1 0 0 4096 Sep 23 22:20 ..
drwxr-xr-x 2 1000 1000 4096 Sep 20 02:00 backups
drwxr-xr-x 3 1000 1000 4096 Jan 24 2025 encoded-video
drwxr-xr-x 2 1000 1000 4096 Jan 21 2025 library
drwxr-xr-x 2 1000 1000 4096 Jan 21 2025 profile
drwxr-xr-x 3 1000 1000 4096 Jan 21 2025 thumbs
drwxr-xr-x 3 1000 1000 4096 Jan 21 2025 upload
# The user running the container
nagyv@server-2:~$ kubectl exec -n immich immich-server-75cd9c4954-98h27 -- id -u
1000
# The external mount
nagyv@server-2:~$ ls -lna /mnt/raid1/immich/
total 32
drwxr-xr-x 8 1000 1000 4096 Jan 21 2025 .
drwxr-xr-x 10 0 0 4096 Sep 21 22:44 ..
drwxr-xr-x 2 1000 1000 4096 Sep 20 02:00 backups
drwxr-xr-x 3 1000 1000 4096 Jan 24 2025 encoded-video
drwxr-xr-x 2 1000 1000 4096 Jan 21 2025 library
drwxr-xr-x 2 1000 1000 4096 Jan 21 2025 profile
drwxr-xr-x 3 1000 1000 4096 Jan 21 2025 thumbs
drwxr-xr-x 3 1000 1000 4096 Jan 21 2025 upload
# The same NFS mount
nagyv@server-2:~$ kubectl exec -n immich immich-server-75cd9c4954-98h27 -- ls -lna /usr/src/app/upload
total 32
drwxr-xr-x 8 1000 1000 4096 Jan 21 2025 .
drwxr-xr-x 1 0 0 4096 Sep 23 22:20 ..
drwxr-xr-x 2 1000 1000 4096 Sep 20 02:00 backups
drwxr-xr-x 3 1000 1000 4096 Jan 24 2025 encoded-video
drwxr-xr-x 2 1000 1000 4096 Jan 21 2025 library
drwxr-xr-x 2 1000 1000 4096 Jan 21 2025 profile
drwxr-xr-x 3 1000 1000 4096 Jan 21 2025 thumbs
drwxr-xr-x 3 1000 1000 4096 Jan 21 2025 upload
# The user running the container
nagyv@server-2:~$ kubectl exec -n immich immich-server-75cd9c4954-98h27 -- id -u
1000
and the above setup still fails with
immich-server-6c7f9799fd-ddhjc [Nest] 23 - 09/23/2025, 10:45:22 PM LOG [Api:StorageService] Verifying system mount folder checks, current state: {"mountChecks":{"thumbs":true,"upload":true,"backups":true,"library":true,"profile":true,"encoded-video":true}} immich-server-6c7f9799fd-ddhjc [Nest] 23 - 09/23/2025, 10:45:22 PM ERROR [Api:StorageService] Failed to write/usr/src/app/upload/encoded-video/.immich: Error: EACCES: permission denied, open '/usr/src/app/upload/encoded-video/.immich'
immich-server-6c7f9799fd-ddhjc api worker exited with code 1
immich-server-6c7f9799fd-ddhjc [Nest] 23 - 09/23/2025, 10:45:22 PM LOG [Api:StorageService] Verifying system mount folder checks, current state: {"mountChecks":{"thumbs":true,"upload":true,"backups":true,"library":true,"profile":true,"encoded-video":true}} immich-server-6c7f9799fd-ddhjc [Nest] 23 - 09/23/2025, 10:45:22 PM ERROR [Api:StorageService] Failed to write/usr/src/app/upload/encoded-video/.immich: Error: EACCES: permission denied, open '/usr/src/app/upload/encoded-video/.immich'
immich-server-6c7f9799fd-ddhjc api worker exited with code 1
I did not copy the uid:gid for /usr/src/app/upload/encoded-video/.immich, but it's 1000:1000 too. I already checked the linked docs, but could not figure out my issue. Does anyone have any pointers?
4 Replies
Immich
Immich•2w ago
:wave: Hey @nagyviktor, Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:. References - Container Logs: docker compose logs docs - Container Status: docker ps -a docs - Reverse Proxy: https://immich.app/docs/administration/reverse-proxy - Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA Checklist I have... 1. :ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time). 2. :ballot_box_with_check: read applicable release notes. 3. :ballot_box_with_check: reviewed the FAQs for known issues. 4. :ballot_box_with_check: reviewed Github for known issues. 5. :ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy). 6. :ballot_box_with_check: uploaded the relevant information (see below). 7. :ballot_box_with_check: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable (an item can be marked as "complete" by reacting with the appropriate number) Information In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider: - Your docker-compose.yml and .env files. - Logs from all the containers and their status (see above). - All the troubleshooting steps you've tried so far. - Any recent changes you've made to Immich or your system. - Details about your system (both software/OS and hardware). - Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h). - The version of the Immich server, mobile app, and other relevant pieces. - Any other information that you think might be relevant. Please paste files and logs with proper code formatting, and especially avoid blurry screenshots. Without the right information we can't work out what the problem is. Help us help you ;) If this ticket can be closed you can use the /close command, and re-open it later if needed. Successfully submitted, a tag has been added to inform contributors. :white_check_mark:
Sergey Katsubo
Sergey Katsubo•2w ago
What is in the folders, just in case, find /usr/src/app/upload -maxdepth 2 -ls Can you touch .immich from inside the container? (NFS permissions regularly appear here on Discord)
nagyviktor
nagyviktorOP•2w ago
Touch failed 🙀
nagyv@server-2:~$ kubectl exec -n immich immich-server-77d9c4d457-pqwfn -- touch /usr/src/app/upload/encoded-video/.immich
touch: cannot touch '/usr/src/app/upload/encoded-video/.immich': Permission denied
nagyv@server-2:~$ kubectl exec -n immich immich-server-77d9c4d457-pqwfn -- touch /usr/src/app/upload/encoded-video/.immich
touch: cannot touch '/usr/src/app/upload/encoded-video/.immich': Permission denied
But why?! (I removed a few "redundant" lines to be allowed to post)
nagyv@server-2:~$ kubectl exec -n immich immich-server-6b56c67d55-9thjk -- find /usr/src/app/upload -maxdepth 2 -ls
93061122 4 drwxr-xr-x 8 node node 4096 Jan 21 2025 /usr/src/app/upload
93061133 4 drwxr-xr-x 2 node node 4096 Sep 20 02:00 /usr/src/app/upload/backups
...
93061134 4 -rw-r--r-- 1 node node 13 Sep 20 22:53 /usr/src/app/upload/backups/.immich
93062477 21948 -rw-r--r-- 1 node node 22474677 Sep 17 02:00 /usr/src/app/upload/backups/immich-db-backup-1758067200019.sql.gz
93061553 21956 -rw-r--r-- 1 node node 22474942 Sep 12 02:00 /usr/src/app/upload/backups/immich-db-backup-1757635200016.sql.gz
...
93061127 4 drwxr-xr-x 3 node node 4096 Jan 21 2025 /usr/src/app/upload/upload
93061128 4 -rw-r--r-- 1 node node 13 Sep 20 22:53 /usr/src/app/upload/upload/.immich
93061135 4 drwxr-xr-x 258 node node 4096 Jan 30 2025 /usr/src/app/upload/upload/0e8c0466-f7fb-4af2-8113-6401ab0596f0
93061129 4 drwxr-xr-x 2 node node 4096 Jan 21 2025 /usr/src/app/upload/profile
93061130 4 -rw-r--r-- 1 node node 13 Sep 20 22:53 /usr/src/app/upload/profile/.immich
93061131 4 drwxr-xr-x 3 node node 4096 Jan 21 2025 /usr/src/app/upload/thumbs
93061132 4 -rw-r--r-- 1 node node 13 Sep 20 22:53 /usr/src/app/upload/thumbs/.immich
93061139 4 drwxr-xr-x 258 node node 4096 Mar 22 2025 /usr/src/app/upload/thumbs/0e8c0466-f7fb-4af2-8113-6401ab0596f0
93061123 4 drwxr-xr-x 3 node node 4096 Jan 24 2025 /usr/src/app/upload/encoded-video
93061124 4 -rw-r--r-- 1 node node 13 Sep 20 22:53 /usr/src/app/upload/encoded-video/.immich
...
$ kubectl exec -n immich immich-server-544db9fc8-swqx8 -- id
uid=1000(node) gid=1000(node) groups=1000(node)
nagyv@server-2:~$ kubectl exec -n immich immich-server-6b56c67d55-9thjk -- find /usr/src/app/upload -maxdepth 2 -ls
93061122 4 drwxr-xr-x 8 node node 4096 Jan 21 2025 /usr/src/app/upload
93061133 4 drwxr-xr-x 2 node node 4096 Sep 20 02:00 /usr/src/app/upload/backups
...
93061134 4 -rw-r--r-- 1 node node 13 Sep 20 22:53 /usr/src/app/upload/backups/.immich
93062477 21948 -rw-r--r-- 1 node node 22474677 Sep 17 02:00 /usr/src/app/upload/backups/immich-db-backup-1758067200019.sql.gz
93061553 21956 -rw-r--r-- 1 node node 22474942 Sep 12 02:00 /usr/src/app/upload/backups/immich-db-backup-1757635200016.sql.gz
...
93061127 4 drwxr-xr-x 3 node node 4096 Jan 21 2025 /usr/src/app/upload/upload
93061128 4 -rw-r--r-- 1 node node 13 Sep 20 22:53 /usr/src/app/upload/upload/.immich
93061135 4 drwxr-xr-x 258 node node 4096 Jan 30 2025 /usr/src/app/upload/upload/0e8c0466-f7fb-4af2-8113-6401ab0596f0
93061129 4 drwxr-xr-x 2 node node 4096 Jan 21 2025 /usr/src/app/upload/profile
93061130 4 -rw-r--r-- 1 node node 13 Sep 20 22:53 /usr/src/app/upload/profile/.immich
93061131 4 drwxr-xr-x 3 node node 4096 Jan 21 2025 /usr/src/app/upload/thumbs
93061132 4 -rw-r--r-- 1 node node 13 Sep 20 22:53 /usr/src/app/upload/thumbs/.immich
93061139 4 drwxr-xr-x 258 node node 4096 Mar 22 2025 /usr/src/app/upload/thumbs/0e8c0466-f7fb-4af2-8113-6401ab0596f0
93061123 4 drwxr-xr-x 3 node node 4096 Jan 24 2025 /usr/src/app/upload/encoded-video
93061124 4 -rw-r--r-- 1 node node 13 Sep 20 22:53 /usr/src/app/upload/encoded-video/.immich
...
$ kubectl exec -n immich immich-server-544db9fc8-swqx8 -- id
uid=1000(node) gid=1000(node) groups=1000(node)
I found the issue. It was a combination of my NFS export config and the previously created PersistentVolume (obviously) not picking up an NFS configuration change, when I fixed my NFS config. Basically, I forget to remove/recreate the PV when I fixed the NFS export. As I recreated the PV, the pod could start.
Sergey Katsubo
Sergey Katsubo•2w ago
Oh huh 🫣 Cool you resolved it

Did you find this page helpful?