Photos not uploaded in the specified location (but somewhere idk where)

I have a server running ubuntu server that has 4 hdd working as a ZFS pool. ive mounted the pool via zpool and i have access to the files with no problem. In my .env file i set:
UPLOAD_LOCATION=/Z2POOL/.../library
DB_DATA_LOCATION=/Z2POOL/.../postgres
UPLOAD_LOCATION=/Z2POOL/.../library
DB_DATA_LOCATION=/Z2POOL/.../postgres
uploading images worked until i had an error saying i ran out of space. (the pool is about 8 TB) from the start i saw that in the client app there is a message saying "4GB left" or something like that on the bottom left screen. when i cded to the library location i saw that it was completely empty and so is the postgres folder... which means immich uploaded the photos somewhere else and i couldnt find where.. I'm guessing it has something to do with the ZFS Pool, right? could be something else tho. thx
40 Replies
Immich
Immich4mo ago
:wave: Hey @DORON ✡, Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:. References - Container Logs: docker compose logs docs - Container Status: docker ps -a docs - Reverse Proxy: https://immich.app/docs/administration/reverse-proxy - Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA Checklist I have... 1. :ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time). 2. :ballot_box_with_check: read applicable release notes. 3. :ballot_box_with_check: reviewed the FAQs for known issues. 4. :ballot_box_with_check: reviewed Github for known issues. 5. :ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy). 6. :ballot_box_with_check: uploaded the relevant information (see below). 7. :ballot_box_with_check: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable (an item can be marked as "complete" by reacting with the appropriate number) Information In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider: - Your docker-compose.yml and .env files. - Logs from all the containers and their status (see above). - All the troubleshooting steps you've tried so far. - Any recent changes you've made to Immich or your system. - Details about your system (both software/OS and hardware). - Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h). - The version of the Immich server, mobile app, and other relevant pieces. - Any other information that you think might be relevant. Please paste files and logs with proper code formatting, and especially avoid blurry screenshots. Without the right information we can't work out what the problem is. Help us help you ;) If this ticket can be closed you can use the /close command, and re-open it later if needed. Successfully submitted, a tag has been added to inform contributors. :white_check_mark:
Mraedis
Mraedis4mo ago
Could you check docker inspect immich_server | grep -A25 Mounts
DORON ✡
DORON ✡OP4mo ago
type: bind
source: /Z2POOL/.../library
destination: /usr/src/app/upload
type: bind
source: /Z2POOL/.../library
destination: /usr/src/app/upload
mode: rw rw: true prooagation: rprivate @Mraedis
Mraedis
Mraedis4mo ago
ok so at least that is "right" what does lsblk and df -hT tell you?
DORON ✡
DORON ✡OP4mo ago
my disks sda to sdd 2.7T disks and sdf the disk where ubuntu server is installed and sde just some flash drive
Mraedis
Mraedis4mo ago
Ah wait, Ubuntu, you don't have docker snap do you? sudo snap list | grep docker
DORON ✡
DORON ✡OP4mo ago
i think yeah
Mraedis
Mraedis4mo ago
I ask these details to look with you so I can point out anything you miss btw, if you don't want to share private things I understand, but often people miss the obvious 🙂
DORON ✡
DORON ✡OP4mo ago
its not that i dont want to share its just too long to write xD
Mraedis
Mraedis4mo ago
Just copy paste?
DORON ✡
DORON ✡OP4mo ago
different machine
Mraedis
Mraedis4mo ago
screenshots, ssh, so many options 😛
DORON ✡
DORON ✡OP4mo ago
aight
Mraedis
Mraedis4mo ago
Anyway, NOT docker snap or YES docker snap? because if yes docker snap, that's the problem
DORON ✡
DORON ✡OP4mo ago
No description
Mraedis
Mraedis4mo ago
time to throw out docker snap
DORON ✡
DORON ✡OP4mo ago
when i installed ubuntu server there was a list of programs i could install and i saw docker there so i thought it would save me some time, no?
Mraedis
Mraedis4mo ago
Well no Canonical is snap-happy from Ubuntu 22 and onwards Docker snap works fine for small standalone containers, but not for when you need to mount, say, a large dataset to store images
DORON ✡
DORON ✡OP4mo ago
install with apt?
DORON ✡
DORON ✡OP4mo ago
ok let me try that!
Mraedis
Mraedis4mo ago
Do you want to start over or export the data?
DORON ✡
DORON ✡OP4mo ago
i think ill start over i didnt upload too much photos
Mraedis
Mraedis4mo ago
It'll be easiest by far 🙂
DORON ✡
DORON ✡OP4mo ago
after reinstalling docker i have a different problem now... "could not change permissions of directory "/var/lib/postgresql/data" operation not permitted" i ran it with sudo @Mraedis this folder doesnt event exist i created this folder and gave all users permission to it, still didnt work
Mraedis
Mraedis4mo ago
It's whatever your db data location is set to @DORON ✡ The var lib location is inside of the db container
DORON ✡
DORON ✡OP4mo ago
ohh weird that it doesn't have permission to this folder
Zeus
Zeus4mo ago
Is that a network mounted share?
Mraedis
Mraedis4mo ago
It's a folder on the ZFS pool
DORON ✡
DORON ✡OP4mo ago
it wont let me change the permissions! i have changed the owner of that folder and it still wont let me. idk whats going on!
Mraedis
Mraedis4mo ago
have you tried removing the folder entirely (so just leaving the folder 1 step above) and then restarting? So that docker will create the folder itself
DORON ✡
DORON ✡OP4mo ago
will try error creating mount source ... chmod: operation not permitted let me try a new folder maybe
Mraedis
Mraedis4mo ago
looks like docker just has no permissions on that folder is it 664?
DORON ✡
DORON ✡OP4mo ago
775 i think drwxrwxr-x
sudo mkdir test
sudo chmod 777 -R ./test
sudo mkdir test
sudo chmod 777 -R ./test
'Operation not permitted' maybe something wrong with the pool?
Mraedis
Mraedis4mo ago
What's the exact output of ls -la in that directory
Zeus
Zeus4mo ago
Please run df -h and show the output
DORON ✡
DORON ✡OP4mo ago
sever@sever-server:/Z2POOL$ ls -la
total 43
drwxrwxr-x 8 root root 9 May 7 12:57 .
drwxr-xr-x 24 root root 4096 May 6 21:13 ..
-rwxrwxr-x 1 root root 0 May 25 2019 .windows
drwxrwxr-x 2 root root 4 Nov 15 2016 SEVER-AFP
drwxrwx--- 5 root root 6 May 25 2019 SEVER-BACKUP
drwxrwxr-x 3 sever root 4 May 7 12:55 SEVER-NAS
drwxrwxr-x 6 sever sever 7 May 6 21:19 SEVER-SMB
drwxrwxr-x 8 root root 9 May 25 2019 jails
drwxrwxr-x 2 root root 2 May 7 12:57 test
sever@sever-server:/Z2POOL$ ls -la
total 43
drwxrwxr-x 8 root root 9 May 7 12:57 .
drwxr-xr-x 24 root root 4096 May 6 21:13 ..
-rwxrwxr-x 1 root root 0 May 25 2019 .windows
drwxrwxr-x 2 root root 4 Nov 15 2016 SEVER-AFP
drwxrwx--- 5 root root 6 May 25 2019 SEVER-BACKUP
drwxrwxr-x 3 sever root 4 May 7 12:55 SEVER-NAS
drwxrwxr-x 6 sever sever 7 May 6 21:19 SEVER-SMB
drwxrwxr-x 8 root root 9 May 25 2019 jails
drwxrwxr-x 2 root root 2 May 7 12:57 test
sever@sever-server:/Z2POOL$ df -h
Filesystem Size Used Avail Use% Mounted on
tmpfs 784M 1.6M 782M 1% /run
/dev/mapper/ubuntu--vg-ubuntu--lv 58G 12G 43G 22% /
tmpfs 3.9G 0 3.9G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
/dev/sdf2 2.0G 96M 1.7G 6% /boot
Z2POOL 2.8T 256K 2.8T 1% /Z2POOL
Z2POOL/SEVER-NAS 2.8T 256K 2.8T 1% /Z2POOL/SEVER-NAS
Z2POOL/SEVER-BACKUP 4.5T 1.8T 2.8T 39% /Z2POOL/SEVER-BACKUP
Z2POOL/SEVER-AFP 2.8T 256K 2.8T 1% /Z2POOL/SEVER-AFP
Z2POOL/SEVER-SMB 3.5T 717G 2.8T 21% /Z2POOL/SEVER-SMB
Z2POOL/jails 2.8T 256K 2.8T 1% /Z2POOL/jails
Z2POOL/jails/.warden-template-pluginjail-11.0-x64 2.8T 605M 2.8T 1% /Z2POOL/jails/.warden-template-pluginjail-11.0-x64
Z2POOL/jails/plexmediaserver_1 2.8T 4.0G 2.8T 1% /Z2POOL/jails/plexmediaserver_1
Z2POOL/jails/.warden-template-pluginjail-10.3-x64 2.8T 530M 2.8T 1% /Z2POOL/jails/.warden-template-pluginjail-10.3-x64
tmpfs 784M 12K 784M 1% /run/user/1000
sever@sever-server:/Z2POOL$ df -h
Filesystem Size Used Avail Use% Mounted on
tmpfs 784M 1.6M 782M 1% /run
/dev/mapper/ubuntu--vg-ubuntu--lv 58G 12G 43G 22% /
tmpfs 3.9G 0 3.9G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
/dev/sdf2 2.0G 96M 1.7G 6% /boot
Z2POOL 2.8T 256K 2.8T 1% /Z2POOL
Z2POOL/SEVER-NAS 2.8T 256K 2.8T 1% /Z2POOL/SEVER-NAS
Z2POOL/SEVER-BACKUP 4.5T 1.8T 2.8T 39% /Z2POOL/SEVER-BACKUP
Z2POOL/SEVER-AFP 2.8T 256K 2.8T 1% /Z2POOL/SEVER-AFP
Z2POOL/SEVER-SMB 3.5T 717G 2.8T 21% /Z2POOL/SEVER-SMB
Z2POOL/jails 2.8T 256K 2.8T 1% /Z2POOL/jails
Z2POOL/jails/.warden-template-pluginjail-11.0-x64 2.8T 605M 2.8T 1% /Z2POOL/jails/.warden-template-pluginjail-11.0-x64
Z2POOL/jails/plexmediaserver_1 2.8T 4.0G 2.8T 1% /Z2POOL/jails/plexmediaserver_1
Z2POOL/jails/.warden-template-pluginjail-10.3-x64 2.8T 530M 2.8T 1% /Z2POOL/jails/.warden-template-pluginjail-10.3-x64
tmpfs 784M 12K 784M 1% /run/user/1000
the immich data location is in SEVER-NAS/library and /postgres this pool was previously used in a freenas machine by the way
Zeus
Zeus4mo ago
Sorry, I have no idea. Most of us use ZFS managers like proxmox or truenas I think you need to figure out why your ZFS is broken, it’s not immich related
DORON ✡
DORON ✡OP4mo ago
yeah probably thx anyways took me some time but i solved it :D anyways not a immich related issue
Immich
Immich4mo ago
This thread has been closed. To re-open, use the button below.

Did you find this page helpful?