Truenas and Immich migrate to the new storage configuration format
As usual, I can’t get this done quickly without issues.
I ran:
sudo rsync -a --progress --remove-source-files --delete /mnt/.ix-apps/app_mounts/immich/uploads/ /mnt/SAMSUNG_1TB/Immich/data/upload/
But instead of moving all the data to the new location, I see that the data is being copied, and in a few minutes I’ll run out of free space.
My data is around 600 GB, but I only have about 50 GB of free space left.
How can I solve this before I run out of space?
148 Replies
:wave: Hey @dzik,
Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.
References
- Container Logs:
docker compose logs docs
- Container Status: docker ps -a docs
- Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
- Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA
Checklist
I have...
1. :ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time).
2. :ballot_box_with_check: read applicable release notes.
3. :ballot_box_with_check: reviewed the FAQs for known issues.
4. :ballot_box_with_check: reviewed Github for known issues.
5. :ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy).
6. :ballot_box_with_check: uploaded the relevant information (see below).
7. :ballot_box_with_check: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable
(an item can be marked as "complete" by reacting with the appropriate number)
Information
In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:
- Your docker-compose.yml and .env files.
- Logs from all the containers and their status (see above).
- All the troubleshooting steps you've tried so far.
- Any recent changes you've made to Immich or your system.
- Details about your system (both software/OS and hardware).
- Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h).
- The version of the Immich server, mobile app, and other relevant pieces.
- Any other information that you think might be relevant.
Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)
If this ticket can be closed you can use the /close command, and re-open it later if needed.
Successfully submitted, a tag has been added to inform contributors. :white_check_mark:There is another way, I can help with that tomorrow
Well, so can you share
1) Current Immich config (when you click on edit on the app)
2) Current dataset layout
1) Its Immich installed from Truenas catalog (v1.9.28)
Immich data is currently in .ix-apps on SAMSUNG_1TB pool
It might not be ?
Can you show the storage config, clicking the edit here
Using Truenas terminal, coud you cd into
then
du -h --max-depth=1```
and also show the output
the latter command might take time
and I suppose that part of my photos is in new location
From where did you rsync these
From mnt/.ix-apps/app_mounts/immich/uploads/ to /mnt/SAMSUNG_1TB/Immich/data/upload/
OH YEAH
That's write you have space issues
Is there any chance to fix this, or do I need to download everything to my computer and upload it again to TrueNAS?
No, what you can do is move stuff instead of rsync
but it would be better if you have a backup
So far I have part of data in new location and rest in old .... and beside this I cannot use mv (destination folder is not empty). The best option could be do backup but not via filezilla (it will take a week like last time to backup)
Well, you could remove the content of the destination
Once it's done
(it's type correctly upload changed it's name)
Well even without backups this should be fine though
So you could do it
so firstly I have to download
/mnt/SAMSUNG_1TB/Immich/data/upload
because I did
'sudo rsync -a --progress --remove-source-files --delete /mnt/.ix-apps/app_mounts/immich/uploads/ /mnt/SAMSUNG_1TB/Immich/data/upload/Mmmh
But it doesn't seem to have removed them?
But yeah, maybe backup stuff before doign that
tbh looks like its removed from source path because initially it was more than 500GB of data
Oh, well then
Just keep goind with the rsync command
it shouldn't fill up your space
but it did 🙁
Wait
Where is your apps pool ?
Apps isn't installed on the same pool?
Yes, it is
btw: its mean that now I can install immich on my SSD and keep database on HDD. Is that true?
I would say the opposite, but yeah
have Immich database (and maybe thumbnails) on SSD
the rest on HDD
But 1) i have immich from Truenas catalog 2) with new database location i cannot select separate path for thumbnails ( in fact this major reason why i mounted ssd to my hp t620)
TrueNAS [Community] | Immich
This is a community contribution and not officially supported by the Immich team, but included here for convenience.
Read the Multiple Datasets for Immich Storage
That's how I have it installed
Result: 1) its works, 2)I have another problem : some pictures are missing but thumbs are available so if I upload again, Immich recognize it as duplicate
any where is my space if I moved all immich database ?
What do you mean moved all Immich database?
Check in /mnt/.ix-apps/app_mounts/immich
or here /mnt/.ix-apps/app_mounts/
IT's not the thumbnails, is the database, that thinks the files are still there
you would have to delete them (from timeline and from trash) so they can be reuploaded
the biggest folder (upload) moved to new location from /mnt/.ix-apps/app_mounts/ to /mnt/SAMSUNG_1TB/Immich/data/
😮 I have to check 25k photos manually???
and if you do
du -h --max-depth=2 inside the /mnt/.ix-apps/app_mounts/
Isn't there any other stuff that take space?
I thought you said it was only the 4 you had therejust randomly checked but there is much more...
That doesn't seem right then
i thinkt that there is a problem
Oh yeah
but keep them for now
as you miss a lot of photos, something definitely went wrong with your rsync
I restored one snapshot and Im doing backup of that - then I will try to merge with existing datas
Yes, that's the safer solution
rsync should avoid duplicates
dammit, now Im out of space 🙁 and I cannot remove restored snap... im getting sick with this and im gonna to clean everything
I understand that's it's frustrating
You could probably regain space
Deleting the thumbs and video (encoded-video) folder is safe to do
there is no importnat data there
I don't think that in hte ix-applications there is anything relevant either
It's probably your "ancient" data from pre-docker trueNAS versions
You'll have to doublecheck, but if nothing is relevant then it's also 40gb regained
seriously it is safe t delete encoded video? thumbs I already moved to SSD
encoded-video only contains well transcoded video, based on Immich settings, to improve the compatibility/playing on slow network
The originals are saved inside the upload folders alongside the original images
You'll just have to rerun the "transcode video" on "all" inside Immich jobs
It will use your cpu/gpu, but it will regenerate them
ok so Im going to delete encoded video - maybe I will get few GB
anyway something is realy weird coz my database (I mean upload folder) is around 500GB so where is rest space on my HDD
you can also probably delete it in the restored snapshot (though I'm not sure you'll gain space here as you have snapshots)
inside the /mnt/.ix-apps/app_mounts/
Can you, using a terminal, as root, cd into this directory
you might need to connect using ssh to not loose the connection
im using extraputty
yeah, so can you cd into the path I gave?
as root
do a
ll there
and share the output?
can you do
du -h --max-depth=2 this command there also?
It might take a while to run
and share the output
Oh well not in there then
Mmmh
Can you share the whole page there? When clicking on SAMSUNG_1TB
The data is probably in the snapshots as indicated here
do you mean ?
There is a graph on the right of that
Like multiple widgets
Yeah that makes sense, my bad on asking that
What if you cd into /mnt
Then do
zfs list -o name,used,avail,usedbysnapshots,usedbydataset,usedbychildren -r SAMSUNG_1TB
yes, so the 500gb are in the snapshot of your original Immich dataset (in mnt./ix-apps)
I did a backup of restored snap to my pc (seems that there is full database of upload folcer)
sorry its in Polish
and where is the restored snapshot now?
you deleted it?
Upload was 508gb there, not 494
ahhhh
I don't understand how you got that much missing if you rsynced from a snapshot
well a restored snapshot
with rsync I lost only space on HDD, finally I had to use mv
Ok so you might have 97%/98% of the files of your computer
So what you could try to do is
1) Delete your "Immich" Dataset and its snapshots (not the /mnt/.ix-apps/app_mounts/immich, but the other that we see in the screenshot)
2) restore the /mnt/.ix-apps/app_mounts/immich and it's children (especially pgData and uploads)
3) Create back Immich dataset with pgData and data as children datasets
4) After that you try, for the uploads folder to move the files (with the mv command) to a "upload" folder you create using the terminal inside the data dataset that is inside the Immich dataset
5) You rsync the pgData into the pgData of Immich dataset
ohh im little bit lost... so i have to delete immich database (after migration)?
and about delete snapshots i do not understand
But this migration did not work, right? You have files/folders missing, correct?
it works but with missing data
Yes, so deleting it will make you regain the space back
and we will fix it that way
this snapshots is correct?
Yes this snapshot looks like it contains all data, so DON'T DELETE IT
You need to restore it
ok I trust you 😉 so im going to rm immich database
don't call it database it's an entire different thing
okk each time I mean upload folder
Not this way
DElete it from the gui
As it will also delete snapshots linked to it, if you have some
yes Sir !
is that correct? im sorry for tidious question
Yes
Btw can you show what’s inside your ix-applicztiôs dataset ? I only see it used as a smb share
You could also gain 45gb there
Thre is only that there?
yes it is
can you open the pgBackup ?
You never pointed your new Immich install to this dataset, correct?
tbh in the past it was used but idk when it was changed to /.ix-apps/
and right now idk that it is used
It's legacy thne
that's 40gb you could be able to gain
so you think that I can remove it?
so after remove Immich (directory) i have 236GB
I think
I'm not gonna say 100% to not take the responsability though 😄
Can you restore the snapshot there then?
I'll continue tomorrow
It's 2 am for me
the same timezone 🙂 ok THANK you for support - today later I will write to you 😉
What I would "like you to do until I'm "back" is
1) Restore the snapshot
2) check that the folder size inside /mnt/.ix-apps/app_mounts/immich is around the 508gb (or check if the size is around the same as on your computer)
2) rsync the restored content of the snapshot to your computer (in same folder you showed me, this way it'll only sync the non synced one)
copy that !
last question : clone to new dataset or rollback?
I think that cloning to new dataset might need the full size
Maybe check that on the internet
ok I did clone
good night !
Hello again 😉 a small update : I restored snapshot and I downloaded it to my PC (btw due to this backup I have 23GB of 2TB left :D) looks like with previous snapshot, there was a lot of missing data. So now I will try to mv snapshot upload to new repository (but I will wait for your confirmation 😉
Means: create a new dataset at '/mnt/tank/immich' and then mkdir -p /mnt/tank/immich/data/{upload,thumbs,library,profile,backups,encoded-video}?
Nope
It's more like that
then inside data you create the folders
Ok so you have a copy of the good snapshot
What does it contain?
Only the upload folder or the other?
Because you'll have to do a risky manipulation on NAS side, so I rather be certain that you have an unmodiffied copy of everything
Deleting all snapshots on the ix-apps, this way, when you mv stuff, you'll actually not take more space
it is only uploads folder
ohhh I realized what I did... I fucked.. I removed all immich new repository with data and pgData
There isn't a snapshot that could contain the content of backups?
if you have the backups folder and the upload one, it's possible to restore Immich
this is enough?
It looks like, yeah
backups contains only dataset dump
so it doesn't take that much space
Phew, that’s good — I thought I was about to have a heart attack.
Well you have the originals, so worst case it to upload everything again
im almost ready for that but maybe its a chance to avoid this 🙂
ok so now Im going to create dataset
so you have the backups folder also on your computer right?
Can you show the content of it?
name 1.9.7 is wrong but I merged all data with the biggest snapshot
No but I mean here
there is a backups folder
ohhh looks like it quiet old 🙁
not pgBackups
it's an old one
look backups
ok
well september 10th seems to be the latest backups you have
from all snapshots looks like this it the latest and more valuable
anyway its perfect
Yeah, so that is "fine" I suppose you still have the originals somewhere from the last month
So copy this backup folder to your computer also
already have
Then, you can delete all snapshots on the app_mounts/immich
btw. Do I need any different password than main to Immich?
? wdym?
at the begginig I think that I had to create password (probably at Truenas level) to Immich whole repository? but nvm 🙂
In immich config?
Yeah, it doesn't matter
Have you created these yet?
Ok, so now you can
mkdir -p /mnt/tank/immich/data/{upload,thumbs,library,profile,backups,encoded-video}
then mv from the /mnt/.ix-apps/app_mount/immich/uploads (the one you restored) into the upload folder you just created with the mkdir command
If that makes sense
done
so right now we keep snapshots?
Oh no
you have to delete them
ok so im going to delete them from gui
but I cannot delete
any idea?
Oh
did not saw that
that's the clone you created, so you have to delete it first
so its mean that I have to recover upload folder from my pc right?
No
You have to delete the clone (on the nas) first
via extraputty?
so I removed clone and upload gone
