Truenas Scale Move 6 Users
Truenas Scale installation for 6 users. Works fine. No YAML, straight from their App Repository. I can post setup pics if that would help.
However, initially I made some poor dataset/directory location choices and I now want to fix that. I have tried a couple methods to migrate everything backup/resotre, the server@home guides, youtube. Clearly I must be doing it wrong. Would someone mind walking me through it? Or is my best option to start with a fresh install and cli everyone's photos somehow into their new profiles, which again I might need some direction on.
222 Replies
:wave: Hey @Struan,
Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.
References
- Container Logs:
docker compose logs docs
- Container Status: docker ps -a docs
- Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
- Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA
Checklist
I have...
1. :blue_square: verified I'm on the latest release(note that mobile app releases may take some time).
2. :blue_square: read applicable release notes.
3. :blue_square: reviewed the FAQs for known issues.
4. :blue_square: reviewed Github for known issues.
5. :blue_square: tried accessing Immich via local ip (without a custom reverse proxy).
6. :blue_square: uploaded the relevant information (see below).
7. :blue_square: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable
(an item can be marked as "complete" by reacting with the appropriate number)
Information
In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:
- Your docker-compose.yml and .env files.
- Logs from all the containers and their status (see above).
- All the troubleshooting steps you've tried so far.
- Any recent changes you've made to Immich or your system.
- Details about your system (both software/OS and hardware).
- Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h).
- The version of the Immich server, mobile app, and other relevant pieces.
- Any other information that you think might be relevant.
Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)
If this ticket can be closed you can use the /close command, and re-open it later if needed.Show your datasets structures as well as the immich app settings
I might be able to help during this week
Thank you for the consideration and review. Here are images, I think, of what you asked for -
I have to work on other things, so I'll reply and ask more questions later, but why is the majority of the data comming from Nextcloud?
Is there a reason you use both of them?
What do you want to achieve? Have a clean datasets structure for Immich, separated from Nextcloud?
It was my first foray into self hosting. I had used Nextcloud initially for all files, photos, etc. I found it lacking, slow, etc. So I installed Immich, thinking I would eventually import all my back catalog of photos into it. Immich WAS/IS AWESOME. Nextcloud became more of a headache. Several months later, I would like to blow away Nextcloud entirely, move Immich to its own dedicated space going forward, and then separately pursue a file solution (Syncthing, FileRun, Phylum, etc...). So yes I want to have a clean dataset and structure for Immich separated from Nextcloud. Then Nextcloud eliminated.
Ok, but so you never used Nextcloud to upload photos into Immich? Because it should not work, I just want to be sure that you have no "missing" photos
I won't be able to help you right now to migrate, but in the meantime, you can already create the two datasets that will be required as explained in https://immich.app/docs/install/truenas/
Zero missing photos. Datasets will be done momentarily. Thank you for your willingness to help.
Done. I followed the easy steps as it stated the permissions for pgdata would be defined during the apps install process.
Ok, so you can first create a backup of the immich database now, and then shut down the app
Afterwards, you can follow the steps in the following link
You will have to adapt the "from" path names to what you have in Immich settings (the to name should also be adapted to be the path to your dataset, but the last bit -, like upload, library or else, should not change) and the rsync shoudl be run either as root or with sudo before
so like
sudo rsync fromPath toPathGotcha. Okay, when I tried the rsync method before, it embedded the old nextcloud file structure within the new folders. Is this the expected behavior? In other words, Nextcloud was showing up all over the place 🙂
wdym by old nextcloud file strucure? like in some folders there was some nextcloud folders?
The rsync would copy from one mnt to the destination mnt. But inside of the destination mount would be the Nextcloud nested directories. Also when running the rsync commands, I get a prompt titled cmnd and usually type Y and then it runs.
well rsync copy the whole content of the folder, so yeah if there are nextfloud things inside, they will be transfered
But in your path there is folders like upload, library…
Do you rsync including their name? Or your just rsynced the "parent" path so /mnt/fancy/nextlclouddata/…/file/Photos
The later, using just the mnt paths as indicated above. To try and be as exact as possible. No worries, going to follow your direction above shortly, between work calls 🙂
Well so that's normal
you only should rsync the folders you want to rsync
so follow the commands and it should be fine
You could even paste one here, if you want me to check
Excellent will do. And maybe that is also the problem. Immich creates a ton of little subfolders hehehe
But so, just to to be sure, could you list all names in this nexctloud/file/Photo/ thing?
Example a...
But where is that?
Again, I think it happened when I initially set it up with everything kinda going to one place.
Directly in the photo folder? Or in a subfolder?
Like is there a parent folder for that?
bind /mnt/fancy/nextclouddata/Struuan/files/Photos /data/upload
volume /mnt/.ix-apps/docker/volumes/68ca5fea8c44a8ae06edcd84b1a5e02b90f6a6e1c3ac07e418c52a152b124dcf/_data /data
bind /mnt/fancy/nextclouddata /data/backups
bind /mnt/fancy/nextclouddata/Struuan/files/Photos /data/encoded-video
bind /mnt/fancy/nextclouddata/Struuan/files/Photos /data/library
bind /mnt/.ix-apps/app_mounts/immich/profile /data/profile
bind /mnt/fancy/nextclouddata/Struuan/files/Photos /data/thumbs
volume /mnt/.ix-apps/docker/volumes/68ca5fea8c44a8ae06edcd84b1a5e02b90f6a6e1c3ac07e418c52a152b124dcf/_data /data
bind /mnt/fancy/nextclouddata /data/backups
bind /mnt/fancy/nextclouddata/Struuan/files/Photos /data/encoded-video
bind /mnt/fancy/nextclouddata/Struuan/files/Photos /data/library
bind /mnt/.ix-apps/app_mounts/immich/profile /data/profile
bind /mnt/fancy/nextclouddata/Struuan/files/Photos /data/thumbs
Ooh yes hmmm
hahaha, feel free to backout at anytime 🙂
yeah it's more complicated than I thought, your install is a real mess
You said you have 6 users, right?
So is there in that 6 folders with a relatively long, random name?
Yeah, that is why I was wondering if I should just start fresh. Build new users. Then somehow import photos from their old immich install into the new immich install. And let them rebuild facial and thumbs from there.
That's the idea
thumbs and encoded-video can be regenerated
Looks like only 4 ever activated.
Yep so, forget the rsync command except for thoses and the database
okay so only the users and the database, got it.
I'll give you the instructions
1) shutdown immich
sorry
can you give me the path of these?
okay those are the individual storage folders or their photos, not their actual profiles.
the path is /mnt/fancy/nextclouddata/Struuan/files/Photos/26a30c7e-8038-488e-bb66-0826b54da649 etc
no I meant for your new datasets?
mnt/fancy/data/immich/data
mnt/fancy/data/immich/pgdata
For the db:
For the upload folders
You repeat the one for folders chanching the 26a30c7e-8038-488e-bb66-0826b54da649
The Trailing
/ is missing in /mnt/fancy/nextclouddata/Struuan/files/Photos/26a30c7e-8038-488e-bb66-0826b54da649 and that is normal!
You have to keep to make sure that they are / when written, but not for the other
Is that clear?so missing / for each users folder is required, but for database must use /
Yep, and the second parts like mnt/fancy/data/immich/data/upload/ have to have this /
oh okay, so I am still supposed to do the others. I thought I was only doing the above.
I will type it all up on my end and post here to be exact
the others ?
You don't have to rsync for library or encoded-video if that is what you meant
but you have to rsync your 4 users
as after these commands (the db one and the for slighlty different for the upload), you will do
Then lmk
oh the upload is what threw me of.
off
Is that better?
Right and I will do that for each user. Then the postgres data one. Then the 6 you just posted. So a total of 11 rsyncs
Well these are not rsync
ACK, sorry just caught that.
but yeah 11 commands
So do the 5 rsyncs first. Then the 6 line items above
yes, would be the "easiest"
on it.
thank you, sorry it is such a mess.
even though here all command (except the las one) should work if you do them before the rsync
i will do them in the order you have stated. Rsyncs first. Then the commands.
Well next time, learn/read/follow a guide to install this kind of "sensible" app, as I suppose it will be a backup of your photos, it would be better to know how it works
It’s an advice don't take it in the wrong way, TrueNAS gives the impression that something is easy to do, but then you don't really know what it does and you end up in your situation
yep, lmk if there are any issues or when you have finished that, as I will ask you to check differents things before doing anything else
Not taken wrongly at all. I was net new to all of this. I tried following the guides, they confused my old man windows brain. So I went to Youtube, found a guy, followed his advice. Only as I have got more into this, successfuly Arrstack, Mealie, Crafty, Glutun, and some others, did I realize the error of my initial foray. Nextcloud/Immich. And honestly, thought shouldn't be that hard to fix. Several days scouring guides, trying things, nothing working. I finally had to sulk my way here.
Beginning
Ooof.
it is there...
Oh fck
mb
I updated the commands
I forgot the / before /mnt in the second part of all commands
holy heckin that did a thing :0
we are cooking with bacon now
yeah this looks completely different than my previous attempts with rsync
It is having a moment on my profile. Going to take awhile it looks like.
Oh
You can know that if you want
do ctrl + c to cancel the sync
(no worries rsync handles that)
and after -av, you can add --info=progress2
It is still going on my folder, but it is by far the largest at close to 50g. The others are all less than 10g I think.
1st one done, and oops it was 77g
Oh yeah if other are smaller it should be fast
All completed. Only one failed.
Maybe redo this to be sure
sudo touch /mnt/fancy/data/immich/data/encoded-video/.immich
and aftewards your do
And show the outputRedid the following.
yeah the error is normal, the folder you try to create with mkdir already exists , thats why it fails
and thats why I made you do the touch command alone
That output is pages long from the -laR
Oh yeah, I'm dumb
Have you seen my logo. You can't get much dumber than a pet rock...
Try this insteaad
hahaha still pages
well ok
Try ls -la first
and show me the output
and that? does it work?
yes it worked
OK so now
You will edit the permission of the data dataset
Like he dit button here
And you ensure that you have the same users as I do, and tick almost every box (as shown in the screenshot)
I do, but should I check "apply permissions recursively", then "confirm", then apply permissions to child datasets? I do not have the first checked.
Do like in the screenshot
there is also two boxes on the top
I did, and hit Save Access Control List
Yup
all done.
Now you go to edit your Immich app
Edit, or fresh install? Your call.
In the storage section you untick the box Use old storage config
okay.
Mmh
You could fresh install, so it's "better"
Yeah let's do that. And in case it doesn't work, I can fall back if needed.
but first edit the app and save the two password
(especially the database one, we will need to use it)
Got them both. I was going to reuse them anywhos.
SO you can create the new Immich app
You'll have to change the name it uses (it can't be changed)
The port I thing it can reuse it as long as the other Immich isntance is down (which is fine as you'll delete it afterwards)
not sure i follow that last one.
Before the port.
Yes I gave it a new name "immichnew"
yeah you figured it out
What about database storage, I think it is now on my hdd.
Yes, well I suppose
okay picked hdd
/mnt/fancy/data/immich/data is it on HDD ?
yeap
DO you have an SSD pool?
yeap
Gne
Then it's better to use it for the databse xD
cancel the app for now
okay it is called dockers...
Cancelled. Sorry about that, didn't realize it was a thing till we got to it.
Np
So what you do is just create a "pgData" (you can name it how you want) on this docker pool
you can also nest it as it is currently done
so a "blank" immich dataset, and then a children pgData
And afterwards you do something like this
adapt /mnt/dockers/immich/pgdata/ based on how you created the dataset
datasets created
ok so do this then
Btw, you could even put the thumbs inside this SSD pool, but that will be for another day
done
SO you go back to create your Immich App
Okay, SSD selected, at UID and GID
keep it 568
Okay, same port, moving on. Now at Storage Configuration.
There select the data Dataset for the first thing (when you select host path)
Machine learning you can keep it Temporary
And postgres, select the pgData on your SSD pool
and tick the apply permission box
And on the Ressources config, you should at leat put 6144 for ram (recommended by Immmich) or more if you have
And you can definitively assing more CPU if you have more cpu (it's based on threads, so if you have a 4 core cpu with 2threads its 2*4 so 8
and then no gpu, but I do have a modern Intel chip with the built in arc endcoding, so should I check the box or not?
Oh yeah
I do not know your system so I can't help on that lamo
okay, then install button.... scary.
yes
here it goes
deploying
running
all containers?
Well try to check your Immich instance if that works
hmmm it looked promising. but most if not all thumbnails are blurry, and nothing opens an actual image.
Thumbnails should be burry
if you click on one and dowload it, does it work?
This is expected, but can you download it?
yes sorry. Had to figure out, hehehe, download works fine.
Ok so go to you jobs
https://my.immich.app/admin/jobs-status
errrr. where? So so sorry.
And run the "genereate thumbnails" for all
running
11.5k waiting
You should/can also run the transcode videos one
should i wait till the first one finishes?
This ways it rebuilds everything
Nah, it can do both
4.6k waiting on that one.
Wait
rut roh
You said you have an intel cpu
yes
Edit your Immich app then and select openvino in the machine learning stuff
so pause/clear the video transcode and the thumbnails. then go change the app and come back to start the jobs again? just want to make sure I do things in your correct order
No edit the app
No need to stop the job, they'll be stopeed automatically
And should also resume automatically, but maybe not if not, just click on all again
Okay changed, app is updating.
(in the future, you won't need to click on all, It just to make sure that everything is rebuilded now)
https://my.immich/app/admin/system-settings?isOpen=machine-learning+smart-search
Here you could change the AI model your app uses
so off topic, what if anything do you use for file syncing/versioning/cloud access, ie Nextcloud?
we are running again.
If you change it, as written, you'll have to re run the "smart search job" for all
check if jobs are running
I wouldn't know what to put here.
jobs are not running.
run them?
yep
jobs are running
So looks like this is going to take a bit. The transcoding seems to have bottlenecked it
bottom of page there is a list of models available + explaination
Depends for what
Syncthing for sync between devices
Cloud access, there is Opencloud (but it's more complicated to install) or maybe you could use filebrowser to do something similar
Yeah was looking at the new Filebrowser Quantum.
And syncthing.
I could also explain that if you're interested, but only once the big job is done
Oh and if everything is fine (maybe wait a bit) but you could delete the "old immich" app (and select the remove ix-volumes when you delete it)
And all the folders inside your Nextcloud dataset/folder
Yes thank you. Will make sure the kids can access their photos remotely then nuke it.
So this would seem to be the best choice, as it is at the top of the list. ViT-SO400M-16-SigLIP2-384__webli
Yes, that's also the one I use as it performs great in multiple languages including mine
well if English isn't your native language, you have mastered it better than I. But there you have an American education for you. hehe.
Well I'm not bilingual, but my english is fine (especially for a someone of the south part of Belgium where we speak french, I wish I spoke dutch as well, as I would be able to communicate with the people from the north)
Ahhh. I now have extended family in Verdun. Couple Dutch guys on my team, as well as many others from Europe.
Europe is still vague lmao, but that is good to have some diversity in a team! (I think)
But comming back to Immich, there are a lot of things you could modify, you should have a look at the docs
Like using storage template so your files are not saved in a non human readable structure
Dutch guy living in Ireland, handful in the UK, 2 guys in the Netherlands, boss and a handful in France, then several stateside.
Yes will do now that it is in its forever home.
The folder storage sounds better than whatever is going on right now.
Make sure also to have the backup jo enabled
?
so the database is backed up every x (based on cron job) on your HDD pool
The database backup happens everynight. Otherwise still building on my 3,2,1 backup. Still debating who to use for offsite.
yes, that is turned on.
Can't help on that, but you should/could also (if storage is an "issue") avoid backing up the thumbs and encoded-video folders
As they can be regenerated like you're doing now
Gotcha. Might consider that. The bulk of my storage needs are simply the family photos and videos. Over a TB on their own, without the extras. My documents are paltry maybe 200g. Oh and the GTNH server, that is about 20g, even though it backs itself up every 30mins.
GTNH?
Sorry a family of nerds. Gregtech New Horizons modpack for minecraft. We are about 9 months in, 1500hrs or so, leaving about 5000hrs to finish it
And once this is fully operational, I will be looking at Pangolin or the like, to bypass the Cloudflare upload limitations.
Oh well that is impressive, even though 20g seems ok for a modded mc map
Yeah, that's is a complain of a lot of people here
I use a reverse proxy and forward my 443 and 80 ports to it, so I don't have that issue
nice. I don't have a dedicated ip, and also wanted to make it easier on the family by using an actual domain name.
I don't either
There are containers that updates your IP automatically on your "domain provider"
So I have my own domain and it always points to my actual ip (maybe with a small offline sometime, even though I never encoutered any)
So all this is greek to me, still learning. 🙂
Yeah, its a bit complicated when you're not into it, the cloudflare tunnel solution is fine as it is really easy to setup
Pangolin will be harder, and I think you'll need a vps or something similar
It was much easier than Tailscale. Yes Pangolin will need that. Then again, I haven't found a STUPID BASIC GUIDE to setting up reverse proxy and the container you mentioned.
I read the words in the guides, but they don't necessarily work in practice for me, or assume some level of linux understanding.
Lawrence Systems
YouTube
Self-Hosted SSL Simplified: Nginx Proxy Manager
https://lawrence.video/
Looking for an easy way to manage wildcard SSL certificates for your self-hosted web applications? In this tutorial, I’ll walk you through setting up Nginx Proxy Manager using Docker, configuring your DNS correctly, and securing your apps with wildcard certs—all through an intuitive web UI.
Forums post with docker c...
LAWRENCE. He has aged well.
(It's one of the reverse proxy that exists, but its basic enough)
That's one of my reference, I learned a lot from its videos
I can say that as an old man.
I learned most of my home networking from him. Thanks for the video will check it out.
Okay 9.8k photos to go, and 3.7k videos.
Going to be tomorrow probably, will check back in and let you know how it went. Thank you again for all of this!!!!
https://www.youtube.com/watch?v=GarMdDTAZJo
https://www.youtube.com/watch?v=Nf7m3h11y-s
These could also help
Note that it is "less secure" than a cloudflare tunnel though
Awesome thank you, better to use trusted sources than my aimless youtube searching.
Can I buy you a dinner or coffee or something?
Well, I got suggested lot of tech channels like that that explain stuff well and so when I aimless search somehitng they are often recommended
Wdym? It was a pleasure to help
even though I used it as an excuse to delay the preparation of my master thesis defense, so I'll have to work on that RN, otherwise I'm screwed
Didn't know if there was a way to buy you a coffee or dinner via an app or something? Plus I would feel horrible if you weren't prepared for your thesis defense! That is outstanding by the way, great for you.
Well I had to ask if I was allowed to (and we are)
I'm not directly part of Immich, the support crew is a role I've been given just because I helped people and is voluntary work
I'll check what I could create for that
And if you really like Immich you could also support their work
I suppose that this shoudl work https://ko-fi.com/xiticks
Done, hopefully you get it.
Already received, thank you a lot!
Btw it won't give you priority support or anything, so don't ping me if you need help
Though if it is TrueNAS related, as I manage these issue most of the time, asking in #truenas will more than probably get my attention
No worries, and don't plan to have another support issue. Hoping this is set and forgetish.
Btw about "forgetish"
I suppose that is what you do, but check at least once a month/every two month for updates of Immich on TrueNAS
If they're breaking changes, it's harder to select the specific version on TrueNAS, so being really up to date avoid these kind of issue
(Even though it look like it's what you already do as you have a lot of photos and never asked for broken migration stuff)
Woot, everything has completed. Everything is working. THANK YOU!!!!
Also, good luck on your thesis defense!!!
Nice, amazing to read that! Try to check monthly if there aren't update for Immich, as it is not stable yet (and it is also better to have relatively up to date apps)
And thanks
Oh my bad I replied and thought it was another person, I’m lost in the dirfferent threads, sorry for the reminder 😅
I appreciate any and all help. I actually check daily for upgrades on all the apps.
Btw if you want/need help to remove the old, now unused, data, I can help with that later
This weekends adventure will be importing all my old photos. THen reverse proxy setup 🙂
That sounds to be a good plan
Btw don't treat Immich/HexOS as your only backup
Currently local pc, onedrive, and Truenas. Future will be local pc, truenas, 3rd party cloud (researching best pricing and was hoping for one of the Truenas Integrated Solutions).
Well that didn't get me very far. Following the immich cli guide. node.js and npm installed. immich cli installed. Tried running the upload command several different ways and getting a No files found, exiting. You can see some of the folder structure at the top of the image. Then my multiple tries at upload. I even tried the full folder path name, no dice.
Argh seems it doesn't like files with spaces in their name. If I remove the spaces, it imports. Seems a bit daft. Will explore batch file name changing apps for windows to remove spaces I guess, but will wait to hear if you have any suggestions.
Maybe you could also have a look at immich-go
Idk if it will be better
But you could also ask in the main #immich channel
Or open another issue related to that
Otherwise, you could also use the external library function as I said
This way you keep your current folder structure and don't "merge" it with Immich, while still getting all the features of Immich
Will explore, thanks.
Okay, everything is uploaded, all is well. How would I properly delete the old instance of Immich without affecting all the hard work you helped me to accomplish. Then how would I officially close out this request, or do I need to? That will be the finality of it.
All items completed. I just need to close the ticket.
Again thank you for all your help, hope your thesis defense went well, and everyone can now address you as Dr. Xiticks
It went well, thanks! (at least I passed)
As for your question, you can simply remove the app itself (and check the remove ix-volume box)
However, there might still be the different datasets on your system that could take space on your disk for nothing
From what I see, your photos were previously on Nextloud, so I think you can delete them from there to free up space on your NAS as we migrated them to Immich IIRC (double check, because I've lost the thread of this thread)
Hahaha. Well first CONGRATS! Huge accomplishment Doctor er Professor. Second already removed the old install plus all the ix volumes. Lastly, yes, the nextcloud dataset is deleted freeing up NAS room. Is there anything I should do to close out this support thread? Or just leave it and not come back hehehe
No no, it was a master thesis so it's less impressive (idk how it works in the US, but here, in a university, we have 3 years of bachelor, then 2 of master)
You can do the /close command in the chat to close the ticket
Still impressive!!!
This thread has been closed. To re-open, use the button below.