Truenas data set madness
This will be a simple one for some, but I'm a complete noob thought it would be easy to understand it but so much conflicting info.
The two data sets needed by immich "data and pgData" in truenas, do they need to be under the same parent?
I have and 128GB SSD for apps and 2x24Tb mirrored drives for photos etc. Is that pdData data set (some people label as uploads), is that where my phones media will be stored? If so, then I cant set on SSD but HDD, but thought the "data" would be best on the SSD for speed. What am I doing/thinking wrong
9 Replies
:wave: Hey @DaveDee,
Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.
References
- Container Logs:
docker compose logs docs
- Container Status: docker ps -a docs
- Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
- Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA
Checklist
I have...
1. :blue_square: verified I'm on the latest release(note that mobile app releases may take some time).
2. :blue_square: read applicable release notes.
3. :blue_square: reviewed the FAQs for known issues.
4. :blue_square: reviewed Github for known issues.
5. :blue_square: tried accessing Immich via local ip (without a custom reverse proxy).
6. :blue_square: uploaded the relevant information (see below).
7. :blue_square: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable
(an item can be marked as "complete" by reacting with the appropriate number)
Information
In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:
- Your docker-compose.yml and .env files.
- Logs from all the containers and their status (see above).
- All the troubleshooting steps you've tried so far.
- Any recent changes you've made to Immich or your system.
- Details about your system (both software/OS and hardware).
- Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h).
- The version of the Immich server, mobile app, and other relevant pieces.
- Any other information that you think might be relevant.
Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)
If this ticket can be closed you can use the /close command, and re-open it later if needed.No, they can be separated
Though, given your current config, I would set both on HDD as they're mirror
The pgData contains the dataset (so the info of your users, where the photos are saved and stuff like that)
while data will contain the rest (your actual photos, the backup of the database, the generat thumbnails and generated video transcodes)
If you loose the databse, you don't loos the images themselves, but it's a bit harder to "recover"
If you decide to put pgData on your ssd pool, make sure the pg dump setting is activated inside Immich so you have a daily "backup" of the database in case your ssd dies
If you want to optimize it, I would recommend having at least mirrored ssd, and bigger capacities on them (this way you coul also put your thumbnails here so images load faster)
Oh ok, that finally makes sense to me and logical, thanks so much.
I'll probably simply create the dataset on the HDDs, because I have quite a lot to store and SSD's would be expensive for the volumes I need. I'm a semi pro photographer too, hence the requirements. I will be backing all to cloud too.
Another question. The "data" dataset that will copy all my phones media, is that the only folder immich can be assigned/see or can I add other folders, i.e. my photography archive I've already copied over, and other other ones in the future?
A side question to that. Do I need to use the "data" data set folder for phone uploads, as I was thinking of using syncthing as my main copier from phone for media, plus also sync files/documents. So, I would not need two services running on my/family phones. Instead use immich to sort and only be a viewing client?
the pgData doesn't take that much space though, so you could be fine by setting in on any of the two
Yes you can use the mobile phone as only a vewing device, but you can't put your photos with syncthing in the data dataset
It is still required for Immich to work
Btw if you backup to cloud, to save space (and thus money) you should/could exclude the folders "thumbs" and "encoded-video" that will be created inside the data dataset as they can be regenerated by Immich, it's not useful IMO to back them up
But if you plan to use syncthing, and also see your already archived photos, you should have a look at the external libraries
The way you set them up in TrueNAS is explained here
If you need/want more help for that you can always ping me here, or ask in the #truenas channel
Thanks again, crystal clear, finally. Thanks for tip on cloud, that I must do as yeah, it all costs.
I've had a brief look at external libraries, googling questions and reading some other threads.
Ideally as I'm using my truenas server as a NAS for my other photos and archiving, hopefully immich can still access assigned folders and update its library as I add more via smb on windows.
If I do use immich to save media to server from phines, I assume I can't assign "data" as a smb, so I can acces them via windows. I tend to use my phone to capture RAW photos too, so would be ideal to easily access them via windows like a NAS to edit?
I need to investigate and test further, it's not as simple as I thought to setup for a novice like me, though willing to learn. Thanks for your help this far, truly appreciate it.
If you upload to data from phone, you could set a samba share to READ them, but you can't do any modification to them (I'll ask if that is possible, but I don't think)
But if you use synthing on your Phone (or photosync could also work) and you put that in an external library it could work
Are you a developer? You explain in the most precise clear way. Thats another good reason to use syncthing types for my use case. I've not heard of photosync, thanks, I'll have a look.
I'll set data set folder in samba setting, reading would be ok for now but will only test it first. Hopefully it doesnt trip immich in use.
I still need to use immich on phones as the client, so my families can all see the shared images, whilst syncthing backs them up from each device to once location.
I've written down quite a lot, my requirements and what immich can/cannot do, like a flow chart. You've helped a lot, thanks.
Well not an Immich dev, but I'm ending my master theis in computer science so that might be that…
You will probably need to give your samba user to at least read it then
And you might need to set the storage template so you can navigate easily (photos will be stored in the library folder), as default structure. used by immich (stored in upload folder) is not "humanly navigable"
A masters! That makes sense for sure.
Excellent, I'll add to that to my notes of must do.
So far, a much more clearer outlook on my requirements, some in theory but hopeful it can work. Thanks ever so much and also, I wish you the best with your thesis. I remember doing mine a long time ago in CAD engineering, although not for a masters.