I
Immich5mo ago
Gonza

corrupted SD card, need to migrate immich settings to SSD

It´s been a year since I´ve been running Immich on my Raspberry pi 4B without any problems. OS on an SD card and all the data set to save on an external HDD via USB. Last update I reboot everything and SD card appears to be corrupt so raspberry pi wont boot. Bought an SSD and am planning to clone everything directly but some OS files are corrupt. I have an image of the SD card with all data in it but I dont know how to proceed to copy all Portainer (where i had immich) to the new SSD. Would highly appreciate help, because if not I would just have to duplicate everything and it´d be a mess with my IT knowledge. Thank you
355 Replies
Immich
Immich5mo ago
:wave: Hey @Gonza, Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:. References - Container Logs: docker compose logs docs - Container Status: docker ps -a docs - Reverse Proxy: https://immich.app/docs/administration/reverse-proxy - Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA Checklist I have... 1. :ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time). 2. :ballot_box_with_check: read applicable release notes. 3. :ballot_box_with_check: reviewed the FAQs for known issues. 4. :ballot_box_with_check: reviewed Github for known issues. 5. :ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy). 6. :ballot_box_with_check: uploaded the relevant information (see below). 7. :blue_square: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable (an item can be marked as "complete" by reacting with the appropriate number) Information In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider: - Your docker-compose.yml and .env files. - Logs from all the containers and their status (see above). - All the troubleshooting steps you've tried so far. - Any recent changes you've made to Immich or your system. - Details about your system (both software/OS and hardware). - Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h). - The version of the Immich server, mobile app, and other relevant pieces. - Any other information that you think might be relevant. Please paste files and logs with proper code formatting, and especially avoid blurry screenshots. Without the right information we can't work out what the problem is. Help us help you ;) If this ticket can be closed you can use the /close command, and re-open it later if needed. Successfully submitted, a tag has been added to inform contributors. :white_check_mark:
Gonza
GonzaOP5mo ago
Anyone willing to help out a noob? I just need to know which folder to copy and paste to the new install so i can maintain all my docker settings and installation files
Zeus
Zeus5mo ago
the data is in the folders specified by UPLOAD_LOCATION and DB_DATA_LOCATION in the .env file there is builtin database backups, which are stored in the backups folder of UPLOAD_LOCATION, which you can restore if that's all you have
Gonza
GonzaOP5mo ago
Oh i have the data, since i stored it on another drive, but i was wondering if i could restore all immich settings on docker portainer such as users albums etc. Which i had installed on the sd card Thank you for taking the time to help btw @Zeus Really appreciate it
Zeus
Zeus5mo ago
This shows you how to restore the database
Gonza
GonzaOP5mo ago
If im understanding, the database is where data is stored, which in my case was an external hdd that i conserve with all data in it. What im missing and trying to recover is the immich installation files (portainer) with all the settings i had set, users, shared albums etc. I understand these settings were not in the database, they were stored on my sd card, right?
schuhbacca
schuhbacca5mo ago
Users, albums, and such are all stored in the database. Nothing other than probably the docker images was on the SD card. If you follow Zeus link and restore the DB you should be all set.
Gonza
GonzaOP5mo ago
So if i do a fresh install, install docker and set a container with immich, all fresh and new, i connect the database and follow the link, i should be same as before?
schuhbacca
schuhbacca5mo ago
That I am not sure of. Zeus may know if you can point at the DB directory vs needing to restore an actual backup
Zeus
Zeus5mo ago
I think you should restore a DB backup. Especially if there was corruption of any kind, even on a different device
Gonza
GonzaOP5mo ago
Thank you guys. I'll try to follow the link instructions hi again, im trying to follow instructions to restore my previous immich settings, yet instructions are only on docker compose. I have everything set on portainer. Could i get some help to do the same but on portainer/docker? I know this is very demanding on my side, but would really like to proceed with the restore with someone monitoring so i dont lose all my files on the database hdd, im, not sure if by redoing all the process, i can overwrite my existing files. Just let me know if this is viable or not, i wont risk it by myself @schuhbacca @Zeus the problem here is that i do not have any backups, my backup folder is empty. I have all the rest of the folders,Encoded video, Library, Profile, thumbs and upload all full of my data and just as it was before my SD with the OS corrupted so i have no backup file to pull everything and follow the instructions
Zeus
Zeus5mo ago
If your backups folder is empty then you don’t have many options. You should make a copy of your current DB_DATA_LOCATION for safety then try starting a new immich, pointed to that same DB_DATA_LOCATION
Gonza
GonzaOP5mo ago
i just did that, fresh immich pointing to data location, and pointing to upload (all my hdd) and it just starts fresh. recognises how theres 130gb of data but it makes me create a new acc and nothing on it i tried loging in with one of my old users but it wont recognise it
Nicholas Flamy
Nicholas Flamy5mo ago
Take some screenshots of your configuration. Share you .env or whatever you're using to set these thing in portainer, as well as what the directory structure looks like.
Gonza
GonzaOP5mo ago
sure! this time i tried with a regular docker install, not via portainer
Nicholas Flamy
Nicholas Flamy5mo ago
For config that is text (like .env, you can send it between: ```
Gonza
GonzaOP5mo ago
i can send screenshots ov my docker compose and my env
Nicholas Flamy
Nicholas Flamy5mo ago
And ```
Nicholas Flamy
Nicholas Flamy5mo ago
ENV_CONFIG_EXAMPLE=WHATEVER
ENV_CONFIG_EXAMPLE=WHATEVER
No description
Gonza
GonzaOP5mo ago
Gonza
GonzaOP5mo ago
Gonza
GonzaOP5mo ago
this is my HDD with all the folders from old install
No description
Gonza
GonzaOP5mo ago
i can upload more info if needed
Nicholas Flamy
Nicholas Flamy5mo ago
@Zeus I'm guessing the issue is that db data is set to the same as upload I can't help much rn because I've got some stuff I gotta do, but basically, you should NOT have those the same @Gonza
Gonza
GonzaOP5mo ago
should i change UPLOAD_LOCATION to /mnt/Skyhawk/upload? or what am i supposed to do
Nicholas Flamy
Nicholas Flamy5mo ago
don't change anything yet stop immich first
Gonza
GonzaOP5mo ago
I can wait. Honestly i just want to restore what i had. if you can help tomorrow at any time i can get online whenever
Nicholas Flamy
Nicholas Flamy5mo ago
then ls /mnt/Skyhawk/upload
Gonza
GonzaOP5mo ago
immich is not running
Gonza
GonzaOP5mo ago
No description
Zeus
Zeus5mo ago
Sorry am out with the spouse this evening but leave lots of info about your setup here and someone will give some info We can only reply based on what’s shown to us
Gonza
GonzaOP5mo ago
i can upload all the info you need, im just not aware of what can be helpful and what cant
Nicholas Flamy
Nicholas Flamy5mo ago
Did you have storage template enabled? If you don't know then ls library and if you see a bunch of stuff then the answer is yes
Gonza
GonzaOP5mo ago
No description
Gonza
GonzaOP5mo ago
guess not
Nicholas Flamy
Nicholas Flamy5mo ago
But I think the solution will be to move everything in /mnt/Skyhawk into a subdirectory such as /mnt/Skyhawk/immich/storage or /mnt/Skyhawk/immichStorage Then you might have to import everything because afaict there's no database. This looks like storage template enabled.
Gonza
GonzaOP5mo ago
sounds complex i dont want to take up your time if youre busy but would appreciate steps on how to do so
Nicholas Flamy
Nicholas Flamy5mo ago
I'll try and help you later.
Gonza
GonzaOP5mo ago
okay. its 3am where i live
Nicholas Flamy
Nicholas Flamy5mo ago
Get some sleep.
Gonza
GonzaOP5mo ago
if you can, just leave steps here whenever youre free and ill check them out tomorrow
Nicholas Flamy
Nicholas Flamy5mo ago
never good to mess with important data late at night.
Gonza
GonzaOP5mo ago
agreed, but since i was getting your help i didnt want to let this opportunity go i depend on help cause my knowledge doesnt get me anywhere 😅 Let me know when you're available @Nicholas Flamy
Zack
Zack5mo ago
I appreciate that you're concerned with what's happened, but if you aren't able to recover from a failure like this on your own, you probably shouldn't be running Immich. If you don't understand how to take backups and restore them, you're going to end up losing all your data What happened to your SD card could happen to your hard drive too, whether it's hardware failure or otherwise
Gonza
GonzaOP5mo ago
Hi Zack, I understand your vision on this situation. I like to improve and continue learning on things I like over time, this is one of them in particular, personal projects related to technology. All the important data I have on my hard drive is backed up on another hard drive I have, so realistically if my main hard drive dies, I'll always keep my photos and videos safe. The only part I do not have secured is my OS and all Immich settings, and by having these situations I continue learning about how Immich and many other programs work, including Linux itself. I understand that by not being able to solve these issues myself, I'm asking here for help and therefore learning thanks to somebody's time, because I wouldn't be able to do so on my own. If that's the issue, I fully understand, and can move onto something easier or not reliant on people's time. Otherwise, I'd like to continue updating my immich and overcoming problems like this one, knowing that the worst case scenario, my photos and videos are safe on another hdd and having the possibility to migrate them to another easier platform.
Zack
Zack5mo ago
That's fair enough. At least you have backups of the main data. For future, don't store the database on an SD card and ensure you have backups of it. With regards to seeking help, that is fine, that's what these threads are ultimately for. All I would say is don't go pinging people who have been helping you so far. They will see alerts from the messages in the thread if they are still subscribed to the thread anyway, but if they've chosen to stop assisting a ping will pull them right back in 🙂 In this case, if you didn't have the database stored on the hard drives, and don't have database backups, there's not much that can be done. You will realistically have to setup Immich again with new users and re-import the pictures from the hard drive
Gonza
GonzaOP5mo ago
Great! My intention wasn't to pull anyone in, just to let them know my evolution on the issue. This noted, I won't continue pinging anyone😅 Ill proceed to create a folder on the hdd for the database and import all the photos manually. One last detail, I do have a .img file of the SD that corrupted with all the data in it before I wiped it clean. Is there any way of retrieving the old DB from it?
Nicholas Flamy
Nicholas Flamy5mo ago
Maybe. You might be able to mount it. Look up how to mount .img linux
Gonza
GonzaOP5mo ago
by the way, I'd like to thank Zeus, Nicholas Flamy and Zack on your time and help with this case (not pinging)😁
Nicholas Flamy
Nicholas Flamy5mo ago
Make sure you do this. If the HDD is ONLY for immich Then
Gonza
GonzaOP5mo ago
It is only for immich
Nicholas Flamy
Nicholas Flamy5mo ago
Just make immichStorage and immichDB Otherwise, make a dir immich And make dirs inside of it
Gonza
GonzaOP5mo ago
It is just for immich, so i can create a db dir and an upload dir. and move all the folders it has to the upload dir. Correct? and point the db to the db dir if im understanding correctly
Nicholas Flamy
Nicholas Flamy5mo ago
Yes. But for the upload dir Call it something like data or storage because otherwise you'll get it mixed up with the upload dir inside of it.
Gonza
GonzaOP5mo ago
i named it UPLOAD in all caps to not get it mixed maybe i will get it mixed up anyway
Nicholas Flamy
Nicholas Flamy5mo ago
Let me know how you plan to move the folders into it.
Gonza
GonzaOP5mo ago
just by sudo mv upload /UPLOAD/upload im trying to mount the .img file and see if i can recover the db from there
Nicholas Flamy
Nicholas Flamy5mo ago
Alright.
Gonza
GonzaOP5mo ago
not the most efficient way probably
Nicholas Flamy
Nicholas Flamy5mo ago
Make sure you move the other dirs the same way. This is probably the best way.
Gonza
GonzaOP5mo ago
done, yep all dirs moved
Nicholas Flamy
Nicholas Flamy5mo ago
Since theoretically, it won't read, then rewrite the data, instead just change the pointers.
Gonza
GonzaOP5mo ago
i have a question
Gonza
GonzaOP5mo ago
on my hdd i have /UPLOAD (dir where i moved all the folders) and inside/UPLOAD there's the upload dir (the one immich creates on the .env i should point the upload to /UPLOAD, or to /UPLOAD/upload now i see why the UPLOAD name wasnt the brightest idea
Nicholas Flamy
Nicholas Flamy5mo ago
Yeah Point to /UPLOAD
Gonza
GonzaOP5mo ago
good, thx all set. Im flashing the .img to an sd card, and then will proceed to mount it to my existing OS. Where can i look for the database? is there a default place where it creates ? or just get my explorer goggles on
Nicholas Flamy
Nicholas Flamy5mo ago
Do you have the original docker compose file and env file?
Gonza
GonzaOP5mo ago
i have the original docker compose not the .env but i had it all installed in portainer, which was running on the OS, so maybe the stack.env file is in the portainer files?
Nicholas Flamy
Nicholas Flamy5mo ago
Idk portainer so I can't answer that.
Gonza
GonzaOP5mo ago
okay ill look into it if no luck, how should i import all the photos and videos into a new immich install?
Nicholas Flamy
Nicholas Flamy5mo ago
This will probably be the way to go. If you can't find the env and from that where the db is, then this.
Gonza
GonzaOP5mo ago
i assume there is no way of making the fresh install just read the info from the already existing folders okay, does this look like the db ?
Gonza
GonzaOP5mo ago
No description
Zack
Zack5mo ago
Nope, that is typesense which is very old 😛
Gonza
GonzaOP5mo ago
😅 ill keep looking
Nicholas Flamy
Nicholas Flamy5mo ago
My db folder.
No description
Gonza
GonzaOP5mo ago
great, thx! no luck searching for db. im looking at instructions on immich cli but im not familiar with the api part. when configuring it , requires to modify the api key. What am I supposed to set there? and another observation, i created a fresh docker with immich, yet the DB_DATA location where database is supposed to be saved, hasnt populated yet no folders or files in the dir
Nicholas Flamy
Nicholas Flamy5mo ago
Where did you get this file? Are you on a modded Discord client? If so, your message to file plugin is broken.
Nicholas Flamy
Nicholas Flamy5mo ago
No description
Nicholas Flamy
Nicholas Flamy5mo ago
(I tried to view it on Android as well.) One thing I noticed is that your docker compose has postgres hard set as a docker volume rather than a mount.
Gonza
GonzaOP5mo ago
I normally save my docker-compose.yml file from the last update and i just copied and pasted it to discord from the txt file on my windows pc. I think i just copied and pasted and messed up the format i believe in portainer, postgres is a volume itself. But dont take my word on it, not too sure
Nicholas Flamy
Nicholas Flamy5mo ago
That's where your db went.
Gonza
GonzaOP5mo ago
this is the text file i tried to send im lost rn
Nicholas Flamy
Nicholas Flamy5mo ago
That came clean.
Gonza
GonzaOP5mo ago
No description
Gonza
GonzaOP5mo ago
this is where my db was? i have the whole .img of the SD where it all was running.
Nicholas Flamy
Nicholas Flamy5mo ago
Yes, the pgdata docker volume. I don't know where the actual data for docker volumes is stored but you would have to look there.
Gonza
GonzaOP5mo ago
Ill keep trying tomorrow. Thanks again for all the help. maybe i can find the data on the .img of my old os Okay, i think i got it i found the old database in a docker volume let me add a photo of what that looks taking this into account its taking long to copy the whole folder not sure if thats normal
Nicholas Flamy
Nicholas Flamy5mo ago
Let me know before you try to start immich again.
Gonza
GonzaOP5mo ago
yep, it got stuck copying the folder let me show you
Gonza
GonzaOP5mo ago
No description
Gonza
GonzaOP5mo ago
should be the immich_pgdata right?
Nicholas Flamy
Nicholas Flamy5mo ago
I think so
Gonza
GonzaOP5mo ago
damn. cant believe it still nothing tho 😂
Nicholas Flamy
Nicholas Flamy5mo ago
You'll need to update your docker compose to match the official docker compose: https://github.com/immich-app/immich/blob/main/docker/docker-compose.yml#L66
Gonza
GonzaOP5mo ago
if i get the database i think im going to cry its taking forever to copy the folder i think my raspberry is overloading to copy this folder there's no way i can do this from a windows machine, right?
Nicholas Flamy
Nicholas Flamy5mo ago
You would have to look it up.
Gonza
GonzaOP5mo ago
its on it 500 files out of 1600 ill just wait
Nicholas Flamy
Nicholas Flamy5mo ago
It's still progressing right?
Gonza
GonzaOP5mo ago
yep
Gonza
GonzaOP5mo ago
No description
Nicholas Flamy
Nicholas Flamy5mo ago
Then just leave it and do something else and come back to it.
Gonza
GonzaOP5mo ago
its on it, yep
Nicholas Flamy
Nicholas Flamy5mo ago
Probably the best bet.
Gonza
GonzaOP5mo ago
will do!
Gonza
GonzaOP5mo ago
Gonza
GonzaOP5mo ago
doesnt look good could this be because of data corruption?
Nicholas Flamy
Nicholas Flamy5mo ago
potentially, yes
Gonza
GonzaOP5mo ago
let me try from windows how much bad luck do i need to have exactly my db files corrupted okay i have it i have the whole folder
Gonza
GonzaOP5mo ago
No description
Gonza
GonzaOP5mo ago
does this look correct?
Nicholas Flamy
Nicholas Flamy5mo ago
Yeah, my main worry would be symlinks. I don't know if there are any in the db folders though
Gonza
GonzaOP5mo ago
dont know what symlinks are
Nicholas Flamy
Nicholas Flamy5mo ago
similar to windows shortcuts
Gonza
GonzaOP5mo ago
okay um i could just copy the whole docker folder right?
Zeus
Zeus5mo ago
I don’t think Postgres uses symlinks at all
Nicholas Flamy
Nicholas Flamy5mo ago
explain...
Gonza
GonzaOP5mo ago
well, since i now have access to the whole broken OS i could copy the entire docker folder containing volumes, and everything else
Gonza
GonzaOP5mo ago
No description
Nicholas Flamy
Nicholas Flamy5mo ago
Sure, copy it to your Windows PC in case there's something you need later on. But Don't copy it to your RPi4 and overwrite docker.
Gonza
GonzaOP5mo ago
okay
Nicholas Flamy
Nicholas Flamy5mo ago
If there's corrupt files, that won't be fun.
Gonza
GonzaOP5mo ago
right okay so now i have the database folder, how should i proceed?
Nicholas Flamy
Nicholas Flamy5mo ago
copy it to where you specified in your .env But before you start immich, maybe download the latest official docker compose file and then if there's anything you need to change (so HW Accel Transcoding or Machine Learning)
Gonza
GonzaOP5mo ago
okay between testing and all, i deleted my SD card so im preparing a fresh install of rasp OS
Nicholas Flamy
Nicholas Flamy5mo ago
Do not specify storage in the docker compose, the default compose file specifies environment variables as storage locations and those can be set in the .env
Gonza
GonzaOP5mo ago
all fresh and new do you recommend docker or portainer for immich installation setup?
Nicholas Flamy
Nicholas Flamy5mo ago
docker
Gonza
GonzaOP5mo ago
okay
Gonza
GonzaOP5mo ago
yep, just give me a sec. still installing os on SD okay do i need hwaccel and ml?
Nicholas Flamy
Nicholas Flamy5mo ago
You didn't have it before so dw about it.
Gonza
GonzaOP5mo ago
okay got the docker compose and the -env file thats wher im at
Nicholas Flamy
Nicholas Flamy5mo ago
nvm immich currently has no RPi HW Transcode support anyway. So no.
Gonza
GonzaOP5mo ago
okay do i follow steps normally? populate .env as per usual
Nicholas Flamy
Nicholas Flamy5mo ago
These values should mostly be fine. Except the storage but we've already been over that yes
Gonza
GonzaOP5mo ago
but wait now i have two folders in my hdd DB_DATA and UPLOAD so i should point the upload location to UPLOAD and data location to DB_DATA correct?
Nicholas Flamy
Nicholas Flamy5mo ago
yes
Gonza
GonzaOP5mo ago
okay done i need to transfer the pgdata folder i have on my windows pc to the DB_DATA on the hdd right?
Nicholas Flamy
Nicholas Flamy5mo ago
yes, before starting immich
Gonza
GonzaOP5mo ago
okay scp? let me look up the command to do it right via ssh on it, transferring the folder (2gb)
Nicholas Flamy
Nicholas Flamy5mo ago
umm the db folder is 2gb?
Gonza
GonzaOP5mo ago
yep done 1.13gb mb
Nicholas Flamy
Nicholas Flamy5mo ago
gb mb? oh "my bad"
Gonza
GonzaOP5mo ago
yea sry now just boot immich docker? btw, should i be pointing on .env to my DB_DATA folder on hdd or to immich_pgdata inside DB_DATA folder?
Nicholas Flamy
Nicholas Flamy5mo ago
Yeah, I was going to ask about that. It seems you copied immich_pgdata into DB_DATA rather than the contents of immich_pgdata into DB_DATA so, do the latter
Gonza
GonzaOP5mo ago
okay ill change -env
Nicholas Flamy
Nicholas Flamy5mo ago
also, you may need to change permissions
Gonza
GonzaOP5mo ago
and with the UPLOAD?
Gonza
GonzaOP5mo ago
No description
Nicholas Flamy
Nicholas Flamy5mo ago
to the directory containing library, thumbs etc yes
Gonza
GonzaOP5mo ago
No description
Gonza
GonzaOP5mo ago
.env looking like this just to confirm
Nicholas Flamy
Nicholas Flamy5mo ago
seems fine
Gonza
GonzaOP5mo ago
cool now just standard docker start? or docker compose up
Nicholas Flamy
Nicholas Flamy5mo ago
what it tells you here
Gonza
GonzaOP5mo ago
yea, just wanted to confirm no other steps before just going ahead
Gonza
GonzaOP5mo ago
okay forgot to install docker cool
Gonza
GonzaOP5mo ago
on it, yep
Nicholas Flamy
Nicholas Flamy5mo ago
For Debian-based I usually do everything visible in this screenshot.
Nicholas Flamy
Nicholas Flamy5mo ago
No description
Gonza
GonzaOP5mo ago
i assume raspberry pi os is debian based
Nicholas Flamy
Nicholas Flamy5mo ago
pretty sure, yes
Gonza
GonzaOP5mo ago
never installed docker desktop where it says add the repository to apt sources, where am i supposed to run that? im doing line by line on command terminal maybe im noobing this out
Nicholas Flamy
Nicholas Flamy5mo ago
docker or docker desktop?
Gonza
GonzaOP5mo ago
docker desktop
Nicholas Flamy
Nicholas Flamy5mo ago
docker is the container system, docker desktop is the proprietary GUI app that sets everything up for you.
Gonza
GonzaOP5mo ago
yep, never did desktop had portainer, which i imagine is similar
Nicholas Flamy
Nicholas Flamy5mo ago
Ehh, not so much.
Gonza
GonzaOP5mo ago
im lost in the instructions right now
Nicholas Flamy
Nicholas Flamy5mo ago
Docker desktop doesn't have much config editing. (I think some is paywalled.)
Gonza
GonzaOP5mo ago
currently following your screenshot on the web im running line by line on the terminal am i correct? not sure if i have to make a dir for all this or a file
Nicholas Flamy
Nicholas Flamy5mo ago
Line by line:
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
At once:
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian \
$(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian \
$(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
Last:
sudo apt-get update
sudo apt-get update
Gonza
GonzaOP5mo ago
sry im making you go through all this 😅 all done now
Gonza
GonzaOP5mo ago
now i can just docker compose up -d right? done ummm should i reboot? -bash: docker: command not found
Nicholas Flamy
Nicholas Flamy5mo ago
are you sshed?
Gonza
GonzaOP5mo ago
yep ia m
Nicholas Flamy
Nicholas Flamy5mo ago
close it and reopen it
Gonza
GonzaOP5mo ago
nope stil not found did i miss something?
Nicholas Flamy
Nicholas Flamy5mo ago
This?
No description
Gonza
GonzaOP5mo ago
correct head is exploding 4am here and i feel like you must be hating me rn i would if i were you, so all good
Nicholas Flamy
Nicholas Flamy5mo ago
Dude, no one is hating anyone here.
Gonza
GonzaOP5mo ago
okay just letting you know i really appreciate all your help
Nicholas Flamy
Nicholas Flamy5mo ago
If I was hating I wouldn't still be helping lol. Thank you
Gonza
GonzaOP5mo ago
okay now up docker up it should load all my old data directly? or should i set up something in order for it to work?
Nicholas Flamy
Nicholas Flamy5mo ago
We hope. 🤞
Gonza
GonzaOP5mo ago
okay still pulling images one sec i should now just try log in as if nothing happened?
Nicholas Flamy
Nicholas Flamy5mo ago
yes
Gonza
GonzaOP5mo ago
moment of truth
Nicholas Flamy
Nicholas Flamy5mo ago
🤞
Gonza
GonzaOP5mo ago
after all this im going to need to study hard how to make backups on my hdd and on the ssd with raspOS
Nicholas Flamy
Nicholas Flamy5mo ago
Yeah.
Gonza
GonzaOP5mo ago
okay, all started nothing on port 8863 yet 2283*
Nicholas Flamy
Nicholas Flamy5mo ago
No description
Nicholas Flamy
Nicholas Flamy5mo ago
This is just DB It backs up to a folder in UPLOAD
Gonza
GonzaOP5mo ago
ummmm one sec nothing up on my ip:2283
Gonza
GonzaOP5mo ago
No description
Nicholas Flamy
Nicholas Flamy5mo ago
try docker compose down then docker compose up so we can easily see logs
Gonza
GonzaOP5mo ago
ummm no good news here how can i show you everything?
Gonza
GonzaOP5mo ago
No description
Gonza
GonzaOP5mo ago
is this readable? it says dir already exists and is not empty
Gonza
GonzaOP5mo ago
Gonza
GonzaOP5mo ago
there, much better
Nicholas Flamy
Nicholas Flamy5mo ago
hit Ctrl+C one time to stop the container
Gonza
GonzaOP5mo ago
done maybe we should continue tomorrow, i gotta get some sleep at some point
Nicholas Flamy
Nicholas Flamy5mo ago
yep
Gonza
GonzaOP5mo ago
any ideas on how to get this on track though? just to go to bed with or without hope 🤣
Nicholas Flamy
Nicholas Flamy5mo ago
Not sure, the important info is here:
immich_postgres | The files belonging to this database system will be owned by user "postgres".
immich_postgres | This user must also own the server process.
immich_postgres |
immich_postgres | The database cluster will be initialized with locale "en_US.utf8".
immich_postgres | The default database encoding has accordingly been set to "UTF8".
immich_postgres | The default text search configuration will be set to "english".
immich_postgres |
immich_postgres | Data page checksums are enabled.
immich_postgres |
immich_postgres | initdb: error: directory "/var/lib/postgresql/data" exists but is not empty
immich_postgres | If you want to create a new database system, either remove or empty
immich_postgres | the directory "/var/lib/postgresql/data" or run initdb
immich_postgres | with an argument other than "/var/lib/postgresql/data".
immich_postgres exited with code 0
immich_postgres | The files belonging to this database system will be owned by user "postgres".
immich_postgres | This user must also own the server process.
immich_postgres |
immich_postgres | The database cluster will be initialized with locale "en_US.utf8".
immich_postgres | The default database encoding has accordingly been set to "UTF8".
immich_postgres | The default text search configuration will be set to "english".
immich_postgres |
immich_postgres | Data page checksums are enabled.
immich_postgres |
immich_postgres | initdb: error: directory "/var/lib/postgresql/data" exists but is not empty
immich_postgres | If you want to create a new database system, either remove or empty
immich_postgres | the directory "/var/lib/postgresql/data" or run initdb
immich_postgres | with an argument other than "/var/lib/postgresql/data".
immich_postgres exited with code 0
Go to bed with hope. It might just be permissions.
Gonza
GonzaOP5mo ago
good ill text here tomorrow when im done working thanks a ton again nicholas, for real. my appreciation is really intense towards you rn
Nicholas Flamy
Nicholas Flamy5mo ago
Try chown -R 999 /mnt/Skyhawk/DB_DATA to set postgres as the owner of the db. Hopefully fixing permissions will make it work. If I got the dir wrong let me know before you run it (it should be the path set as the db data location in the .env)
Gonza
GonzaOP5mo ago
hi there! the DB_DATA is the folder where i copied immich_pgdata into
Nicholas Flamy
Nicholas Flamy5mo ago
chown -R 999 /mnt/Skyhawk/DB_DATA/immich_pgdata
Gonza
GonzaOP5mo ago
okay im booting up the system one second and ill update should be with sudo, right? says permission denied okay done should I try now to docker compose up ?
Nicholas Flamy
Nicholas Flamy5mo ago
yes no -d the first time just so we can watch it
Gonza
GonzaOP5mo ago
okay, same error as before says directory already exists let me paste everything here so you can have a propper look
Gonza
GonzaOP5mo ago
Gonza
GonzaOP5mo ago
wait
Nicholas Flamy
Nicholas Flamy5mo ago
It kinda looks like it never stopped.
Gonza
GonzaOP5mo ago
i just booted up the raspberry pi it was off im getting more text hold up
Nicholas Flamy
Nicholas Flamy5mo ago
okay
Gonza
GonzaOP5mo ago
Nicholas Flamy
Nicholas Flamy5mo ago
sudo chown -R postgres /mnt/Skyhawk/DB_DATA/immich_pgdata try that
Gonza
GonzaOP5mo ago
alright ctrl C
Nicholas Flamy
Nicholas Flamy5mo ago
see if it makes any difference
Gonza
GonzaOP5mo ago
onece then taht
Nicholas Flamy
Nicholas Flamy5mo ago
👍
Gonza
GonzaOP5mo ago
chown: invalid user: ‘postgres’
Nicholas Flamy
Nicholas Flamy5mo ago
huh okay
Gonza
GonzaOP5mo ago
immich_postgres?
Nicholas Flamy
Nicholas Flamy5mo ago
send screenshots of ls -la /mnt/Skyhawk/DB_DATA and ls -la /mnt/Skyhawk/DB_DATA/immich_pgdata
Gonza
GonzaOP5mo ago
No description
Gonza
GonzaOP5mo ago
No description
Nicholas Flamy
Nicholas Flamy5mo ago
sudo
Gonza
GonzaOP5mo ago
No description
Nicholas Flamy
Nicholas Flamy5mo ago
that's why
sudo ls -la /mnt/apps/data/immich-compose-test/docker/postgres
total 90
drwx------ 21 netdata apps 28 Dec 15 14:09 .
drwxrwxrwx 4 apps apps 8 Dec 6 11:32 ..
-rwxrwxrwx 1 netdata apps 3 Nov 7 07:05 PG_VERSION
drwxrwxrwx 7 netdata apps 7 Dec 15 14:06 base
-rw------- 1 netdata docker 44 Dec 15 14:06 current_logfiles
drwxrwxrwx 2 netdata apps 60 Dec 15 14:06 global
drwxrwxrwx 2 netdata apps 19 Dec 15 14:06 log
drwxrwxrwx 2 netdata apps 2 Nov 7 07:05 pg_commit_ts
drwxrwxrwx 2 netdata apps 2 Nov 7 07:05 pg_dynshmem
-rwxrwxrwx 1 netdata apps 4821 Nov 7 07:05 pg_hba.conf
-rwxrwxrwx 1 netdata apps 1636 Nov 7 07:05 pg_ident.conf
drwxrwxrwx 4 netdata apps 5 Dec 15 14:09 pg_logical
drwxrwxrwx 4 netdata apps 4 Nov 7 07:05 pg_multixact
drwxrwxrwx 2 netdata apps 2 Nov 7 07:05 pg_notify
drwxrwxrwx 2 netdata apps 2 Nov 7 07:05 pg_replslot
drwxrwxrwx 2 netdata apps 2 Nov 7 07:05 pg_serial
drwxrwxrwx 2 netdata apps 2 Nov 7 07:05 pg_snapshots
drwxrwxrwx 2 netdata apps 5 Dec 15 14:09 pg_stat
drwxrwxrwx 2 netdata apps 2 Dec 15 14:09 pg_stat_tmp
drwxrwxrwx 2 netdata apps 3 Nov 7 07:05 pg_subtrans
drwxrwxrwx 2 netdata apps 2 Nov 7 07:05 pg_tblspc
drwxrwxrwx 2 netdata apps 2 Nov 7 07:05 pg_twophase
drwxrwxrwx 4 netdata apps 5 Nov 7 07:05 pg_vectors
drwxrwxrwx 3 netdata apps 50 Nov 7 07:15 pg_wal
drwxrwxrwx 2 netdata apps 3 Nov 7 07:05 pg_xact
-rwxrwxrwx 1 netdata apps 88 Nov 7 07:05 postgresql.auto.conf
-rwxrwxrwx 1 netdata apps 28835 Nov 7 07:05 postgresql.conf
-rwxrwxrwx 1 netdata apps 229 Dec 15 14:06 postmaster.opts
sudo ls -la /mnt/apps/data/immich-compose-test/docker/postgres
total 90
drwx------ 21 netdata apps 28 Dec 15 14:09 .
drwxrwxrwx 4 apps apps 8 Dec 6 11:32 ..
-rwxrwxrwx 1 netdata apps 3 Nov 7 07:05 PG_VERSION
drwxrwxrwx 7 netdata apps 7 Dec 15 14:06 base
-rw------- 1 netdata docker 44 Dec 15 14:06 current_logfiles
drwxrwxrwx 2 netdata apps 60 Dec 15 14:06 global
drwxrwxrwx 2 netdata apps 19 Dec 15 14:06 log
drwxrwxrwx 2 netdata apps 2 Nov 7 07:05 pg_commit_ts
drwxrwxrwx 2 netdata apps 2 Nov 7 07:05 pg_dynshmem
-rwxrwxrwx 1 netdata apps 4821 Nov 7 07:05 pg_hba.conf
-rwxrwxrwx 1 netdata apps 1636 Nov 7 07:05 pg_ident.conf
drwxrwxrwx 4 netdata apps 5 Dec 15 14:09 pg_logical
drwxrwxrwx 4 netdata apps 4 Nov 7 07:05 pg_multixact
drwxrwxrwx 2 netdata apps 2 Nov 7 07:05 pg_notify
drwxrwxrwx 2 netdata apps 2 Nov 7 07:05 pg_replslot
drwxrwxrwx 2 netdata apps 2 Nov 7 07:05 pg_serial
drwxrwxrwx 2 netdata apps 2 Nov 7 07:05 pg_snapshots
drwxrwxrwx 2 netdata apps 5 Dec 15 14:09 pg_stat
drwxrwxrwx 2 netdata apps 2 Dec 15 14:09 pg_stat_tmp
drwxrwxrwx 2 netdata apps 3 Nov 7 07:05 pg_subtrans
drwxrwxrwx 2 netdata apps 2 Nov 7 07:05 pg_tblspc
drwxrwxrwx 2 netdata apps 2 Nov 7 07:05 pg_twophase
drwxrwxrwx 4 netdata apps 5 Nov 7 07:05 pg_vectors
drwxrwxrwx 3 netdata apps 50 Nov 7 07:15 pg_wal
drwxrwxrwx 2 netdata apps 3 Nov 7 07:05 pg_xact
-rwxrwxrwx 1 netdata apps 88 Nov 7 07:05 postgresql.auto.conf
-rwxrwxrwx 1 netdata apps 28835 Nov 7 07:05 postgresql.conf
-rwxrwxrwx 1 netdata apps 229 Dec 15 14:06 postmaster.opts
(My env points to /mnt/apps/data/immich-compose-test/docker/postgres)
Gonza
GonzaOP5mo ago
i see your permissions are full and mine arent
Nicholas Flamy
Nicholas Flamy5mo ago
sudo ls -la /mnt/Skyhawk/DB_DATA/immich_pgdata/_data Don't do that Mine are only like that because it's a test instance.
Gonza
GonzaOP5mo ago
No description
Nicholas Flamy
Nicholas Flamy5mo ago
It's bad practice. yep so your env needs to point to that but
Gonza
GonzaOP5mo ago
i wont be doing anything without your approval at this point, dw 😂 so it needs to point to _data folder if im understanding
Gonza
GonzaOP5mo ago
No description
Gonza
GonzaOP5mo ago
like here, right?
Nicholas Flamy
Nicholas Flamy5mo ago
hold on
Gonza
GonzaOP5mo ago
okay
Nicholas Flamy
Nicholas Flamy5mo ago
I think a move would be better So that there's not pointless subdirs (that will cause confusion in the future)
Gonza
GonzaOP5mo ago
right so just mv everything to /DB_DATA
Nicholas Flamy
Nicholas Flamy5mo ago
wait I seem to remember something about mv caring about whether you put a slash at the end just did a test
Gonza
GonzaOP5mo ago
okay
Nicholas Flamy
Nicholas Flamy5mo ago
I would personally go with a different dir name All caps is usually for envs
Gonza
GonzaOP5mo ago
im all ears okay we can change both
Nicholas Flamy
Nicholas Flamy5mo ago
also, if you want to use the same one, you'll have to rename the current and then mv the files.
Gonza
GonzaOP5mo ago
i dont mind changing at all we could do something like database and files? for the DB and upload
Nicholas Flamy
Nicholas Flamy5mo ago
So I'd say immich_db or just pgdata
Gonza
GonzaOP5mo ago
okay
Nicholas Flamy
Nicholas Flamy5mo ago
for UPLOAD it could be immich_storage or something
Gonza
GonzaOP5mo ago
immich_db works for me cool now, new dirs?
Nicholas Flamy
Nicholas Flamy5mo ago
so, sudo mv /mnt/Skyhawk/DB_DATA/immich_pgdata/_data /mnt/Skyhawk/immich_db do not mkdir
Gonza
GonzaOP5mo ago
okay
Nicholas Flamy
Nicholas Flamy5mo ago
That way it will rename _data rather than put it inside a dir.
Gonza
GonzaOP5mo ago
done and same for the upload? sudo mv /mnt/Skyhawk/UPLOAD /mnt/Skyhawk/immich_storage
Nicholas Flamy
Nicholas Flamy5mo ago
sudo mv /mnt/Skyhawk/UPLOAD /mnt/Skyhawk/immich_storage make sure to update env to match
Gonza
GonzaOP5mo ago
still have the DB_DATA should i rm?
Nicholas Flamy
Nicholas Flamy5mo ago
not yet
Gonza
GonzaOP5mo ago
on it
Nicholas Flamy
Nicholas Flamy5mo ago
In general, don't run any delete commands until after something works. Especially while moving important data Also, you still have a copy of the db on your Windows PC right?
Gonza
GonzaOP5mo ago
i do
Nicholas Flamy
Nicholas Flamy5mo ago
Good
Gonza
GonzaOP5mo ago
No description
Gonza
GonzaOP5mo ago
good?
Nicholas Flamy
Nicholas Flamy5mo ago
Looking good. 👍
Gonza
GonzaOP5mo ago
okay done change permissions on any folder? or just docker compose up?
Nicholas Flamy
Nicholas Flamy5mo ago
try compose up
Gonza
GonzaOP5mo ago
looking good
Nicholas Flamy
Nicholas Flamy5mo ago
🤞
Gonza
GonzaOP5mo ago
moment of truth no errors for the moment
Nicholas Flamy
Nicholas Flamy5mo ago
Did you try the WebUI?
Gonza
GonzaOP5mo ago
looks like it identified an existing db not yet, im giving it time to start up just did working
Nicholas Flamy
Nicholas Flamy5mo ago
Oh also, how big is this HDD? Woo
Gonza
GonzaOP5mo ago
3tb i can not believe this its working
Nicholas Flamy
Nicholas Flamy5mo ago
Okay, now stop immich. Ctrl+C then docker compose up -d
Gonza
GonzaOP5mo ago
Nicholas Flamy
Nicholas Flamy5mo ago
That way it'll keep going.
Gonza
GonzaOP5mo ago
you are a genius everything is exactly the same albums shared map photos thumbnails all intact
Nicholas Flamy
Nicholas Flamy5mo ago
Yay!
Gonza
GonzaOP5mo ago
this is mad work on your behalf dude fr
Nicholas Flamy
Nicholas Flamy5mo ago
Now, back up this stuff.
Gonza
GonzaOP5mo ago
need to read on how to there was an article on it on immich site right?
Kiefff
Kiefff5mo ago
good shit, glad you got up and running again
Gonza
GonzaOP5mo ago
this was all nicholas. Absolute master
Gonza
GonzaOP5mo ago
creds to him
Nicholas Flamy
Nicholas Flamy5mo ago
https://immich.app/docs/guides/template-backup-script One way to do it You're welcome for the help.
Gonza
GonzaOP5mo ago
No description
Gonza
GonzaOP5mo ago
ik its in spanish but that should do it once a week, and keep the last 4 weeks, right?
Nicholas Flamy
Nicholas Flamy5mo ago
Yeah.
Gonza
GonzaOP5mo ago
okay should i also do the script?
Nicholas Flamy
Nicholas Flamy5mo ago
If you're willing to restore the db from a backup, the only folder you need to backup is /mnt/Skyhawk/immich_storage I think that for saving the backups. If you have another means of backing up that's fine. This makes a copy of the DB and puts it in /mnt/Skyhawk/immich_storage/backups
Gonza
GonzaOP5mo ago
alright, so I have another hdd (2tb) which i bought with the intention of every month or so, have the main (3tb) disk cloned to the (2tb) one in case the main one dies
Nicholas Flamy
Nicholas Flamy5mo ago
Also, the way it copies is safe, regular copying might be unsafe because there could be an in progress transaction (I'm not sure if this is true, just my guess).
Gonza
GonzaOP5mo ago
with this method of backing up, i could retrieve directly from the 2tb disk both the database and the content
Nicholas Flamy
Nicholas Flamy5mo ago
So how do you do it? A script?
Gonza
GonzaOP5mo ago
that's my next step i tried rsync, but my raspberry just unmounts both disks when i run it maybe it overloads or something i have to figure out a way of cloning the main disk to the secondary rsnapshot i think it was called
Nicholas Flamy
Nicholas Flamy5mo ago
Check #immich or #off-topic. Those are the places to ask about backing up/cloning methods.
Gonza
GonzaOP5mo ago
great and one last thing should i install docker desktop?
Nicholas Flamy
Nicholas Flamy5mo ago
Up to you, from what I've seen it's bad for managing docker compose.
Gonza
GonzaOP5mo ago
its bad?
Nicholas Flamy
Nicholas Flamy5mo ago
Or rather, I haven't found a GUI button for docker compose up and docker compose down
Gonza
GonzaOP5mo ago
thing is, i always updated immich on portainer, so now without any gui i guess ill have to learn i stopped the stack, then updated the .yml and back up
Nicholas Flamy
Nicholas Flamy5mo ago
It's mainly status and container management but not seemingly focused on compose.
Gonza
GonzaOP5mo ago
okay ill try without gui
Gonza
GonzaOP5mo ago
so now, for a standard version update, i just docker compose down and up right? okay answered with your link
Nicholas Flamy
Nicholas Flamy5mo ago
I don't think so. Alright.
Gonza
GonzaOP5mo ago
docker compose pull && docker compose up -d thats the command for an updte
Nicholas Flamy
Nicholas Flamy5mo ago
docker compose pull is the update part. this just runs the start command after running the update command
Gonza
GonzaOP5mo ago
yep yep i just copied and pasted alright well after all your help, if i can be of any for you let me know will now focus on keeping a clean backup
Nicholas Flamy
Nicholas Flamy5mo ago
Make a backup right now, even if it's not that clean.
Gonza
GonzaOP5mo ago
of the whole hdd?
Nicholas Flamy
Nicholas Flamy5mo ago
But yes, do that. maybe If you can then yes
Gonza
GonzaOP5mo ago
im not sure how to proceed id have to read about it
Nicholas Flamy
Nicholas Flamy5mo ago
Well, you could just stop immich, copy the db and storage folders to another drive, then work on a real backup.
Gonza
GonzaOP5mo ago
hmm
Nicholas Flamy
Nicholas Flamy5mo ago
Just in case something goes wrong between now and having a real backup.
Gonza
GonzaOP5mo ago
you mean with command like cp?
Nicholas Flamy
Nicholas Flamy5mo ago
yeah Also, for recursive copying, I usually do cp -a (So copy a folder and all of it's contents (as well as stuff like permissions and more because of -a rather than -r))
Gonza
GonzaOP5mo ago
okay, so basic command would be sudo cp /mnt/Skyhawk /mnt/2hdd -a for example?
Nicholas Flamy
Nicholas Flamy5mo ago
put the -a after cp not after the paths (don't know if it makes a different for cp but in general that's the case)
Gonza
GonzaOP5mo ago
okay, just insterted the backup hdd, and it does something weird it unmounted the main drive previously sda and now i have this
Gonza
GonzaOP5mo ago
No description
Gonza
GonzaOP5mo ago
no sda, main disk unmounted itself and now i have sdb and sdc
Nicholas Flamy
Nicholas Flamy5mo ago
stop immich if you can
Gonza
GonzaOP5mo ago
i did
Nicholas Flamy
Nicholas Flamy5mo ago
Is this all over USB on the RPi4?
Gonza
GonzaOP5mo ago
yes
Nicholas Flamy
Nicholas Flamy5mo ago
Are these portable external drives? or proper HDDs in an adapter?
Gonza
GonzaOP5mo ago
so one of them is a Skyhawk on a powered sata to usb cable (main disk) 3TB and the other one is an external hdd WDPassport (backup disk) 2TB im going to format the backup drive to ext4 so i can use it on the raspberry I also have an SSD (240gb) that i bought to install the OS in it, instead of having it on the SD card, due to reliability issues, yet when i install everything on the ssd and set the rasp to boot on usb it just wont boot. Thats the next step after having the backup situation solved😂 Ive been trying with everything closed, but just as always, when i run the cp command, both disks unmount and appear with different letters in lsblk (sda->sdb) ill keep looking into it tomorrow. thanks again for all the help nicholas!
Mraedis
Mraedis5mo ago
I didn't read all of this but when setting permissions, be sure to set the group as well so chown -R 999:999 blablabla
Zack
Zack5mo ago
In this case they didn't actually have any group permissions allowed so it doesn't really matter 😛
Nicholas Flamy
Nicholas Flamy5mo ago
It doesn't need group to work though. Also, the problem was some subdirs It's all sorted though!
Xygon
Xygon5mo ago
Just reading some pieces here, but if sda to sdb switch is part of your problems, it’s a good reminder that mounts should use the /dev/by-uuid (or whatever it is) and not the block device handle
Nicholas Flamy
Nicholas Flamy5mo ago
@Gonza check this advice.
Gonza
GonzaOP5mo ago
Unfortunately I wont have time today to work on it, but I setup my main disk to automatically mount to /mnt/Skyhawk on /etc/fstab with its UUID. Still switches when I try to run something like a cp command on it If this isn't what you mean by mounting by uuid, let me know how to! Thx
Nicholas Flamy
Nicholas Flamy5mo ago
@Gonza did you delete messages?
Gonza
GonzaOP5mo ago
i did i figured the problem out, my hdd isnt mounting on its own. im troubleshooting
Nicholas Flamy
Nicholas Flamy5mo ago
Okay, at least send one message saying nvm to the messages I deleted. Or just say nvm without deleting.
Gonza
GonzaOP5mo ago
sry, will do next time. I thought no one read them, didnt want to grab your attention
Nicholas Flamy
Nicholas Flamy5mo ago
I got a notification about it, that's all.
Gonza
GonzaOP5mo ago
mb

Did you find this page helpful?