immich ingesting thumbs and preview files into main gallery.
hi guys, im very much a noob and tried running immich initially on my docker desktop and switched to ubuntu 24 machine and installed it and configured it yesterday. initially when the ingestion was done via the mounted paths i only saw the original pictures within the timeline. after midnight batch job run this morning i've seen only the thumbs and preview jpegs appear in the timeline. i've used a bit of copilot help to get to this point of all the buffalo and facial recognition services to run without any issues. current version of immich server is v1.137.3 , ive attached env ,compose, docker logs, docker ps a , my linux folder structure. inside photos where the subfolders of my pictures sit. cant think of anything else that ive missed . but any help is much appriciated! the server is working properly via my android phone and other windows pcs and obviously displaying thumps and preview files across all devices. i've had to manually list all the critical directories in the compose file so that the server didnt quit due to a critical error hence the very long volume list in compose. ihad to modify the default machine learning url to the following so it would communicate properly with the stack last night after days of troubleshooting via copilot which did not pick it up until i noticed it lol -http://immich_machine_learning:3003/ , if i remove any of the forced mounts like upload or encode-video then the server will simply quit and UI wont start up. please let me know if this is a simple settings i can adjust or where these additional files are being ingested and ending up on the main timeline. cheers guys!
81 Replies
:wave: Hey @stefan,
Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.
References
- Container Logs:
docker compose logs docs
- Container Status: docker ps -a docs
- Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
- Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA
Checklist
I have...
1. :blue_square: verified I'm on the latest release(note that mobile app releases may take some time).
2. :blue_square: read applicable release notes.
3. :blue_square: reviewed the FAQs for known issues.
4. :blue_square: reviewed Github for known issues.
5. :blue_square: tried accessing Immich via local ip (without a custom reverse proxy).
6. :blue_square: uploaded the relevant information (see below).
7. :blue_square: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable
(an item can be marked as "complete" by reacting with the appropriate number)
Information
In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:
- Your docker-compose.yml and .env files.
- Logs from all the containers and their status (see above).
- All the troubleshooting steps you've tried so far.
- Any recent changes you've made to Immich or your system.
- Details about your system (both software/OS and hardware).
- Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h).
- The version of the Immich server, mobile app, and other relevant pieces.
- Any other information that you think might be relevant.
Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)
If this ticket can be closed you can use the /close command, and re-open it later if needed.How did you come to this compose? It's like a wild patchwork of things that maybe work, maybe not. @stefan
very much troubleshooting via copilot whenever the server hit a critical error and exited. carried hardcoding most elements into the compose file to make sure its worked
i've seperated the core immich operational folders now away from the external library located in immich/photos , the core thumbs,encode-video etc folders are now located in immich/setup
This part is just wild honestly
Should be:
/home/ravin/immich/photos:/data
There, doneif i didnt specify the paths immich server would exit critially saying it cant find thumbs for example
That'd be because you borked things a lot and borked it even harder
Are things moved or duplicated? Because if starting the move or fresh is an option I would go that route
From your symptoms you've managed to half bake the internal move + put thumbs into the upload folder
absolutely , i am very much at the limits with coding and used a half baked AI agent, i've moved the core folders out of immich/photos and doing a full rebuild
so far no thumbs have been injected and only the full res pics are in the timeline
For future help, please don't use AI for Immich
It really has no idea what it's doing
yes clearly i'll troubleshoot via you lovely folks here
From the posts that use AI that I've seen it just applies "fixes" from different issues all over the place
So what is the current compose you're using?
yes absolutely. its made me lose days in trying to sort this out
its still the wild west but shes holding
I don't trust that thing at all 👀
youre welcome to give me a slimmed down compose mate. as i dont have it in me to write it lol
like this:
Doesn't even exist :p
i just started out with having this under compose and the server would fail on restart -- /home/ravin/immich/photos:/usr/src/app/photos
so delete the lines for disable watch
I'm working on it, sec
no stress cheers
here is the env file -# Core paths
UPLOAD_LOCATION=/usr/src/app/upload
DB_DATA_LOCATION=./postgres
TZ=Australia/Melbourne
Immich version
IMMICH_VERSION=release
Database config
DB_USERNAME=postgres
DB_PASSWORD=postgres
DB_DATABASE_NAME=immich
DB_HOST=database
DB_PORT=5432
Redis config
REDIS_HOST=redis
REDIS_PORT=6379
Machine Learning service override
MACHINE_LEARNING_URL=http://immich_machine_learning:3003
env:
This should be it
right! thanks heaps! . would this load if i run it without crashing when it tries to validate the core immich files within the respective folders. also when im creating the new external library path what would that be set to? /data
Ah there was an external library...?
If you had one it was mounted on an internal path which is obviously a bad idea
UPLOAD_LOCATION=/home/ravin/immich/setup in the env file is meant to be for the pictures or for the core immich folders with immich files? , yes although the files are at immich/photo i thought i had to mount it as an external drive
or path i should say
immich/photos are the pics and immich/setup /thumbs etc are the core immich file locations
Just to be clear, you can't upload into an external library through the immich app/web interface
If you want to use one, DO NOT use the paths /data or /usr/src/app/upload
This should be safe for instance
you can have a look at the structure and let me know what you think
the core folders have been deleted beside photos and in setup
Forgot to change /setup to /photos
Are there pictures inside library/encoded-video you wanted saved?
library I understand but encoded-video doesn't make a lot of sense 😛
I mean inside of
/home/ravin/immich/photos here ^that encoded-video is the immich config so all of them have been taken out of photos and reside in setup
add /home/ravin/immich/photos:/external under volumes?
Yes
and then the Immich GUI library mount path would be /external
got it
volumes:
- ${UPLOAD_LOCATION}:/data
-/home/ravin/immich/photos:/external
- /etc/localtime:/etc/localtime:ro
you going to be online for a bit mate? if i stop the container and modify this incase i run into any issues?
sure
cheers
we are back in business mate. now do i have to fully reindex everything with the new path? it was going through facial recognition when i stopped the stack
ive added /external and it validated and i deleted my old path
do i need to purge everthing that cached already and delete the external library first i just realised i remoed my old path and just replaced it with /external and didnt fully delete the old one hence the pics are still visible in the timeline
@Mraedis we are back in business mate. now do i have to fully reindex everything with the new path? it was going through facial recognition when i stopped the stack
[7:59 PM]
ive added /external and it validated and i deleted my old path
[8:01 PM]
do i need to purge everything that cached already, and delete the external library first? i just realised i removed my old path and just replaced it with /external and didn't fully delete the old one hence the pics from my old path are still visible in the timeline
mmm
What does it say the filepath for those still visible is?
/usr/src/app/photos/Photos from 2023/IMG-20230702-WA0013.jpg
/usr/src/app/photos/ was my old path , i havent run the scan on the new path yet, although i've added it. should i delete the entire library first so just reindex with the new path /external now added in?
some photo links are now broken for obvious reasons when i select them but its still on the timeline
Yeah I guess delete would be the prudent way to go 👀
It's just metadata, what could go wrong
okie dokie mate
i believe i owe you a 6 pack 🍻
I don't accept beers for troubleshooting here because that would make me an alcoholic
Lol Fair call! We all need to head to AA down in Oz 😂
Cheers for your help mate
No problem, enjoy
sorry mate one more question before i go, do i enable or keep - Enable face import
Import faces from image EXIF data and sidecar files in its default setting which is off?
Doesn't really matter tbh
Only works if you have faces in there
Which you probably don't
ok sweet i'll leave it off then.
Because otherwise you wouldn't be asking 😛
so it wont have any impact on facial search and recognition which is currently running
indeed, just leave it off
It's for third party tools outside of immich
sweet cheers
@Mraedis sorry to bug you mate. Have a general question just regarding how I can setup immich to back up to my nas just in case I have a catastrophic hardware failure and need to recover the current settings including all the facial tags I've put time into compiling. Is there a way to periodically run a full backup? Do I have to save the entire picture gallery as a nas as an off line copy and save the immich core folders alongside it, for me to do a simple restore and scan for missing meta data if I were to recover from a terminal failure.
If you make a clone of the UPLOAD_LOCATION and have working automated backups (should be on by default) you'll have all you need
https://immich.app/docs/administration/backup-and-restore/
Cheers. I'll have a look at the documentation.
if i copy the entire directory inc. upload and thumbs and backups across to my nas then ,that should be enough along with my original gallery to restore immich completely without rebuilding all the facial data in redis and ML?
what do you mean along with your original gallery @stefan ?
ah right the external
Yes
Keep a copy of your compose / env as well 😉
perfect. yes due to all the issues with copilot troubleshooting im running a local copy of my gallery in the linux machine, i'm ideally hoping to install borg in my synology ds223j although it mentions its not fully compatible so thinking maybe i'll use my win 11 machine with docker to run borg and hopefully be able to run routine backups from the linux machine across the nas mount via the windows docker network mount. i've taken a copy of your env and compose file and the whole immich/setup directory
the j in ds223j stands for "this is too low power"
lol yes its in fact my first entry into NAS as a whole
That's what it's advertised as btw, entry (:
all the letters/numbers for synology have meaning
223 means 2 disks years (20)23
ah gotcha! , im very much an amateur so the 2 disks is more than enough for me setup jellyfin media and my pictures 😂 . i'm in the process of copying my photos and setup folders from immich across to the nas as a secondary backup. do you know much about setting up borg on a windows docker environment and possibly run a schedule to automatically backup the entire immich setup nightly with perhaps a max of 14
Nothing at all I'm afraid
not a problem! i did read up on the backup process for immich so due to any terminal faliure , its best to restore the entire file structure and then run the stack for the first time?
Not the stack
You restore the UPLOAD_LOCATION
Then you create the files/folders/containers by doing
docker compose create
This will create but not start
Then you start the database container and restore from DB dump
See also https://immich.app/docs/administration/backup-and-restore/#manual-backup-and-restoreso you restore from the path above first and run docker compose create
docker compose down -v # CAUTION! Deletes all Immich data to start from scratch
Uncomment the next line and replace DB_DATA_LOCATION with your Postgres path to permanently reset the Postgres database
rm -rf DB_DATA_LOCATION # CAUTION! Deletes all Immich data to start from scratch
docker compose pull # Update to latest version of Immich (if desired)
docker compose create # Create Docker containers for Immich apps without running them
docker start immich_postgres # Start Postgres server
sleep 10 # Wait for Postgres server to start up
Check the database user if you deviated from the default
gunzip --stdout /home/ravin/immich/setup/backups/immich-db-backup-20250818T020000-v1.137.3-pg14.18.sql.gz\
| sed "s/SELECT pg_catalog.set_config('search_path', '', false);/SELECT pg_catalog.set_config('search_path', 'public, pg_catalog', true);/g" \
| docker exec -i immich_postgres psql --dbname=postgres --username=<DB_USERNAME> # Restore Backup
docker compose up -d # Start remainder of Immich apps
does that look right?
Don't forget to fill in the <blanks>
Yeah of course 😊. Beyond that, there isn't anything else Is there? One more thing I was wondering is, how do I get my Ubuntu 24 to boot up the immich stack automatically in case of reboot for one reason or another? I had to manually run the stack again after a software update
is it ubuntu server 24?
Are you using docker desktop perhaps?
Ubuntu desktop 24 sorry
Docker desktop is there too but I gave trying to run immich on it due to permission issues and installed Ubuntu desktop on another mini pc solely to run immich 😁
Going to guess and say that docker daemon isn't set to auto-start
Absolutely not 😂
Ah it's actually my fault
I forgot
restart: always for your immich_server containerservices:
immich-server:
image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
container_name: immich_server
user: "1028:100"
env_file:
- .env
ports:
- "2283:2283"
volumes:
- ${UPLOAD_LOCATION}:/data
- /home/ravin/immich/photos:/external
- /etc/localtime:/etc/localtime:ro
depends_on:
- redis
- database
do you want me to add it after depends_on
also no issues at all mate. its all sweet. you've helped me out heaps!
It's in every other container, just copy it from there
as long as it's in the right indentation it's good
services:
immich-server:
image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
container_name: immich_server
user: "1028:100"
env_file:
- .env
ports:
- "2283:2283"
volumes:
- ${UPLOAD_LOCATION}:/data
- /home/ravin/immich/photos:/external
- /etc/localtime:/etc/localtime:ro
depends_on:
- redis
- database
restart: always
thats what i've done under server
indentation is correct 😁
I can't tell because you didn't put it in a code block 👀
Three backticks `
ahh fair enough! i'm using Notepad++ so can easliy see the indentation markers ,ive composed down and restarted again so hopefully if the pc does boot it should start up automatically
im on v1.137.3 , is there any significant updates on 1.138.1 compared to my version? i'm bit stressed about breaking the stack 😫
Just update
All fun fixes
how do i update. very much a noob question there lol . by the way i did buy a single user license , some dough towards further dev
docker compose pull
docker compose up -dshould i compose the stack down first?
no
pull just downloads, regardless of what is running, and doesn't create new containers
It does however move the tag
release to the new image
which compose up then uses to (re)create the containerssweet cheers!
v1.138.1 is up and running 😊