Model Caching & Preloading
Hi all, I'd like to know if anyone had success properly setting up and mounting a directory for the models / caching so Immich doesn't have to keep downloading them when the container restarts or when it unloads the model after a period.
I already posted about this topic here with more detail/context but sounds like this is the proper place for it https://discord.com/channels/979116623879368755/1434005199575322674
8 Replies
:wave: Hey @CyDefUnicorn,
Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.
References
- Container Logs:
docker compose logs docs
- Container Status: docker ps -a docs
- Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
- Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA
Checklist
I have...
1. :blue_square: verified I'm on the latest release(note that mobile app releases may take some time).
2. :blue_square: read applicable release notes.
3. :blue_square: reviewed the FAQs for known issues.
4. :blue_square: reviewed Github for known issues.
5. :blue_square: tried accessing Immich via local ip (without a custom reverse proxy).
6. :blue_square: uploaded the relevant information (see below).
7. :blue_square: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable
(an item can be marked as "complete" by reacting with the appropriate number)
Information
In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:
- Your docker-compose.yml and .env files.
- Logs from all the containers and their status (see above).
- All the troubleshooting steps you've tried so far.
- Any recent changes you've made to Immich or your system.
- Details about your system (both software/OS and hardware).
- Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h).
- The version of the Immich server, mobile app, and other relevant pieces.
- Any other information that you think might be relevant.
Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)
If this ticket can be closed you can use the /close command, and re-open it later if needed.One thing I will say is that I'm using two diff .env files; one .env for the immich-server, and one .envml for the ML container. Both .env and .envml files have the MACHINE_LEARNING_CACHE_FOLDER=/mnt/NAS_Storage/Immich/models/cache line set, but when I redeploy the contianers and watch the logs, I still see "Downloading recognition model 'buffalo_l' to /mnt/NAS_Storage/Immich/models/cache/facial-recognition/buffalo_l/recognition/model.onnx. This may take a while.". I also created the directories exacty as shown in the log
Here is my COMPOSE FILE with relevant sections:
name: immich
services:
immich-server:
container_name: immich_server
image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
extends:
file: hwaccel.transcoding.yml
service: nvenc # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
volumes:
# Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
- ${UPLOAD_LOCATION}:/data
- ${THUMB_LOCATION}:/data/thumbs
- ${ENCODED_VIDEO_LOCATION}:/data/encoded-video
- ${PROFILE_LOCATION}:/data/profile
- ${BACKUP_LOCATION}:/data/backups
- /etc/localtime:/etc/localtime:ro
- /mnt/Storage/Other Pictures:/mnt/Storage/Other Pictures:rw
env_file:
- .env
ports:
- '2283:2283'
depends_on:
- redis
- database
restart: always
healthcheck:
disable: false
#deploy:
#resources:
#limits:
#cpus: '6'
immich-machine-learning:
container_name: immich_machine_learning
# For hardware acceleration, add one of -[armnn, cuda, rocm, openvino, rknn] to the image tag.
# Example tag: ${IMMICH_VERSION:-release}-cuda
image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-cuda
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 2
capabilities:
- gpu
- compute
- video
volumes:
- model-cache:/cache
env_file:
- .envml
restart: always
ports:
- 3003:3003
healthcheck:
disable: false
here is my .ENV file:
You can find documentation for all the supported env variables at https://docs.immich.app/install/environment-variables
The location where your uploaded files are stored
UPLOAD_LOCATION=/mnt/NAS_Storage/Immich/Library
THUMB_LOCATION=/mnt/NAS_Storage/Immich/thumbs
ENCODED_VIDEO_LOCATION=/mnt/NAS_Storage/Immich/encoded-video
PROFILE_LOCATION=/mnt/NAS_Storage/Immich/profile
BACKUP_LOCATION=/mnt/NAS_Storage/Immich/backups
MACHINE_LEARNING_CACHE_FOLDER=/mnt/NAS_Storage/Immich/models/cache
The location where your database files are stored. Network shares are not supported for the database
DB_DATA_LOCATION=./postgres
To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
TZ=Etc/EST
The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=v2
Number of cores available to the Immich server
CPU_CORES=12
DB_PASSWORD=REDACTED
Ports
IMMICH_HOST=0.0.0.0
IMMICH_PORT=2283
DB Tuning
DB_STORAGE_TYPE=SSD
DB_USERNAME=postgres
DB_DATABASE_NAME=immich
Here is my .ENVML FILE for the ML container:
Machine Learning
MACHINE_LEARNING_MODEL_TTL=300
MACHINE_LEARNING_MODEL_TTL_POLL_S=0
MACHINE_LEARNING_CACHE_FOLDER=/mnt/NAS_Storage/Immich/models/cache
MACHINE_LEARNING_REQUEST_THREADS=4
MACHINE_LEARNING_DEVICE_IDS=0,1
MACHINE_LEARNING_WORKERS=2
MACHINE_LEARNING_WORKER_TIMEOUT=120
Listen on all int
IMMICH_HOST=0.0.0.0
IMMICH_PORT=3003
MACHINE_LEARNING_PRELOADCLIPTEXTUAL=ViT-SO400M-16-SigLIP2-384webli
MACHINE_LEARNING_PRELOADCLIPVISUAL=ViT-SO400M-16-SigLIP2-384webli
MACHINE_LEARNING_PRELOADFACIAL_RECOGNITIONRECOGNITION=buffalo_l
MACHINE_LEARNING_PRELOADFACIAL_RECOGNITIONDETECTION=buffalo_l
Remove
And remove
Change it to
Looks like that did it; made the changes, restarted the ML container and so far even with multiple restarts, it's preloading and not seeing anythign being downloaded, at least not after the first run. Initially it was giving me constants errors and restarting so I deleted the directories and models and just let it make the directories on its' own which it did. After few reboots, no downloads.
Thank you for the help, I really appreciate the clear and straight to the point error point out and solution
With the default compose models are never re-downloaded unless you explicitely remove the volumes
MACHINE_LEARNING_MODEL_TTL=300 means "unload the model after 5 minutes"
If you want to keep them loaded set it at 0Yeah I've toyed with that too. I can't think of a reason to offload it since I may want to search for something specific at any point
I only load clip textual, as the rest are for background tasks, but if you don't need the ram for anything else there's no harm in keeping everything loaded
I have one 3090 for Immich, with some VRAM room still, and another for other things like Open Webui, Comfy etc.
I had them both available to immich to split the load of the background tasks but figured that was overkill