Immich•5mo ago

Immich ML Defaults to CPU Always

Good day, I am working on setting up an Immich server and wanted to break off the ML part to a server with a Nvidia GPU installed. I deployed the Immich-ML v1.123.0 container to this GPU enabled server. Updated the compose file to include the Nvidia GPU settings but no matter what I do I can not get the ML container to use the GPU. It always comes back with ['CPUExecutionProvider'] as the execution provider. It works, as the Immich server does facial recognition fine, but I want to lever the GPU for processing. I have a number of other Docker containers running services that leverage the GPU, so I know the Docker/Nvidia setup is working. Nvidia drivers versions are CUDA 12.5 and driver release 555. I checked to make sure the GPU computer version was new enough, it is version 8.6. Docker Compose File:

name: immich
services:
  machine-learning:
    image: ghcr.io/immich-app/immich-machine-learning:v1.123.0
    volumes:
      - model-cache:/cache
    restart: unless-stopped
    ports:
      - 3003:3003
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

volumes:
  model-cache:

name: immich
services:
  machine-learning:
    image: ghcr.io/immich-app/immich-machine-learning:v1.123.0
    volumes:
      - model-cache:/cache
    restart: unless-stopped
    ports:
      - 3003:3003
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

volumes:
  model-cache:

Any suggestions? There is 4GB of VRAM for the GPU is that enough to load the models? I don't get any errors saying it can't. I can provide a log snippet if necessary. Appreciate your help, thank you.

4 Replies

Immich•5mo ago

:wave: Hey @walloutlet, Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:. References - Container Logs: docker compose logs docs - Container Status: docker ps -a docs - Reverse Proxy: https://immich.app/docs/administration/reverse-proxy - Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA Checklist I have... 1. :blue_square: verified I'm on the latest release(note that mobile app releases may take some time). 2. :blue_square: read applicable release notes. 3. :blue_square: reviewed the FAQs for known issues. 4. :blue_square: reviewed Github for known issues. 5. :blue_square: tried accessing Immich via local ip (without a custom reverse proxy). 6. :blue_square: uploaded the relevant information (see below). 7. :blue_square: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable (an item can be marked as "complete" by reacting with the appropriate number) Information In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider: - Your docker-compose.yml and .env files. - Logs from all the containers and their status (see above). - All the troubleshooting steps you've tried so far. - Any recent changes you've made to Immich or your system. - Details about your system (both software/OS and hardware). - Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h). - The version of the Immich server, mobile app, and other relevant pieces. - Any other information that you think might be relevant. Please paste files and logs with proper code formatting, and especially avoid blurry screenshots. Without the right information we can't work out what the problem is. Help us help you ;) If this ticket can be closed you can use the /close command, and re-open it later if needed.

bo0tzz•5mo ago

You're not using the right image. https://immich.app/docs/features/ml-hardware-acceleration

walloutletOP•5mo ago

Thanks for the reply... where is the list of models that I can choose from? I've read that specific page a hundred times but either my mind is just missing the link or section or it's no longer there. Models currently being used are: - buffalo_l - ViT-B-32__openai Oh you mean image... like container image. Man I've been at this too long today. I gotchu I missed the -cuda Duh It was juuuusssttt off my screen on the webpage That did the trick! Much appreciated

Immich•5mo ago

This thread has been closed. To re-open, use the button below.

Gaming

Programming

Immich ML Defaults to CPU Always

Did you find this page helpful?