walloutlet Posts - Answer Overflow

walloutlet

•Created by walloutlet on 12/24/2024 in #help-desk-support

Immich ML Defaults to CPU Always

Good day, I am working on setting up an Immich server and wanted to break off the ML part to a server with a Nvidia GPU installed. I deployed the Immich-ML v1.123.0 container to this GPU enabled server. Updated the compose file to include the Nvidia GPU settings but no matter what I do I can not get the ML container to use the GPU. It always comes back with ['CPUExecutionProvider'] as the execution provider. It works, as the Immich server does facial recognition fine, but I want to lever the GPU for processing. I have a number of other Docker containers running services that leverage the GPU, so I know the Docker/Nvidia setup is working. Nvidia drivers versions are CUDA 12.5 and driver release 555. I checked to make sure the GPU computer version was new enough, it is version 8.6. Docker Compose File:

name: immich
services:
  machine-learning:
    image: ghcr.io/immich-app/immich-machine-learning:v1.123.0
    volumes:
      - model-cache:/cache
    restart: unless-stopped
    ports:
      - 3003:3003
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

volumes:
  model-cache:

name: immich
services:
  machine-learning:
    image: ghcr.io/immich-app/immich-machine-learning:v1.123.0
    volumes:
      - model-cache:/cache
    restart: unless-stopped
    ports:
      - 3003:3003
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

volumes:
  model-cache:

Any suggestions? There is 4GB of VRAM for the GPU is that enough to load the models? I don't get any errors saying it can't. I can provide a log snippet if necessary. Appreciate your help, thank you.

12 replies

Gaming

Programming