Immich•2d ago

Remote ML container not using GPU

OS: Windows NT4 (Joking haha its Windows 11) CPU: AMD Ryzen 9 5900X GPU 3080 RTX Nvidia I am processing about 1 asset every 2 seconds 🙁 Docker Compose

name: immich_remote_ml

services:
  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, rocm, openvino, rknn] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-cuda
    extends:
       file: hwaccel.ml.yml
       service: cuda # set to one of [armnn, cuda, rocm, openvino, openvino-wsl, rknn] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    restart: always
    ports:
      - 3003:3003



volumes:
  model-cache:

name: immich_remote_ml

services:
  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, rocm, openvino, rknn] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-cuda
    extends:
       file: hwaccel.ml.yml
       service: cuda # set to one of [armnn, cuda, rocm, openvino, openvino-wsl, rknn] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    restart: always
    ports:
      - 3003:3003



volumes:
  model-cache:

21 Replies

Immich•2d ago

:wave: Hey @mikekay1, Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:. References - Container Logs: docker compose logs docs - Container Status: docker ps -a docs - Reverse Proxy: https://immich.app/docs/administration/reverse-proxy - Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA Checklist I have... 1. :ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time). 2. :ballot_box_with_check: read applicable release notes. 3. :ballot_box_with_check: reviewed the FAQs for known issues. 4. :ballot_box_with_check: reviewed Github for known issues. 5. :ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy). 6. :ballot_box_with_check: uploaded the relevant information (see below). 7. :ballot_box_with_check: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable (an item can be marked as "complete" by reacting with the appropriate number) Information In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider: - Your docker-compose.yml and .env files. - Logs from all the containers and their status (see above). - All the troubleshooting steps you've tried so far. - Any recent changes you've made to Immich or your system. - Details about your system (both software/OS and hardware). - Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h). - The version of the Immich server, mobile app, and other relevant pieces. - Any other information that you think might be relevant. Please paste files and logs with proper code formatting, and especially avoid blurry screenshots. Without the right information we can't work out what the problem is. Help us help you ;) If this ticket can be closed you can use the /close command, and re-open it later if needed. Successfully submitted, a tag has been added to inform contributors. :white_check_mark:

mikekay1OP•2d ago

I do reverse proxy but for this it shouldn't matter, since ML docker isn't doing anything

Mraedis•2d ago

What are you running this compose on?

mikekay1OP•2d ago

wsl

Mraedis•2d ago

Did you add the URL in your immich instance?

mikekay1OP•2d ago

I did photos.domain.com for ML box I am using ip:port everything communication wise is okay its the ML docker not doing its lifting, could it be because it uses localhost container by default?

Mraedis•2d ago

if you do a curl http://<ML-IP>:3003 what does it say

mikekay1OP•2d ago

if I access it using a browser I get json let me try from a linux box

Mraedis•2d ago

The containers are accessed in order, so if the localhost one is on top then yes this won't work it doesn't do load balancing

mikekay1OP•2d ago

nah thats not the case I removed and overwrote it

Mraedis•2d ago

and the docker logs for the remote ML container? it should say something about not finding a GPU provider

mikekay1OP•2d ago

ah good point let me check docker engine windows app is garbage runs once per startup, let me try command to get logs

mikekay1OP•2d ago

now I didnt run with cuda-wsl because my build didn't work did I miss soemthing?

Mraedis•2d ago

try executing docker exec -it immich_machine_learning nvidia-smi

mikekay1OP•2d ago

sorry kid came home

mikekay1OP•2d ago

I just tried to shutdown wsl and update no go should i increase the number of concurrent processes to like 100 or more? Its doing 200 a second now and my gpu is up at 50% its an SMB directly and things are traversing the 10G network over a SAN. I am okay with this. We can close this, I did end up pulling the hwaccel.ml.yml file and directly pasting the deploy inline im compose In*

Mraedis•2d ago

Curious that worked while it should just be exactly the same result but thanks for reporting back!

Immich•2d ago

This thread has been closed. To re-open, use the button below.

Gaming

Programming

Remote ML container not using GPU

Did you find this page helpful?