Immich•3w ago

Remote ML

I have no idea what I am doing wrong for remote ML

services:
  immich-machine-learning:
    image: ghcr.io/immich-app/immich-machine-learning:release-cuda
    container_name: immich-ml
    restart: unless-stopped
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
      - NVIDIA_DRIVER_CAPABILITIES=compute,utility
      - TRANSFORMERS_CACHE=/cache
    volumes:
      - ./cache:/cache
    ports:
      - "3003:3003"

services:
  immich-machine-learning:
    image: ghcr.io/immich-app/immich-machine-learning:release-cuda
    container_name: immich-ml
    restart: unless-stopped
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
      - NVIDIA_DRIVER_CAPABILITIES=compute,utility
      - TRANSFORMERS_CACHE=/cache
    volumes:
      - ./cache:/cache
    ports:
      - "3003:3003"

My remote ml file it loads into memory, but GPU utilisation is 0%

14 Replies

Immich•3w ago

:wave: Hey @kizaru3805, Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:. References - Container Logs: docker compose logs docs - Container Status: docker ps -a docs - Reverse Proxy: https://immich.app/docs/administration/reverse-proxy - Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA Checklist I have... 1. :ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time). 2. :ballot_box_with_check: read applicable release notes. 3. :ballot_box_with_check: reviewed the FAQs for known issues. 4. :ballot_box_with_check: reviewed Github for known issues. 5. :ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy). 6. :ballot_box_with_check: uploaded the relevant information (see below). 7. :ballot_box_with_check: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable (an item can be marked as "complete" by reacting with the appropriate number) Information In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider: - Your docker-compose.yml and .env files. - Logs from all the containers and their status (see above). - All the troubleshooting steps you've tried so far. - Any recent changes you've made to Immich or your system. - Details about your system (both software/OS and hardware). - Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h). - The version of the Immich server, mobile app, and other relevant pieces. - Any other information that you think might be relevant. Please paste files and logs with proper code formatting, and especially avoid blurry screenshots. Without the right information we can't work out what the problem is. Help us help you ;) If this ticket can be closed you can use the /close command, and re-open it later if needed.

bo0tzz•3w ago

Did you configure the server to use it?

kizaru3805OP•3w ago

yes

kizaru3805OP•3w ago

I kept getting this error

kizaru3805OP•3w ago

Then I reduced maximum concurrent jobs to 5 now it fits into memory, but i get nothing, just shows gpu memory getting full without any processing

kizaru3805OP•3w ago

I don't understand this

jsdev•3w ago

Looks like you are seeing gpu utilization. Set concurrent jobs to 1 and see if it processes anything.

kizaru3805OP•3w ago

Ok. Thank you.

kizaru3805OP•3w ago

This is what i get with 1

t8585•3w ago

https://jellyfin.org/docs/general/post-install/transcoding/hardware-acceleration/nvidia#configure-with-linux-virtualization:~:text=%3A/media-,runtime%3A%20nvidia,-deploy%3A%0A%20%20%20%20%20%20resources

    runtime: nvidia

    runtime: nvidia

NVIDIA GPU | Jellyfin

This tutorial guides you on setting up full video hardware acceleration on NVIDIA GPU via NVENC.

t8585•3w ago

Might also need to do this; https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html#configuring-docker to make the runtime work https://discord.com/channels/979116623879368755/994044917355663450/1416788792471191622 If youre using WSL this might work

sudo nvidia-ctk runtime configure --runtime=docker sudo systemctl restart docker

services:
  immich-machine-learning:
    image: ghcr.io/immich-app/immich-machine-learning:release-cuda
    container_name: immich-ml
    restart: unless-stopped
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
      - NVIDIA_DRIVER_CAPABILITIES=compute,utility
      - TRANSFORMERS_CACHE=/cache
    runtime: nvidia
    volumes:
      - ./cache:/cache
    ports:
      - "3003:3003"

services:
  immich-machine-learning:
    image: ghcr.io/immich-app/immich-machine-learning:release-cuda
    container_name: immich-ml
    restart: unless-stopped
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
      - NVIDIA_DRIVER_CAPABILITIES=compute,utility
      - TRANSFORMERS_CACHE=/cache
    runtime: nvidia
    volumes:
      - ./cache:/cache
    ports:
      - "3003:3003"

kizaru3805OP•3w ago

I get this with your compose file @t8585

Gaming

Programming

Remote ML

Did you find this page helpful?