Immich•9mo ago

Unable to Utilize GPU (CUDA) for Machine Learning in Docker

NappingSodion · 2025-03-31T08:15:36.021Z

**Environment:** - **OS:** Windows 10 Pro - **Docker:** - Desktop Engine: 28.0.1 (with WSL2 Ubuntu 22.04 backend) - Compose: v2.33.1-desktop.1 - **GPU:** NVIDIA GeForce RTX 2070 - Driver Version: 572.47 - CUDA Version: 12.8 **Steps Taken:** 1. Followed the official documentation to update `compose.yml` and redeployed the container using: ```bash docker compose down && docker compose up -d ``` 2. Successfully tested GPU integration using the sample compose file from [Docker's GPU support guide](https://docs.docker.com/compose/how-tos/gpu-support/). The container correctly displayed GPU info (Driver Version, CUDA version, and GPU name). - **Note:** Observed a minor discrepancy in `NVIDIA-SMI` version: - Native Windows: `572.47` - Docker container: `570.86.17` 3. Attempted to restart Immich with the updated compose configuration (details in the following conversation). **Issue:** The `immich/immich_machine_learning` container logs shows it still using CPU to process ML stuff and no CUDA usage shown in windows taskmgr(details in the following conversation). **Question:** I'm unsure how to proceed further. Any guidance on resolving this GPU utilization issue would be greatly appreciated.

Environment:

OS: Windows 10 Pro
Docker:
- Desktop Engine: 28.0.1 (with WSL2 Ubuntu 22.04 backend)
- Compose: v2.33.1-desktop.1
GPU: NVIDIA GeForce RTX 2070
- Driver Version: 572.47
- CUDA Version: 12.8
  Steps Taken:

Followed the official documentation to update

compose.yml

compose.yml

compose.yml

compose.yml and redeployed the container using:

docker compose down && docker compose up -d

docker compose down && docker compose up -d

docker compose down && docker compose up -d

docker compose down && docker compose up -d

Successfully tested GPU integration using the sample compose file from Docker's GPU support guide. The container correctly displayed GPU info (Driver Version, CUDA version, and GPU name).
- Note: Observed a minor discrepancy in
```
NVIDIA-SMI
```
  NVIDIA-SMI
```
NVIDIA-SMI
```
  NVIDIA-SMI
  version:
  - Native Windows:
    572.47
    572.47
    572.47
    572.47
  - Docker container:
    570.86.17
    570.86.17
    570.86.17
    570.86.17
Attempted to restart Immich with the updated compose configuration (details in the following conversation).
Issue:
The
```
immich/immich_machine_learning
```
immich/immich_machine_learning
```
immich/immich_machine_learning
```
immich/immich_machine_learning
container logs shows it still using CPU to process ML stuff and no CUDA usage shown in windows taskmgr(details in the following conversation).
Question:
I'm unsure how to proceed further. Any guidance on resolving this GPU utilization issue would be greatly appreciated.

Immich•3/31/25, 8:15 AM

:wave: Hey @NappingSodion,

Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.

References

Container Logs:
```
docker compose logs
```
```
docker compose logs
```
docs
Container Status:
```
docker ps -a
```
```
docker ps -a
```
docs
Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA

Immich•3/31/25, 8:15 AM

Checklist

I have...

:ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time).
:ballot_box_with_check: read applicable release notes.
:ballot_box_with_check: reviewed the FAQs for known issues.
:ballot_box_with_check: reviewed Github for known issues.
:ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy).
:ballot_box_with_check: uploaded the relevant information (see below).
:ballot_box_with_check: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable

(an item can be marked as "complete" by reacting with the appropriate number)

Information

In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:

Your docker-compose.yml and .env files.
Logs from all the containers and their status (see above).
All the troubleshooting steps you've tried so far.
Any recent changes you've made to Immich or your system.
Details about your system (both software/OS and hardware).
Details about your storage (filesystems, type of disks, output of commands like
```
fdisk -l
```
```
fdisk -l
```
and
```
df -h
```
```
df -h
```
).
The version of the Immich server, mobile app, and other relevant pieces.
Any other information that you think might be relevant.

Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)

If this ticket can be closed you can use the

/close

/close

command, and re-open it later if needed.

NappingSodionOP•3/31/25, 8:16 AM

config:

...
  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities:
                - gpu
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always
    healthcheck:
      disable: false
...

...
  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities:
                - gpu
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always
    healthcheck:
      disable: false
...

log:

Initializing Immich ML v1.130.3
INFO     Starting gunicorn 23.0.0
INFO     Listening at: http://[::]:3003 (9)
INFO     Using worker: app.config.CustomUvicornWorker
INFO     Booting worker with pid: 10
INFO     Started server process [10]
INFO     Waiting for application startup.
INFO     Created in-memory cache with unloading after 300s of inactivity.
INFO     Initialized request thread pool with 8 threads.
INFO     Application startup complete.
INFO     Loading detection model 'buffalo_l' to memory
INFO     Setting execution providers to ['CPUExecutionProvider'], in descending order of preference
INFO     Loading recognition model 'buffalo_l' to memory
INFO     Setting execution providers to ['CPUExecutionProvider'], in descending order of preference
INFO     Loading visual model 'ViT-B-32__openai' to memory
INFO     Setting execution providers to ['CPUExecutionProvider'], in descending order of preference

Initializing Immich ML v1.130.3
INFO     Starting gunicorn 23.0.0
INFO     Listening at: http://[::]:3003 (9)
INFO     Using worker: app.config.CustomUvicornWorker
INFO     Booting worker with pid: 10
INFO     Started server process [10]
INFO     Waiting for application startup.
INFO     Created in-memory cache with unloading after 300s of inactivity.
INFO     Initialized request thread pool with 8 threads.
INFO     Application startup complete.
INFO     Loading detection model 'buffalo_l' to memory
INFO     Setting execution providers to ['CPUExecutionProvider'], in descending order of preference
INFO     Loading recognition model 'buffalo_l' to memory
INFO     Setting execution providers to ['CPUExecutionProvider'], in descending order of preference
INFO     Loading visual model 'ViT-B-32__openai' to memory
INFO     Setting execution providers to ['CPUExecutionProvider'], in descending order of preference

IImmich ## Checklist I have... 1. :ballot_box_with_check: verified I'm on the latest rel...

ImmichAPP•3/31/25, 8:17 AM

Successfully submitted, a tag has been added to inform contributors. :white_check_mark:

Unable to Utilize GPU (CUDA) for Machine Learning in Docker

References

Checklist

Information

Similar Threads

Similar Threads

References

Checklist

Information

Similar Threads