Face Detection Error (CUDA runtime version)
I'm getting the following error when I try and run face detection in Immich.
*************** EP Error ***************
EP Error /onnxruntime_src/onnxruntime/core/providers/cuda/cuda_call.cc:121 std::conditional_t<THRW, void, onnxruntime::common::Status> onnxruntime::CudaCall(ERRTYPE, const char*, const char*, ERRTYPE, const char*, const char*, int) [with ERRTYPE = cudaError; bool THRW = true; std::conditional_t<THRW, void, onnxruntime::common::Status> = void] /onnxruntime_src/onnxruntime/core/providers/cuda/cuda_call.cc:114 std::conditional_t<THRW, void, onnxruntime::common::Status> onnxruntime::CudaCall(ERRTYPE, const char*, const char*, ERRTYPE, const char*, const char*, int) [with ERRTYPE = cudaError; bool THRW = true; std::conditional_t<THRW, void, onnxruntime::common::Status> = void] CUDA failure 35: CUDA driver version is insufficient for CUDA runtime version ; GPU=0 ; hostname=310fec5ac57b ; file=/onnxruntime_src/onnxruntime/core/providers/cuda/cuda_execution_provider.cc ; line=245 ; expr=cudaSetDevice(info_.device_id);
I'm running this through docker on Unraid. I've tried setting NVIDIA_VISIBLE_DEVICES to my gpuid and NVIDIA_DRIVER_CAPABILITIES to all (although I'm not sure this is necessary because from what I've seen it looks like dockerfile just sets these to all. I've also set MACHINE_LEARNING_GPU_ACCELERATION to cuda. I only have a 1050ti (maybe this card is too old?) but it's running the latest drivers available in the unraid nvidia plugin (this was after upgrading from 545.[something]). Attached is my nvidia-smi output (I don't have the output from prior to the driver upgrade).
7 Replies
:wave: Hey @binarymelon,
Thanks for reaching out to us. Please follow the recommended actions below; this will help us be more effective in our support effort and leave more time for building Immich :immich:.
References
- Container Logs:
docker compose logs docs
- Container Status: docker compose ps docs
- Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
Checklist
1. :ballot_box_with_check: I have verified I'm on the latest release(note that mobile app releases may take some time).
2. :ballot_box_with_check: I have read applicable release notes.
3. :ballot_box_with_check: I have reviewed the FAQs for known issues.
4. :ballot_box_with_check: I have reviewed Github for known issues.
5. :ballot_box_with_check: I have tried accessing Immich via local ip (without a custom reverse proxy).
6. :ballot_box_with_check: I have uploaded the relevant logs, docker compose, and .env files using the buttons below or the /upload command.
7. :ballot_box_with_check: I have tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable
(an item can be marked as "complete" by reacting with the appropriate number)
If this ticket can be closed you can use the /close command, and re-open it later if needed.@binarymelon uploaded
Sorry, the logs are kind of intermingled because of the unraid logs. I'll see if I can find the actual log files.
Successfully submitted, a tag has been added to inform contributors. :white_check_mark:
🤦♂️ I had forgotten to add
--runtime=nvidia.This thread has been closed. To re-open, use the button below.