Error CUDA 12.8 GPU detection
Environment:
- RunPod GPU Instance (RTX A5000, 24GB VRAM)
- Ray Serve deployment
- Piper TTS model via ONNX Runtime
- CUDA 12.8
Issue:
Since today, experiencing GPU detection failure when deploying a Ray Serve application with Piper TTS model. The ONNX Runtime correctly detects CUDAExecutionProvider but fails during model initialization.
Error Details:
CUDA failure 100: no CUDA-capable device is detected ; GPU=-1 when using [('CUDAExecutionProvider', {'cudnn_conv_algo_search': 'HEURISTIC'})]
Falling back to ['CPUExecutionProvider'] and retrying.
Has anyone encountered similar GPU detection issues ?
Any guidance would be appreciated! 🙏
3 Replies
Very likely a pod that does not have cuda 12.8
Use the filter on the pod deployment page and select 12.8 and 12.9 so your on a new enough pod
I have filtered on it but it still doesn't detect the GPU
Type nvidia-smi
Does it list it with the right cuda?