GPU not detected on RunPod serverless - HELP!!
Hey everyone,
I'm running into an issue on RunPod serverless endpoint. Despite having CUDA 12.4.1 set up in my Docker container, my models are initializing on CPU instead of GPU.
My logs show: "Initializing pipeline on cpu"
I'm trying to run Whisper model which would really benefit from GPU acceleration. I've included all the CUDA requirements in my Dockerfile and have torch 2.4.1 in my requirements.txt.
Has anyone encountered a similar issue with RunPod serverless? Is there something specific I need to configure to make it recognize the GPU?
Any help would be greatly appreciated!
Thanks in advance
I'm running into an issue on RunPod serverless endpoint. Despite having CUDA 12.4.1 set up in my Docker container, my models are initializing on CPU instead of GPU.
My logs show: "Initializing pipeline on cpu"
I'm trying to run Whisper model which would really benefit from GPU acceleration. I've included all the CUDA requirements in my Dockerfile and have torch 2.4.1 in my requirements.txt.
Has anyone encountered a similar issue with RunPod serverless? Is there something specific I need to configure to make it recognize the GPU?
Any help would be greatly appreciated!
Thanks in advance
