Problem with RunPod cuda base image. Jobs stuck in queue forever
Hello, I'm trying to do a request to a serverless endpoint that uses this base image on its Dockerfile
I want the serverside to run the
If I use the cuda base image it does not run
The thing is that if I use this base image:
It does run both
So my questions are:
- Why is the problem happening in the cuda base image?
- What are the implications of using the 2nd base image? Are there cuda or pytorch dependencies missing here?
- What base image should I use? What do I do?
FROM runpod/base:0.4.0-cuda11.8.0I want the serverside to run the
input_fn function when I do the request. This is part of the server side code:If I use the cuda base image it does not run
input_fn, I only see the debug prints from model_fn and then the job stays in queue forever (photo).The thing is that if I use this base image:
FROM python:3.11.1-busterIt does run both
input_fn and model_fnSo my questions are:
- Why is the problem happening in the cuda base image?
- What are the implications of using the 2nd base image? Are there cuda or pytorch dependencies missing here?
- What base image should I use? What do I do?
