no cuda gpu detected

I dont know if this is a general problem or not

import torch torch.cuda.is_available()
gives error might not have cuda gpu

I have tried
FROM runpod/pytorch:2.4.0-py3.11-cuda12.4.1-devel-ubuntu22.04
nvidia/cuda:12.1.0-cudnn8-devel-ubuntu20.04
nvidia/cuda:12.3.2-cudnn9-runtime-ubuntu22.04
pytorch/pytorch:2.4.0-cuda12.4-cudnn9-devel

All running in secure cloud :/
nvidia-smi seems fine, so does pip show torch , cuda version of it is installed.

This happens 1/4 of the time from my testing so far.

Solution

When deploying a pod click filters then you will get will get these options. There is a pull down for CUDA Versions.