ArtOfficial
RRunPod
•Created by ArtOfficial on 5/2/2025 in #⛅｜pods-clusters
Cuda Driver Version issues
The error below shows up on various machines. I am using the following base images:
nvidia/cuda:12.8.0-cudnn-devel-ubuntu22.04
nvidia/cuda:12.6.0-cudnn-devel-ubuntu22.04
nvidia/cuda:12.5.1-cudnn-devel-ubuntu22.04
Machine ID: lmrw66dp9b9b6r
For some reason when I change the cuda version, the error message changes based on the base version
cuda:12.8 shows cuda>=12.8
cuda:12.6 shows cuda>=12.6
cuda:12.5 shows cuda>=12.5
error starting container: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running prestart hook #0: exit status 1, stdout: , stderr: Auto-detected mode as 'legacy'
nvidia-container-cli: requirement error: unsatisfied condition: cuda>=12.8, please update your driver to a newer version, or use an earlier cuda container: unknown
16 replies
RRunPod
•Created by ArtOfficial on 5/1/2025 in #⛅｜pods-clusters
failed to create shim task: OCI runtime create failed:
I only see this issue when using the 5090 pods, and not all of them either. the base docker image is nvidia/cuda:12.8.0-cudnn-devel-ubuntu22.04
error starting container: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error setting cgroup config for procHooks process: openat2 /sys/fs/cgroup/memory/docker/52fd6559d48734b8cef57a73bd60c7e11468d50c7f3d34265f3c1ccadd5a6e88/memory.limit_in_bytes: no such file or directory: unknown
5 replies