Runpod error starting container
2024-03-07T14:40:19Z error starting container: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: Auto-detected mode as 'legacy'
Inconsistency detected by ld.so: ../sysdeps/x86_64/dl-machine.h: 534: elf_machine_rela_relative: Assertion `ELFW(R_TYPE) (reloc->r_info) == R_X86_64_RELATIVE' failed!
nvidia-container-cli: detection error: driver rpc error: failed to process request: unknown
I restart pod but still error
15 Replies
Can you provide more informations?
Pod id will also be useful
pod id is: 7k5m1uf3rz4yoy, EU-RO-1
Looks like there is an issue with hardware, checking with DC team on this.
If you are using network volume, please terminate the pod and create a new one so you can access your data.
If you are using local storage, then we need to wait until the hardware issue is fixed.
New pod will be placed in to different host.
Before terminating the old pod, please first create new pod and verify data if its network storage.
Yes. We have terminated.
Another issue @Satish @Papa Madiator
2024-03-13T15:30:27Z start container
2024-03-13T15:30:30Z error starting container: Error response from daemon: driver failed programming external connectivity on endpoint 4bjf5n4vpbv46f-0 (8de4860d317dc036a9e7527a00d592ee7d1a29b8262ac119438c8b579757f7c4): Error starting userland proxy: listen tcp4 0.0.0.0:40168: bind: address already in use
2024-03-13T15:30:31Z start container
2024-03-13T15:30:32Z error starting container: Error response from daemon: driver failed programming external connectivity on endpoint 4bjf5n4vpbv46f-0 (507df3a0f7d63f508f43d9709b724576e2ef3dac3f7ebda732741e3e311e4822): Error starting userland proxy: listen tcp4 0.0.0.0:40168: bind: address already in use
2024-03-13T15:30:49Z start container
2024-03-13T15:30:52Z error starting container: Error response from daemon: driver failed programming external connectivity on endpoint 4bjf5n4vpbv46f-0 (1cd76168203a4e67ac7c0dd9e6b70325d35ddb46d6dda00f388454475bbb55d0): Error starting userland proxy: listen tcp4 0.0.0.0:40168: bind: address already in use
2024-03-13T15:31:07Z start container
40168 is already being used
is it serverless or pod?
runpod
pod in gpu-cloud
pod id: 4bjf5n4vpbv46f
I have terminated it
so if you terminated it we will wont know what was issue 😄
ok. If I see this issue again, I'll notify you, and don't terminate for debug your side
Hi @Madiator2011 Pod ID 9rb76yk8uvvy0q has issues:
2024-03-29T04:17:37Z Status: Image is up to date for runpod/stable-diffusion:web-ui-10.2.1
2024-03-29T04:17:47Z create container runpod/stable-diffusion:web-ui-10.2.1
2024-03-29T04:17:55Z pending image pull runpod/stable-diffusion:web-ui-10.2.1
2024-03-29T04:18:07Z error pulling image: Error response from daemon: Get "https://registry-1.docker.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
cc @Satish
Tried restarting pod?
Yes, I have restart pod yet
2024-03-29T15:42:33Z create pod network
2024-03-29T15:42:33Z create container runpod/stable-diffusion:web-ui-10.2.1
2024-03-29T15:42:41Z pending image pull runpod/stable-diffusion:web-ui-10.2.1
2024-03-29T15:42:54Z error pulling image: Error response from daemon: Head "https://registry-1.docker.io/v2/runpod/stable-diffusion/manifests/web-ui-10.2.1": Get "https://auth.docker.io/token?scope=repository%3Arunpod%2Fstable-diffusion%3Apull&service=registry.docker.io": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
PodID: rcdd36seu3owfw
I think zone EU-SE-1 has problem.
please take care
PodID: ive5fzoddxzfbb still error
2024-03-29T23:17:01Z create container runpod/stable-diffusion:web-ui-10.2.1
2024-03-29T23:17:09Z pending image pull runpod/stable-diffusion:web-ui-10.2.1
2024-03-29T23:17:16Z error pulling image: Error response from daemon: Get "https://registry-1.docker.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
PodID: imw1s5o3l2tyhs still error
2024-03-30T09:06:19Z error pulling image: Error response from daemon: Head "https://registry-1.docker.io/v2/runpod/stable-diffusion/manifests/web-ui-10.2.1": Get "https://auth.docker.io/token?scope=repository%3Arunpod%2Fstable-diffusion%3Apull&service=registry.docker.io": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
2024-03-30T09:06:36Z create container runpod/stable-diffusion:web-ui-10.2.1
2024-03-30T09:06:44Z pending image pull runpod/stable-diffusion:web-ui-10.2.1
2024-03-30T09:07:07Z web-ui-10.2.1 Pulling from runpod/stable-diffusion
2024-03-30T09:07:07Z Digest: sha256:315f00cc67b03de0e04f33f9a3650a3bd019cd4a48c2c7d95060bc5d140e619b
PodID: p2oe90065eiswb still error
I have many issue on zone EU-SE-1
Any admin help me check please
@Satish @Papa Madiator
This issue effect many user on our production. Please fix this issue
@Polar help me check it please
yesterday I raise this issue but still no one support 😦
Hi, I'm not too sure what the issue is here, can you contact support on our site, should be on the bottom right of our dashboard (the purple icon)
I have contact but it reply as a bot. No one support
@quang son
I don't see the pod p2oe90065eiswb now. In the logs, it says it failed to download the Docker image. The server where the pod tried to create now has other images running fine. Did you try creating the pod again in any other region? I will try running the same image on another machine and let you know if I find anything