Runpod•7mo ago

Docker pull randomly gets stuck

When launching a Pod on

EU-RO-1

EU-RO-1

EU-RO-1

EU-RO-1 (Nvidia 4090), the download speed drops to almost nothing at some point. This doesn't happen every time, sometimes the download completes without issues.

Pod ID:

dp302es4x8ad1b

dp302es4x8ad1b

dp302es4x8ad1b

dp302es4x8ad1b

Solution

Transitioned to serverless and it just works

Jump to solution

TomSOP•6/20/25, 12:45 PM

Restarting worked (pod ID q4t7idqcpjryg3q4t7idqcpjryg3). However, as we use Runpod as a part of an automated system that starts a pod, performs calculations, retrieves them and terminates it (relying on the fact that this is done), this makes it essentially unusable for our use as it can get randomly stuck.

TomSOP•6/20/25, 12:47 PM

If there is a way to store the Docker file in local storage, that would be amazing as it would significantly reduce the startup times and prevent this issue.

TTomS If there is a way to store the Docker file in local storage, that would be amazi...

Henky!!•6/20/25, 12:54 PM

If you hit the same machine its cached

Henky!!•6/20/25, 12:55 PM

They got a limited space to cache ofc

HHenky!!If you hit the same machine its cached

TomSOP•6/20/25, 12:56 PM

Ah, okay that's useful, but if I hit the network issue then the image hangs forever :/

TTomS Ah, okay that's useful, but if I hit the network issue then the image hangs fore...

Henky!!•6/20/25, 12:57 PM

It happens unfortunately, sometimes they dont even begin downloading

Henky!!•6/20/25, 12:58 PM

If you can make a check if they booted after X

TomSOP•6/20/25, 12:58 PM

Okay but that's an insane solution

TomSOP•6/20/25, 1:18 PM

I'll play around with Serverless, it feels like it fits the usage better now that I'm reading through it

TTomS I'll play around with Serverless, it feels like it fits the usage better now tha...

Henky!!•6/20/25, 2:25 PM

Serverless i find to restrictive compared to pods and those can have provision errors to

Henky!!•6/20/25, 2:25 PM

Pods usually work, except when a machine got glitched

HHenky!!Serverless i find to restrictive compared to pods and those can have provision e...

TomSOP•6/20/25, 3:55 PM

What do you mean provision errors?

Henky!!•6/20/25, 3:55 PM

Where a worker doesn't start

Henky!!•6/20/25, 3:55 PM

Same underlying machines after all

TomSOP•6/20/25, 3:56 PM

Ah ok. Yeah for me the only issues are the docker pulls

TomSOP•6/20/25, 3:56 PM

So caching the image should fix it

TTomS So caching the image should fix it

Henky!!•6/20/25, 4:00 PM

Potentiially a different docker repo may to

HHenky!!Potentiially a different docker repo may to

TomSOP•6/20/25, 4:02 PM

What do you mean different repo. The issue is the pull, not the repo itself

Henky!!•6/20/25, 4:03 PM

Instead of docker hub another host, in case its not runpod side

HHenky!!Instead of docker hub another host, in case its not runpod side

TomSOP•6/20/25, 4:05 PM

Ah, right, my mistake. Good idea, I'll see if that helps

Drew•7/8/25, 1:20 AM

I have a similar issue Tom, it's quite annoying how variable the start times can be. I am running into this frequently now if I choose to use runpod to scale my application.

Solution

DDrew I have a similar issue Tom, it's quite annoying how variable the start times can...

TomS•7/9/25, 4:43 PM

Transitioned to serverless and it just works

TomSOP•7/9/25, 4:44 PM

Would highly recommend doing so as well, it's pretty seamless

TomSOP•7/9/25, 4:45 PM

Doing so runs into a different issue, which is that sometimes all machines are used and so the request will take a while to be processed, but it's no longer 'if you get unlucky you're screwed'

Docker pull randomly gets stuck

Similar Threads

Similar Threads

Similar Threads