Are Docker images cached?
My Docker image registry is hosted on a VPS. If it were to go down, would the serverless workers no longer be able to start up or are the images cached inside Runpod?
7 Replies
They're cached on the servers you've used before, so as long as we don't need to pull it for the first time you're good to go :)
but there's no guarantee runpod will not decide to choose a new server where it's not already cached, is there?
Not particularly, you could limit the GPUs or regions you want us to serve your serverless jobs from but that may be a net negative if you get a lot of volume at once we won't be able to guarantee we can scale to match.
Really bad idea would be to change your settings so you scale up really quickly and send your endpoint with a bunch of no-op requests just to start the workers :thinkMan:
Actually... you might be able to get by with doing the opposite. Let your Pods stay alive a little longer so if you're under a load we just batch them to your existing Pods sooner than you risk spawning new ones.
that sounds very hacky :D
I think i'll just use github packages for this then which should be more reliable than my single vps
but I think it would be cool for Runpod to download the image only a single time and then store it in its own CDN for faster lookup and reliability
We're working on something like this
or perhaps a way to specify fallback images
You should hear from me in #📢｜announcements about it on or after the 28th of this month lol
I like this a lot :thinkMan: