Hello, I have been seeing an increase in delay time while workers boot up, even while using Flashboot. I am using 1.5.3 which seems to have improved it a bit but not significantly. Having said that, is there an API that can be called to boot up the worker if there is an incoming request in let's say 5 seconds. This would ensure that the worker is warm and ready when it arrives. Does idle timeout perform a similar function?
Continue the conversation
Join the Discord to ask follow-up questions and connect with the community
R
Runpod
We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!