Idle time: High Idle time on server but not getting tasks from queue
I'm testing servers with high Idle time to keep alive and get new tasks, but the worker is showing idle and finished but not getting new tasks from the Queue.
Is there any event or state I need to add to the handler?
8 Replies
My Delay time is 60 seconds but as you can see on each execution the request execution time is 120 seconds besides the fact that the task is only 15s, but the server still hanging with the same task
Share ur endpoint ID? Also im unable to replicate on my own endpoint. I think either way, u dont need to have high idle time if the idle time is > than cold start time isnt worth it.
But as long as u return it should terminate the job? I guess when the job is done and return do u have logs on the timestamps?
That is insanely weird that all ur execution times are perfect inline tho due to this idle
Yeah i set my idle time to 30 seconds, and sent like 30 requests to it, and have varying execution times. Maybe can also scale ur max to 0 and back to ur max? just to manually reset it all? seems weird
yeah, I'll do that, thanks
Scaling back to 0 and max solved thanks @justin [Not Staff]
Yeah, there is something wrong with the handling of throttled workers, one of my other endpoints was completely throttled for 2 days and when I scaled it down to zero and back up again, it came right again and doesn't have any throttled workers anymore.
I think his issue was he set an idle time but his workers were waiting for idle time to complete before pulling jobs off the queue so he got a range of perfect execution times of 120 seconds cause they were waiting for his idle time lol. weird
seems to be working now, probably cached dockerfile? not sure
the execution time is back to normal, from a batch of 20 tasks only one failed, and that must be GPU because there is no log output
and yes, a lot of throttled workers pushes the cold start servers to unreliable state