R

Runpod•6d ago

sintecs

Serverless: nightly workers stuck “initializing” on SOME endpoints (delay time spikes)

I’m seeing a very consistent Serverless scaling issue for ~the last week, but IMPORTANT: this does NOT happen to all my endpoints — only to some of them. Other endpoints remain stable during the same time window.

Example affected endpoint ID: owufvdufc1h5x2

Time window (daily):

• 18:00–23:00 UTC

Symptoms (only on some endpoints, during that window):

• The endpoint often can’t maintain the required number of workers (sometimes it drops to 0)
• Workers remain in “initializing” for a long time
• “Delay time” jumps from ~1s to up to ~30 minutes at peak
  Outside this window, everything works normally.

My worker is packaged in Docker and pulled from Docker Hub. I know “initializing” can include downloading the Docker image to a new worker, and requests arriving during initialization will be delayed.

Question: Can I find out the root cause of these nightly failures on only some endpoints? Any recommended fixes/settings (min workers, caching models, scaling config), and is there any specific log/request history I should collect from the Console Workers tab to help you investigate?