Image Generation Stuck Until New Requests Are Sent
It typically takes 5–10 seconds to generate an image. However, sometimes a request doesn’t enter the processing queue until another request is sent.
This issue occurred multiple times today, and I recorded it. In the example I captured, my friend's first request stayed in the queue for an unusually long time. I asked him to send a second request to try and trigger the first one to start processing, but that didn’t work. When he submitted a third request, it finally caused the first request to begin processing, followed by the second, and then the third.
This issue occurs frequently and significantly impacts the user experience, as requests that should complete in 5–10 seconds end up taking several minutes.
13 Replies
It's not because the comfyui is still loading?
No, image generation is usually completed within 5–10 seconds. I recorded the video to show that requests only started processing after I triggered them by submitting a new one, which should not be necessary.
Yes and I see the same worker processed 3 requests after waiting for some moments
Can you check the logs of the workers
Maybe you should check if your worker is ready or still hasn't called the handler function
If not, I want you try this, just remove that specific worker that processed your requests ( from worker tab)
Then send 3-5 request immediately without any good delay between them
I showed the workers at 0:44: I had 2 running workers, 1 idle worker, and 2 throttled workers
This doesn’t happen all the time, but it pops up about once a week and definitely hurts the user experience. I even had a meeting with @Tim to try and show the issue live, it didn’t happen back then (which was a good thing haha)
We agreed that if it happened again and I could catch it on video, that would be helpful. So that’s what I’ve done now, so I hope this helps the team figure out what’s going wrong
i see
i noticed the scaling here is weird(ineffecient) tho, 1 worker end up serving your request but 3 end up running?
im not sure how to help without extra details, maybe logs would help. but sure contacting staff and letting them to debug on runpod site could help
That's right and I'm curious about why that happens. If 3 workers are running, I'd expect them to generate images separately but only 1 of them ended up processing the request
I only generated the images shown in the video today, and here are today's logs. Thank you @Jason for your time and help
Did you open a support ticket for this?
Maybe, can you Edit your endpoint then check if it's queue delay or request count (scaling type)
I think i can't see the logs well, but if you can try rerunning your request and check if your workers are in a ready state to receive and process your Request (add some prints after model loaded)
I noticed another issue related to serverless, and Tim told me to make a thread in serverless about it, so that's what I did with this one too
They are set like this
thanks for reporting this @Barış and thanks for your support already @Jason
I also encounter this issue lots of times!
The request sometime stuck in the queue, and all workers still stay in idle, doesnt pop the queue to work.
And this possibley be fixed when i send other requests to the queue, and make lots of request waiting in the queue, then the first worker start from idle to running