Request stuck in "IN_QUEUE" status
Hi,
I tried to setup a endpoint for "rembg" package using cog. It seems to run fine locally, but when I hit the serverless endpoint the request is always "IN_QUEUE" even though the logs shows that the server is up and running. Do you know what could be the issue?
17 Replies
Are all your workers throttled?
Ah you mean throttled by cpu utilization?
Why did you set max workers to 1? That is a bad idea, leave it at defaulrt of 3.
The default of 3 is the sane default, setting it to 1 will cause you endless issues, nightmares and a world of pain.
Ah I see, it was initially 3 but I was still facing the same issue and it was costing quite a bit so I switched it to 1 😓
You don't pay for max workers, only for active workers. With max workers you only pay for the requests.
Okay set it back to 3. But how do i resolve the issue of the request being stuck in "IN_QUEUE"?
Are all 5 workers throttled?
Nope
Are any of your requests actually processing successfully?
Nope
Then there is something wrong with your handler, click on one of the workers and check the logs.
hmm I see this. When I run the docker container locally, it processes the request successfully. Not unable to get it work on serverless
@mrmvp Mind sending your rp hander?
There is something erong with it!
Looks like the
/health-check API is not working and resulting in an HTTP 404 not found, and then looks like you are wrapping the health check API call in a try, except block and which tries to access the status field of the JSON response from /health-check but its failing because status does not exist within the HTTP 404 response and throws a KeyError exception as a result.I have been following this tutorial https://blog.runpod.io/replicate-cog-migration/
I have not changed anything in the handler code inside the
cog-workerRunPod Blog
Serverless | Migrating and Deploying Cog Images on RunPod Serverles...
Switching cloud platforms or migrating existing models can often feel like a Herculean task, especially when it necessitates additional developmental efforts. This guide aims to simplify this process for individuals who have deployed models via replicate.com or utilized the Cog framework. Through a few straightforward steps, you'll learn how