running synchronous workload on serverless
I want to run comfyUI synchronously using my web-server wrapper: https://github.com/ImmarKarim/comfyui-api-wrapper . it has functionalities that are very useful to me.
my backend sends an API request to the above wrapper and expects the response in the same connection. to get this working on runpod serverless, can you please help clarify:
1) can I deploy this wrapper out-of-the-box and expect it to work in runpod's serverless ecosystem?
2) I used the /runsyc endpoint & it still responded with "status: in_queue" (async-like) - why did this happen? isn't /runsyc completely synchronous?
3) if I were to use the loadbalancer-type endpoint, will it scale up the number of instances in following condition: # of requests > # of running instances ? will it buffer the request while scaling up or just outright drop it?
4 Replies
for the wrapped if its a rest api, then use load balancer serverless, make sure you define /ping in the api, need a health check
you can check this example
https://docs.runpod.io/serverless/load-balancing/build-a-worker
Runpod Documentation
Build a load balancing worker - Runpod Documentation
Learn how to implement and deploy a load balancing worker with FastAPI.
Thanks for responding.
If I go with the load balancer serverless:
1) Will it scale up the number of instances as traffic increases and all running instances are used up? Basically autoscaling based on number of requests?
2) And in that case, will it buffer the request while a new instance comes up or should I expect serverless to return 429 (or drop the request)?
Unknown User•2w ago
Message Not Public
Sign In & Join Server To View