How to set max concurrency per worker for a load balancing endpoint?
I'm trying to configure the maximum concurrency for each worker on my serverless load balancing endpoint, but I can't seem to find the setting in the new UI.
4 Replies
Unknown User•3w ago
Message Not Public
Sign In & Join Server To View
@nullable , were you initially setting up your endpoint. When creating an endpoint on Serverless, we do the calculation for you. Once then endpoint is setup, you can then edit the endpoint and adjust as needed.
I mean in the queue based serverless worker, there is a concurrent handler that I can control concurrency for each worker https://docs.runpod.io/serverless/workers/concurrent-handler, I want to know how to do same control for load balancing workers
Runpod Documentation
Welcome to Runpod - Runpod Documentation
Explore our guides and examples to deploy your AI/ML application on Runpod.
Unknown User•3w ago
Message Not Public
Sign In & Join Server To View