Dockerless dev and deploy, async handler need to use async ?

handler.py in HelloWorld project, there is not 'async' before def handler(job): . But in serverless endpoint, there are Run and RunSync . So if I want to use async handler, Is it necessary to go like this ? Or am I misunderstanding something? async def async_generator_handler(job): for i in range(5):
output = f"Generated async token output {i}" yield output for a large language model await asyncio.sleep(1) runpod.serverless.start( { "handler": async_generator_handler,
"return_aggregate_stream": True, } )
ashleyk44d ago
I don't understand the question, but here are docs for an async handler. https://docs.runpod.io/serverless/workers/handlers/handler-async By the way run and runsync are independent of your handler type. You can still use run even if your handler is not async.
Asynchronous Handler | RunPod Documentation
RunPod supports the use of asynchronous handlers, enabling efficient handling of tasks that benefit from non-blocking operations. This feature is particularly useful for tasks like processing large datasets, interacting with APIs, or handling I/O-bound operations.
fanbing44d ago
Understood . For my ai generate image , use async handler is a better choice ? Given that serverless has already implemented the creation of a new worker for each new request, would it be sufficient to use a synchronous handler, without the need for an asynchronous handler? Since I want customers to receive their portraits as quickly as possible, and each worker's GPU is dedicated solely to processing the task of generating that customer's portrait,not for other customer.
Want results from more Discord servers?
Add your server
More Posts