Dockerless dev and deploy, async handler need to use async ?
handler.py in HelloWorld project, there is not 'async' before def handler(job): . But in serverless endpoint, there are Run and RunSync . So if I want to use async handler, Is it necessary to go like this ? Or am I misunderstanding something?
async def async_generator_handler(job):
for i in range(5):
output = f"Generated async token output {i}"
yield output
for a large language model
await asyncio.sleep(1)
runpod.serverless.start(
{
"handler": async_generator_handler,
"return_aggregate_stream": True,
}
)
async def async_generator_handler(job):
for i in range(5):
output = f"Generated async token output {i}"
yield output
for a large language model
await asyncio.sleep(1)
runpod.serverless.start(
{
"handler": async_generator_handler,
"return_aggregate_stream": True,
}
)
