I have wrote async handler. Messages are realy small, about several kilobites
async for msg in search.run_search_generator(request): start_time = time.perf_counter() yield msg print("elapsed_time", (time.perf_counter() - start_time) * 1000)
And I have measured how much time every yield from the job takes and it's about 160 ms. It's quite a lot for my use case and increases time twice for the whole job execution. What are my options ?
Continue the conversation
Join the Discord to ask follow-up questions and connect with the community
R
Runpod
We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!