Asynchronous Batch API

Maybe this is your answer: "Batch processing is useful for large workloads such as summarization or embeddings when there is no human interaction. Using the batch API will guarantee that your requests are fulfilled eventually, rather than erroring out if Cloudflare does have enough capacity at a given time" https://developers.cloudflare.com/workers-ai/features/batch-api/
Cloudflare Docs
Asynchronous batch processing lets you send a collection (batch) of inference requests in a single call. Instead of expecting immediate responses for every request, the system queues them for processing and returns the results later.
Asynchronous Batch API
Was this page helpful?