Hi, I am using the batch api for the

Hi, I am using the batch api for the Markdown API; I am getting a 50% error rate when converting images to Markdown. I am using 10/batch and 10/concurrent requests. example errors I am getting:

{'name': '123_image_5.png', 'mimeType': 'image/png', 'format': 'error', 'error': 'Error converting image to markdown text: 3040: Capacity temporarily exceeded, please try again.'}

{'name': '123_image_5.png', 'mimeType': 'image/png', 'format': 'error', 'error': 'Error converting image to markdown text: 3040: Capacity temporarily exceeded, please try again.'}

RazmjooOP•11/22/25, 5:11 PM

the queueRequest=truequeueRequest=true doesn't seems to be compatible with this endpoint, and if I send indiviual request or 1/per batch it gonna take very long (approax 2.25million items need to be processed);

RazmjooOP•11/22/25, 5:11 PM

what is the point of batch processing if we have low limits; or single processing if we get rate limited

RazmjooOP•11/22/25, 5:16 PM

Additionally, the endpoint returns 200200 when some of the files fail, and some of them succeed, so the AI gateway won't rely on performing a retry since it is considered a success; e.g, 43349ae4-23ad-4201-bbd6-b67e903bf25043349ae4-23ad-4201-bbd6-b67e903bf250 request 6/10 succeed and four failed, but the gateway considers this as a success.

ngghghfjgnghgngnhhghngn•11/22/25, 9:29 PM

Hi. What do you mean by 'batch api'? Is it the fact that you can convert several files in the same request? Or is it related to https://developers.cloudflare.com/workers-ai/features/batch-api/

For context, even though the Markdown Conversion service lives under the Workers AI product, it's a separate service. Please see the relevant documentation: https://developers.cloudflare.com/workers-ai/features/markdown-conversion/

Cloudflare Docs

Asynchronous Batch API

Asynchronous batch processing lets you send a collection (batch) of inference requests in a single call. Instead of expecting immediate responses for every request, the system queues them for processing and returns the results later.

Cloudflare Docs

Markdown Conversion

Markdown is essential for text generation and large language models (LLMs) in training and inference because it can provide structured, semantic, human, and machine-readable input. Likewise, Markdown facilitates chunking and structuring input data for better retrieval and synthesis in the context of RAGs, and its simplicity and ease of parsing a...

RRazmjoo the `queueRequest=true` doesn't seems to be compatible with this endpoint, and i...

ngghghfjgnghgngnhhghngn•11/22/25, 9:31 PM

queueRequestqueueRequest is a request parameter available when calling Workers AI models. The Markdown Conversion API uses a different set of endpoints and it does not support this feature (yet

)

RRazmjoo Additionally, the endpoint returns `200` when some of the files fail, and some o...

ngghghfjgnghgngnhhghngn•11/22/25, 9:37 PM

The endpoint always returns a 200200 response right now but the success of failure of a request can be seen in the response's

success

success

attribute. These requests do not go through AI Gateway: they are not Workers AI model invocations in the normal sense, as it's a totally different set of endpoints.

RazmjooOP•11/22/25, 9:50 PM

I do multiple files with one request... i think queueRequest is universal. also I need the gateway for caching and request retries, etc... but i guess it won't work on this

RazmjooOP•11/22/25, 9:50 PM

for now i do manually type check if it's an error and do manual retry for the ones are failing...

RRazmjoo I do multiple files with one request... i think queueRequest is universal. also ...

ngghghfjgnghgngnhhghngn•11/22/25, 10:31 PM

queueRequestqueueRequest is not supported on the Markdown Conversion API endpoints. We have that feature on the roadmap, although with no ETA

ngghghfjgnghgngnhhghngn•11/22/25, 10:32 PM

While this might not entirely fit your needs, regarding "caching and request retries": since Markdown Conversion doesn't go through AI Gateway, would using a Workflow (provides durable execution, steps can be retried automatically) suit your use case?

ngghghfjgnghgngnhhghngn•11/22/25, 10:33 PM

Also, for the future, link you account so we can provide better assistance

ngghghfjgnghgngnhhghngn•11/22/25, 10:33 PM

?link

SuperHelpflare•11/22/25, 10:33 PM

Please use the /link command to authenticate with your Cloudflare account and link it to your Discord user. This helps Cloudflare employees access your details more quickly and provide support. It also makes future support requests easier, since your account will already be linked.

RazmjooOP•11/22/25, 10:36 PM

I think workflow would work, but last time I remember it was single threaded; is it still the case?

ngghghfjgnghgngnhhghngn•11/22/25, 10:40 PM

it is still single threaded since it runs on a Javascript runtime. Multiple requests can run concurrently, though: there is await Promise.all([...])await Promise.all([...]) that "runs all promises at the same time". A common use case is to run several steps concurrently this way (since they return promises that can be awaited together)

RazmjooOP•11/22/25, 10:42 PM

this wouldn't work if it doesn't support semaphore; it will be 2 million tasks if use singular and couple hundreds if i request them at the same time; so we need to set concurrency limit to make it work, but also don't get rate limited, or DoS the service

RazmjooOP•11/22/25, 10:42 PM

let me check await Promise.all([...])await Promise.all([...]) maybe something there ...

RRazmjoo this wouldn't work if it doesn't support semaphore; it will be 2 million tasks i...

ngghghfjgnghgngnhhghngn•11/22/25, 10:49 PM

I believe this can be handled well by Workflows: they recently made some changes that should accomodate "bigger" workloads. If you have questions regarding whether your workload would fit the Workflows model, don't hesitate to ask in #workflows

Hi, I am using the batch api for the

Similar Threads

Similar Threads

Similar Threads