How can I force concurrency? I have a

How can I force concurrency? I have a queue with different messages - sometimes can take several minutes to finish processing a single message.

welzOP•10/27/25, 8:11 AM

This is my current consumer config

{
  "queue": "my-queue",
  "max_batch_size": 1,
  "max_batch_timeout": 0,
  "max_retries": 5,
  "retry_delay": 10
}

{
  "queue": "my-queue",
  "max_batch_size": 1,
  "max_batch_timeout": 0,
  "max_retries": 5,
  "retry_delay": 10
}

It currently seems to wait until the first message completes before moving on to the next.

welzOP•10/27/25, 9:22 AM

I think this is why

James•10/27/25, 4:57 PM

you're intentionally forcing batches of 1 at a time, so you are depending on the scaling to kick in, but you can also just increase the batch size max and make sure your consumer can then parcel out the batches to run concurrently as it receives them, and then it falls to your app code to scale them out in paralle..

welzOP•10/27/25, 5:28 PM

I'm okay handling it in the worker - but I specifically want a separate invocation for each and don't want to process under the same invocation.

welzOP•10/27/25, 5:29 PM

I'm also concerned about hitting the "6 Simultaneous outgoing connections/request" limit

welzOP•10/27/25, 5:29 PM

https://developers.cloudflare.com/workers/platform/limits/

Cloudflare Docs

Limits

Cloudflare Workers plan and platform limits.

Wwelz I'm okay handling it in the worker - but I specifically want a separate invocati...

James•10/27/25, 6:16 PM

depends how you want to scale it, you can fire and forget workers from the consumer, but youd lose the queue retry guarantees on specific tasks, but fan out would be pretty easy. or push each item from the queue into another queue to consume per worker

welzOP•10/27/25, 6:21 PM

my only alternative was a bunch of queues - but it seems difficult to manage that

Wwelz my only alternative was a bunch of queues - but it seems difficult to manage tha...

James•10/27/25, 6:24 PM

not really, one queue and you can manage your own idempotency/fail handling within the worker you spawn off. concurrency has a price

How can I force concurrency? I have a

Similar Threads

Similar Threads

Similar Threads