Same problem here, `Consumer Delay` is

Same problem here, Consumer Delay is about 32 seconds. My configuration:

max_batch_size = 1
max_batch_timeout = 0
max_retries = 6
dead_letter_queue = "dead-letter-staging"

max_batch_size = 1
max_batch_timeout = 0
max_retries = 6
dead_letter_queue = "dead-letter-staging"

Queue ID 2c9ac00a0def47b6bbe32295f79a50cd AccountId 75d8d68ecf8f9e8155f1ccc7dad82e00

6 Replies

Sid•5mo ago

Note that retries are included in this delay. Is your DLQ also failing to process your message?

toadfansOP•5mo ago

No, my task did not fail, so there is no need for retries. I just triggered the send queue twice in the same request. Will the second one wait for the first one to finish before being consumed?

toadfansOP•5mo ago

I have a task that runs for about 60 seconds. While this task is running in batch, I noticed that subsequent tasks remain in a waiting state (Queued messages) until the old one is completed.This means that new tasks may have to wait nearly 60 seconds to start executing, which is unacceptable. I also found this point in the documentation: https://developers.cloudflare.com/queues/configuration/consumer-concurrency/#why-are-my-consumers-not-autoscaling How can I configure it so that new tasks trigger immediately?

Cloudflare Docs

Consumer concurrency · Cloudflare Queues docs

Consumer concurrency allows a consumer Worker processing messages from a queue to automatically scale out horizontally to keep up with the rate that messages are being written to a queue.

Sid•5mo ago

Yeah so as it stands right now, Queue autoscaling decisions are only made at the end of a batch. What you might be seeing is your Queue running at a concurrency of 1, and then receving a 60s task to process. We won't scale the Queue up until this batch finishes. Once it does, your Queue would scale up and you'd be able to handle multiple tasks at the same time, then it would keep going up from there. Improving this behaviour is something we'd like to do pretty soon though

Unknown User•5mo ago

Message Not Public

Sid•5mo ago

If you do end up using waitUntil, make sure you enable this compat flag: https://developers.cloudflare.com/workers/configuration/compatibility-flags/#queue-consumers-dont-wait-for-ctxwaituntil-to-resolve we intend on making this the default, but it isn’t right now

Cloudflare Docs

Compatibility flags · Cloudflare Workers docs

Opt into a specific features of the Workers runtime for your Workers project.

Gaming

Programming

Same problem here, `Consumer Delay` is

Did you find this page helpful?