Queue consumer concurrency is going to land in just under two weeks: https://developers.cloudflare.c

Queue consumer concurrency is going to land in just under two weeks: https://developers.cloudflare.com/queues/changelog/#consumer-concurrency-upcoming

If you have a use-case where you want to limit concurrency, you’ll want to cap it via the dashboard or wrangler before we enable autoscaling.

Changelog · Cloudflare Queues

Queue consumers will soon automatically scale up concurrently as a queues’ backlog grows in order to keep overall message processing latency down. …

MMatt Silverlock Queue consumer concurrency is going to land in just under two weeks: https://dev...

Erisa•3/21/23, 8:18 AM

@Queues

zegevlier•3/21/23, 1:06 PM

Yup, that's totally fine

MMoretti Thinking about this use case: fetch handler receives request, process and needs...

Matt SilverlockOP•3/21/23, 4:30 PM

Are you trying to serve a real-time response back to a user? If so, how do you intend to do that?

Matt SilverlockOP•3/21/23, 4:31 PM

Queues is best at async work. If you're relying on returning a live response to a client, then it may not make sense, but you'll also need to manage your upstream API (in this case, OpenAI's) rate limits as well vs. your own user demand.

Kai•3/21/23, 5:27 PM

doing something like fetch -> upgrade WS -> push WS to DO & enqueue work item w/ DO Id -> ... queue ... -> eventual return via DO WS -> close WS works decently imo. Can reuse the WS / DO as needed & is decently efficient. It'll be a bit more expensive depending on how you implement it (DOs with WS are never evicted so you pay) but it's possible to do & even implement timeouts and that sort of thing.

MMoretti In our case I want to close the connection with the user and then reply async wi...

Matt SilverlockOP•3/21/23, 6:44 PM

How do you get that response back to the user? e.g. what medium?

MMatt Silverlock How do you get that response back to the user? e.g. what medium?

Matt SilverlockOP•3/21/23, 6:44 PM

(does the user/client poll/retry?)

MMoretti I opened a support ticket yesterday. I can't enable queues on my enterprise acco...

Matt SilverlockOP•3/22/23, 7:56 PM

Can you DM me your account ID / reach out to your account team? Why can’t you enable it? (what’s the error?)

MMoretti I opened a support ticket yesterday. I can't enable queues on my enterprise acco...

Matt SilverlockOP•3/22/23, 7:56 PM

Also: queues

Unsmart•3/22/23, 10:39 PM

What happens when you press agree?

Matt SilverlockOP•3/22/23, 11:01 PM

(We addressed this via DMs. They’re on a legacy subscription with ~14 accounts. Addressing it directly)

KKlowner ah ha, limited to one active consumer!

Emilio•3/23/23, 6:38 PM

Bingo! Our Queues can only have 1 consumer at a time.

Jacob•3/24/23, 2:43 AM

When using ack() on each messages, under what conditions will it retry the non ack’d messages? Is it only if the worker crashes or retry() is called?

Jacob•3/24/23, 2:58 AM

From reading the change log, it looks like non ack’d messages will only be retried when the consumer crashes (throws an unhandled error.)
I’ll add some retry()’s to my consumer that try/catches everything

Hello, I’m Allie!•3/24/23, 2:39 PM

What’s the average time from submitting a task to it being processed? Assuming nothing else is currently being processed

HHello, I’m Allie!What’s the average time from submitting a task to it being processed? Assuming n...

Emilio•3/24/23, 3:40 PM

What do you mean by task? A message? As for processing, do you mean how long it takes the Queue broker to ingest the message? If so, that's done as soon as you send the message, plus whatever latency between the producer worker and the queue broker.

EEmilio What do you mean by task? A message? As for processing, do you mean how long it ...

Hello, I’m Allie!•3/24/23, 3:42 PM

Sorry, haven't touched Queues as much, so not 100% up on the terms. I meant, once a producer creates a task, how long for it to be piped into a consumer?

HHello, I’m Allie!Sorry, haven't touched Queues as much, so not 100% up on the terms. I meant, onc...

Emilio•3/24/23, 3:44 PM

That depends on a lot of variables! How large is your backlog? This will lead to a longer consumer delay. In the case where your Queue has low throughput, and you have a configured a long max_wait_timemax_wait_time then it might take the length of that delay.

EEmilio That depends on a lot of variables! How large is your backlog? This will lead to...

Hello, I’m Allie!•3/24/23, 3:45 PM

Assuming 0 backlog, is it started immediately, or does it wait a little while for more tasks to arrive?

Hello, I’m Allie!•3/24/23, 3:48 PM

By max_wait_timemax_wait_time, do you mean max_batch_timeoutmax_batch_timeout?

HHello, I’m Allie!Assuming 0 backlog, is it started immediately, or does it wait a little while fo...

Emilio•3/24/23, 3:48 PM

Yeah, in that case it will depend on what you configured for max_wait_time_msmax_wait_time_ms -- there is also some latency, which will vary based on how far away the producer worker is from the queue broker (they are not guaranteed to run on the same colo)

HHello, I’m Allie!By `max_wait_time`, do you mean `max_batch_timeout`?

Emilio•3/24/23, 3:49 PM

https://developers.cloudflare.com/api/operations/queue-create-queue-consumer
max_wait_time_msmax_wait_time_ms is a setting you configure on a consumer

Cloudflare API Documentation

Interact with Cloudflare's products and services via the Cloudflare API

EEmilio https://developers.cloudflare.com/api/operations/queue-create-queue-consumer `m...

Hello, I’m Allie!•3/24/23, 3:50 PM

Oh, that's separate from https://developers.cloudflare.com/queues/platform/configuration/#consumer, that's why...

Cloudflare Queues - Configuration · Cloudflare Queues

Cloudflare Queues can be configured using Wrangler, the command-line interface for Cloudflare’s Developer Platform, which includes Workers, R2, and …

EEmilio Yeah, in that case it will depend on what you configured for `max_wait_time_ms` ...

Hello, I’m Allie!•3/24/23, 3:51 PM

What's the default value for max_wait_time_msmax_wait_time_ms? Can it be set to 0?

Emilio•3/24/23, 3:53 PM

The default is 5000ms and yes you can set it to 0

EEmilio The default is 5000ms and yes you can set it to 0

Hello, I’m Allie!•3/24/23, 3:55 PM

Nice nice!

HHello, I’m Allie!Oh, that's separate from https://developers.cloudflare.com/queues/platform/confi...

Matt SilverlockOP•3/25/23, 11:43 AM

It’s the same setting, if not clear.

MMatt Silverlock It’s the same setting, if not clear.

Hello, I’m Allie!•3/25/23, 11:55 AM

Sounds good! And just to make sure, Consumers are guaranteed to not share resources with Producers, right?

Hello, I’m Allie!•3/25/23, 11:56 AM

At least if it isn’t the same script(like a DO being the producer)?

HHello, I’m Allie!Sounds good! And just to make sure, Consumers are guaranteed to not share resour...

Matt SilverlockOP•3/25/23, 11:58 AM

Your producer(s) can be any script. Even if your consumer is defined in the same script, they are invoked entirely separately

Matt SilverlockOP•3/25/23, 11:58 AM

No different from fetch() and scheduled() in the same script.

MMatt Silverlock Your producer(s) can be any script. Even if your consumer is defined in the same...

Hello, I’m Allie!•3/25/23, 12:00 PM

No, I meant like, can they end up sharing a thread/memory? I have a task that is quite resource intensive, so I don’t want my Producers/Consumers to run in the same space and slow down/run out of memory

HHello, I’m Allie!No, I meant like, can they end up sharing a thread/memory? I have a task that is...

Matt SilverlockOP•3/25/23, 12:03 PM

No, they don’t share the same invocation (isolate). There is no relation between the two. The “producer” isn’t really a thing to model - it’s just an API.

Queue consumer concurrency is going to land in just under two weeks: https://developers.cloudflare.c

Similar Threads

Similar Threads

Similar Threads