Cloudflare Developers•3y ago

How do you get that response back to the user? e.g. what medium?

MMatt Silverlock How do you get that response back to the user? e.g. what medium?

Matt SilverlockOP•3/21/23, 6:44 PM

(does the user/client poll/retry?)

MMoretti I opened a support ticket yesterday. I can't enable queues on my enterprise acco...

Matt SilverlockOP•3/22/23, 7:56 PM

Can you DM me your account ID / reach out to your account team? Why can’t you enable it? (what’s the error?)

MMoretti I opened a support ticket yesterday. I can't enable queues on my enterprise acco...

Matt SilverlockOP•3/22/23, 7:56 PM

Also: queues

Unsmart•3/22/23, 10:39 PM

What happens when you press agree?

Matt SilverlockOP•3/22/23, 11:01 PM

(We addressed this via DMs. They’re on a legacy subscription with ~14 accounts. Addressing it directly)

KKlowner ah ha, limited to one active consumer!

Emilio•3/23/23, 6:38 PM

Bingo! Our Queues can only have 1 consumer at a time.

Jacob•3/24/23, 2:43 AM

When using ack() on each messages, under what conditions will it retry the non ack’d messages? Is it only if the worker crashes or retry() is called?

Jacob•3/24/23, 2:58 AM

From reading the change log, it looks like non ack’d messages will only be retried when the consumer crashes (throws an unhandled error.)
I’ll add some retry()’s to my consumer that try/catches everything

Hello, I’m Allie!•3/24/23, 2:39 PM

What’s the average time from submitting a task to it being processed? Assuming nothing else is currently being processed

HHello, I’m Allie!What’s the average time from submitting a task to it being processed? Assuming n...

Emilio•3/24/23, 3:40 PM

What do you mean by task? A message? As for processing, do you mean how long it takes the Queue broker to ingest the message? If so, that's done as soon as you send the message, plus whatever latency between the producer worker and the queue broker.

EEmilio What do you mean by task? A message? As for processing, do you mean how long it ...

Hello, I’m Allie!•3/24/23, 3:42 PM

Sorry, haven't touched Queues as much, so not 100% up on the terms. I meant, once a producer creates a task, how long for it to be piped into a consumer?

HHello, I’m Allie!Sorry, haven't touched Queues as much, so not 100% up on the terms. I meant, onc...

Emilio•3/24/23, 3:44 PM

That depends on a lot of variables! How large is your backlog? This will lead to a longer consumer delay. In the case where your Queue has low throughput, and you have a configured a long max_wait_timemax_wait_time then it might take the length of that delay.

EEmilio That depends on a lot of variables! How large is your backlog? This will lead to...

Hello, I’m Allie!•3/24/23, 3:45 PM

Assuming 0 backlog, is it started immediately, or does it wait a little while for more tasks to arrive?

Hello, I’m Allie!•3/24/23, 3:48 PM

By max_wait_timemax_wait_time, do you mean max_batch_timeoutmax_batch_timeout?

HHello, I’m Allie!Assuming 0 backlog, is it started immediately, or does it wait a little while fo...

Emilio•3/24/23, 3:48 PM

Yeah, in that case it will depend on what you configured for max_wait_time_msmax_wait_time_ms -- there is also some latency, which will vary based on how far away the producer worker is from the queue broker (they are not guaranteed to run on the same colo)

HHello, I’m Allie!By `max_wait_time`, do you mean `max_batch_timeout`?

Emilio•3/24/23, 3:49 PM

https://developers.cloudflare.com/api/operations/queue-create-queue-consumer
max_wait_time_msmax_wait_time_ms is a setting you configure on a consumer

Cloudflare API Documentation

Interact with Cloudflare's products and services via the Cloudflare API

EEmilio https://developers.cloudflare.com/api/operations/queue-create-queue-consumer `m...

Hello, I’m Allie!•3/24/23, 3:50 PM

Oh, that's separate from https://developers.cloudflare.com/queues/platform/configuration/#consumer, that's why...

Cloudflare Queues - Configuration · Cloudflare Queues

Cloudflare Queues can be configured using Wrangler, the command-line interface for Cloudflare’s Developer Platform, which includes Workers, R2, and …

EEmilio Yeah, in that case it will depend on what you configured for `max_wait_time_ms` ...

Hello, I’m Allie!•3/24/23, 3:51 PM

What's the default value for max_wait_time_msmax_wait_time_ms? Can it be set to 0?

Emilio•3/24/23, 3:53 PM

The default is 5000ms and yes you can set it to 0

EEmilio The default is 5000ms and yes you can set it to 0

Hello, I’m Allie!•3/24/23, 3:55 PM

Nice nice!

HHello, I’m Allie!Oh, that's separate from https://developers.cloudflare.com/queues/platform/confi...

Matt SilverlockOP•3/25/23, 11:43 AM

It’s the same setting, if not clear.

MMatt Silverlock It’s the same setting, if not clear.

Hello, I’m Allie!•3/25/23, 11:55 AM

Sounds good! And just to make sure, Consumers are guaranteed to not share resources with Producers, right?

Hello, I’m Allie!•3/25/23, 11:56 AM

At least if it isn’t the same script(like a DO being the producer)?

HHello, I’m Allie!Sounds good! And just to make sure, Consumers are guaranteed to not share resour...

Matt SilverlockOP•3/25/23, 11:58 AM

Your producer(s) can be any script. Even if your consumer is defined in the same script, they are invoked entirely separately

Matt SilverlockOP•3/25/23, 11:58 AM

No different from fetch() and scheduled() in the same script.

MMatt Silverlock Your producer(s) can be any script. Even if your consumer is defined in the same...

Hello, I’m Allie!•3/25/23, 12:00 PM

No, I meant like, can they end up sharing a thread/memory? I have a task that is quite resource intensive, so I don’t want my Producers/Consumers to run in the same space and slow down/run out of memory

HHello, I’m Allie!No, I meant like, can they end up sharing a thread/memory? I have a task that is...

Matt SilverlockOP•3/25/23, 12:03 PM

No, they don’t share the same invocation (isolate). There is no relation between the two. The “producer” isn’t really a thing to model - it’s just an API.

Matt SilverlockOP•3/25/23, 12:04 PM

a producer isn’t a producer: it’s a fetch or scheduled Worker calling an API (send/sendBatch)

Hello, I’m Allie!•3/25/23, 12:04 PM

Oh, sorry. By producer, I meant the isolate calling send/sendBatch(which I guess isn’t actually called the producer)

cb•3/26/23, 3:34 AM

Is there any progress on accessing a Queue from outside of Cloudflare?
Similar to https://docs.upstash.com/qstash

Getting Started | Upstash: Documentation

Introduction

Ccb Is there any progress on accessing a Queue from outside of Cloudflare? Similar t...

zegevlier•3/26/23, 9:02 AM

Not at the moment (I believe), but that is something that is planned as per queues

Zzegevlier Not at the moment (I believe), but that is something that is planned as per http...

Matt SilverlockOP•3/26/23, 10:10 AM

This is where we stand right now, yep.

Ccb Is there any progress on accessing a Queue from outside of Cloudflare? Similar t...

Matt SilverlockOP•3/26/23, 10:11 AM

Are you looking for push (write to) or pull (consume from?). HTTP Pull -- being able to pull from any HTTP-speaking client -- is being worked on.

cb•3/26/23, 10:12 AM

Thanks! Yes @Matt CF Worker Push to Queue -> HTTP Pull to process queue

Aakira.eth any miniflare functions for reading queues and adding data to queues

Emilio•3/28/23, 3:53 PM

Miniflare has rudimentary support for Queues, you can publish and consume messages. Expect full-featured and production-like support in Miniflare v3.

Aakira.eth If I am testing just a consumer locally any way to add messages to the queues?

Emilio•3/29/23, 2:51 PM

Yes miniflare will do that for you automatically!

How do you get that response back to the user? e.g. what medium?

Similar Threads

Similar Threads

Similar Threads