If we moved to support the ability to explicitly acknowledge (“ack”) messages within a batch, should

If we moved to support the ability to explicitly acknowledge (“ack”) messages within a batch, should the default behavior change to require you call

.ack()

.ack()

.ack()

.ack() on every message (or the entire batch) when handling it?

.retry()

.retry()

would exist for the cases where you want to explicitly, negatively acknowledge.

If yes (always require an explicit ack),

this post; if no, reply with why! Keep in mind I am familiar with other messaging systems; there’s not an agreed-upon approach here!

MMatt Silverlock If we moved to support the ability to explicitly acknowledge (“ack”) messages wi...

Matt SilverlockOP•1/12/23, 2:29 PM

(This would be a breaking change for existing beta users unless we grandfathered today’s behavior)

Rrelero90 Question about how consumers are able to interact with a queue: Does a consumer ...

Matt SilverlockOP•1/12/23, 8:11 PM

Today: consumers are effectively single threaded. No concurrency. If your rate of production >>> your consumer completion rate, you’ll generate a backlog. Best practice right now is to have a single Worker consumer per queue vs multiple queues aimed at the same consumer script.

Near future: automatic consumer concurrency that scales out based on consumption rate

Aamitarelli Hi all - how do I access a queue directly from the outside web, without invoking...

Matt SilverlockOP•1/12/23, 8:12 PM

You’ll need to write a Worker - accept a HTTP request or even a WebSocket —> call YOUR_QUEUE.send()YOUR_QUEUE.send(). In the longer term we’re exploring other on-ramps to support this natively

FFlorian I would like the default behavior be to always manually acknowledge a message or...

Matt SilverlockOP•1/13/23, 12:20 PM

Can you expand? So you want to be required to ack() every message, or the batch, but any left unacked on an error response also don’t get retried?

Aaltryne hey, on this topic, I have a very long GPU process (~10 minutes) , and I've basi...

Matt SilverlockOP•1/13/23, 12:23 PM

The consumer will eventually hit running time limits and/or (very likely) connection limits if you are trying to keep a HTTP connection open for several minutes. I’d think about how you can instead have the single threaded machine - your GPU - poll for work when it’s ready vs blocking on it for minutes at a time.

Unsmart•1/13/23, 12:24 PM

I personally like the way AWS does it. By default all messages are assumed to be successfully processed if the function does not error or return a list of message ids that should be retired. And if the function throws an error all messages are retried.

Unsmart•1/13/23, 12:25 PM

Just nice to not have to think about needing to do something to messages you don't want to retry

UUnsmart I personally like the way AWS does it. By default all messages are assumed to be...

Matt SilverlockOP•1/13/23, 12:49 PM

Right, that’s the default behavior (unless you enable client acknowledgment mode) and prevents big retry loops if you forget to call ack(). OTOH you can silently lose messages if you don’t expect it.

Unsmart•1/13/23, 12:52 PM

Ah so both modes will exist you're just asking about which should be the default?
I'd still say what it is currently over manual requiring acks as the default.

UUnsmart Ah so both modes will exist you're just asking about which should be the default...

Matt SilverlockOP•1/13/23, 12:55 PM

I’m not promising anything just yet

Right now we think that the most intuitive approach is:

1. Messages are implicitly ack’d (as they are today) if the handler returns successfully. Users with simple use cases don’t need to do anything special.
2. You can call

.ack()

.ack()

to make partial progress through a batch so that message is not retried if the handler errors
3. (Exists today) you can call retry()retry() or retryAll()retryAll() on a message or a batch to force a retry, even if the handler returns successfully

dave•1/13/23, 3:13 PM

Any recommendations on how to rate limit how often my queue can be processed? i.e. I’m working with a third party API that only allows X amount of operations per minute, so the queue will need to slow down firing off Workers during peak traffic.

Unsmart•1/13/23, 3:18 PM

Would probably be better to use DOs and alarms directly if you want that much control.

UUnsmart Would probably be better to use DOs and alarms directly if you want that much co...

dave•1/13/23, 3:42 PM

Can a consumer push items back into it's own producer queue? I feel like that might be easier than DOs.

Kai•1/13/23, 3:59 PM

has progress been made on retries yet? I read above that .retry() on a single message has been released? Any news about infinite retries / backoff?

Ddave Can a consumer push items back into it's own producer queue? I feel like that mi...

Emilio•1/13/23, 5:00 PM

I wouldn't recommend that. You'll be at risk of building up an insurmountable backlog in your queue. You could try writing the data to an R2 bucket and use cron triggers to spin up a worker on a set schedule that aligns with the rate limit of the external service.

EEmilio I wouldn't recommend that. You'll be at risk of building up an insurmountable ba...

dave•1/13/23, 5:00 PM

yeah, that was a concern that I'd shoot myself in the foot

KKai has progress been made on retries yet? I read above that .retry() on a single me...

Emilio•1/13/23, 5:04 PM

The current implementation allows you to manually retry messages/batches, but it's all or nothing. We're working on allowing you to selectively ack or nack messages in a batch, giving you a TCP-like sliding window that will give you a lot of control over your queue. We're still trying to figure out what the best and most intuitive interface for that might be, based on the feedback we get.

Kai•1/13/23, 5:06 PM

but it's all or nothing

so still only the whole batch?
What about retries? I heard that was planned at one point but never heard about it again

KKai > but it's all or nothing so still only the whole batch? What about retries? I h...

Emilio•1/13/23, 5:07 PM

Yes it's the whole batch, though you can tune your

max_batch_size

max_batch_size

to be as small as you'd like. We also allow you to configure how many times a given ~~batch~~ message can be retried, up to 100. https://developers.cloudflare.com/queues/limits/

Limits · Cloudflare Queues

1 Request adjustments to limits that conflict with your project goals by contacting Cloudflare. To increase a limit, complete the Limit Increase Request Form. 2 …

Aaltryne Is there way to catch the timeout somehow? Also, are the timeouts writted expli...

Matt SilverlockOP•1/13/23, 5:34 PM

Are you catching errors around the fetch() you are presumably making to your GPU machine? How are you otherwise seeing the JSON error? It could also be your end closing the condition if it’s idle too long as well. Holding a minutes long connection open and blocking on it is just going to be prone to failure, regardless of the message bus in-between.

kian•1/13/23, 5:39 PM

Are you calling

response.json()

response.json()

with your

fetch()

fetch()

call that's timing out?

kian•1/13/23, 5:45 PM

Unexpected end of JSON input

Unexpected end of JSON input

isn't the exact error I'd expect but any requests that are left waiting for a response for more than 100 seconds will just get a plain-text

error code: 524

error code: 524

response - or a pretty HTML page if the request headers indicate that's acceptable.

EEmilio Yes it's the whole batch, though you can tune your `max_batch_size` to be as sma...

Kai•1/13/23, 6:57 PM

OK, unfortunately nothing changed then. I'm using this to push a fluctuating load through a rate limited API, but in my experiments it was very inefficient due to constant requing

KKai OK, unfortunately nothing changed then. I'm using this to push a fluctuating loa...

Emilio•1/13/23, 7:31 PM

Would giving you control over the retry interval (min, max, backoff coefficient) help you with your usecase?

Kai•1/13/23, 7:39 PM

yes, very much!

dave•1/13/23, 7:49 PM

likewise here

Aaltryne Yeah looks like my response.json() was outside of the "response.ok" check. Than...

dave•1/13/23, 8:52 PM

Would Pub/Sub maybe work?

Aaltryne Yeah looks like my response.json() was outside of the "response.ok" check. Than...

dave•1/13/23, 8:55 PM

without giving out more info than you'd like, what's the gist of your problem? Large incoming files from users, slow ish non-scalable (to a degree) GPU instances that do processing, and retrying if that processing fails or times out?

dave•1/13/23, 8:55 PM

(I ask because something similar to that is on my TODO list.)

Aaltryne GPU being slow, and single threaded, sometimes taking 20+ minutes to respond And...

dave•1/13/23, 9:05 PM

I still think it doesn't solve the core problem, but have you looked at using ephemeral/spot instances that can scale with requests?

Aaltryne GPU being slow, and single threaded, sometimes taking 20+ minutes to respond And...

Matt SilverlockOP•1/13/23, 9:27 PM

A “pull” based approach (the GPU calls to an endpoint when it’s ready) would make a lot more sense here. The GPU asks the queue for work when it’s idle/finished a job. Queues, SQS, Pub/Sub, Pulsar, etc - this “hang on for minutes or longer for a response” is not really reliable in any system.

MMatt Silverlock A “pull” based approach (the GPU calls to an endpoint when it’s ready) would mak...

Matt SilverlockOP•1/13/23, 9:28 PM

This isn’t something we support today, but just making clear that trying to block on the GPU will cause you issues no matter what “queue” you use.

If we moved to support the ability to explicitly acknowledge (“ack”) messages within a batch, should

Similar Threads

Similar Threads

Similar Threads