I personally like the way AWS does it. By default all messages are assumed to be successfully proces

I personally like the way AWS does it. By default all messages are assumed to be successfully processed if the function does not error or return a list of message ids that should be retired. And if the function throws an error all messages are retried.

UnsmartOP•1/13/23, 12:25 PM

Just nice to not have to think about needing to do something to messages you don't want to retry

UUnsmart I personally like the way AWS does it. By default all messages are assumed to be...

Matt Silverlock•1/13/23, 12:49 PM

Right, that’s the default behavior (unless you enable client acknowledgment mode) and prevents big retry loops if you forget to call ack(). OTOH you can silently lose messages if you don’t expect it.

UnsmartOP•1/13/23, 12:52 PM

Ah so both modes will exist you're just asking about which should be the default?
I'd still say what it is currently over manual requiring acks as the default.

UUnsmart Ah so both modes will exist you're just asking about which should be the default...

Matt Silverlock•1/13/23, 12:55 PM

I’m not promising anything just yet

Right now we think that the most intuitive approach is:

1. Messages are implicitly ack’d (as they are today) if the handler returns successfully. Users with simple use cases don’t need to do anything special.
2. You can call .ack().ack() to make partial progress through a batch so that message is not retried if the handler errors
3. (Exists today) you can call retry()retry() or retryAll()retryAll() on a message or a batch to force a retry, even if the handler returns successfully

dave•1/13/23, 3:13 PM

Any recommendations on how to rate limit how often my queue can be processed? i.e. I’m working with a third party API that only allows X amount of operations per minute, so the queue will need to slow down firing off Workers during peak traffic.

UnsmartOP•1/13/23, 3:18 PM

Would probably be better to use DOs and alarms directly if you want that much control.

UUnsmart Would probably be better to use DOs and alarms directly if you want that much co...

dave•1/13/23, 3:42 PM

Can a consumer push items back into it's own producer queue? I feel like that might be easier than DOs.

Kai•1/13/23, 3:59 PM

has progress been made on retries yet? I read above that .retry() on a single message has been released? Any news about infinite retries / backoff?

Ddave Can a consumer push items back into it's own producer queue? I feel like that mi...

Emilio•1/13/23, 5:00 PM

I wouldn't recommend that. You'll be at risk of building up an insurmountable backlog in your queue. You could try writing the data to an R2 bucket and use cron triggers to spin up a worker on a set schedule that aligns with the rate limit of the external service.

EEmilio I wouldn't recommend that. You'll be at risk of building up an insurmountable ba...

dave•1/13/23, 5:00 PM

yeah, that was a concern that I'd shoot myself in the foot

KKai has progress been made on retries yet? I read above that .retry() on a single me...

Emilio•1/13/23, 5:04 PM

The current implementation allows you to manually retry messages/batches, but it's all or nothing. We're working on allowing you to selectively ack or nack messages in a batch, giving you a TCP-like sliding window that will give you a lot of control over your queue. We're still trying to figure out what the best and most intuitive interface for that might be, based on the feedback we get.

Kai•1/13/23, 5:06 PM

but it's all or nothing

so still only the whole batch?
What about retries? I heard that was planned at one point but never heard about it again

KKai > but it's all or nothing so still only the whole batch? What about retries? I h...

Emilio•1/13/23, 5:07 PM

Yes it's the whole batch, though you can tune your max_batch_sizemax_batch_size to be as small as you'd like. We also allow you to configure how many times a given ~~batch~~ message can be retried, up to 100. https://developers.cloudflare.com/queues/limits/

Limits · Cloudflare Queues

1 Request adjustments to limits that conflict with your project goals by contacting Cloudflare. To increase a limit, complete the Limit Increase Request Form. 2 …

Aaltryne Is there way to catch the timeout somehow? Also, are the timeouts writted expli...

Matt Silverlock•1/13/23, 5:34 PM

Are you catching errors around the fetch() you are presumably making to your GPU machine? How are you otherwise seeing the JSON error? It could also be your end closing the condition if it’s idle too long as well. Holding a minutes long connection open and blocking on it is just going to be prone to failure, regardless of the message bus in-between.

kian•1/13/23, 5:39 PM

Are you calling response.json()response.json() with your

fetch()

fetch()

call that's timing out?

kian•1/13/23, 5:45 PM

Unexpected end of JSON inputUnexpected end of JSON input isn't the exact error I'd expect but any requests that are left waiting for a response for more than 100 seconds will just get a plain-text error code: 524error code: 524 response - or a pretty HTML page if the request headers indicate that's acceptable.

EEmilio Yes it's the whole batch, though you can tune your `max_batch_size` to be as sma...

Kai•1/13/23, 6:57 PM

OK, unfortunately nothing changed then. I'm using this to push a fluctuating load through a rate limited API, but in my experiments it was very inefficient due to constant requing

KKai OK, unfortunately nothing changed then. I'm using this to push a fluctuating loa...

Emilio•1/13/23, 7:31 PM

Would giving you control over the retry interval (min, max, backoff coefficient) help you with your usecase?

Kai•1/13/23, 7:39 PM

yes, very much!

dave•1/13/23, 7:49 PM

likewise here

Aaltryne Yeah looks like my response.json() was outside of the "response.ok" check. Than...

dave•1/13/23, 8:52 PM

Would Pub/Sub maybe work?

Aaltryne Yeah looks like my response.json() was outside of the "response.ok" check. Than...

dave•1/13/23, 8:55 PM

without giving out more info than you'd like, what's the gist of your problem? Large incoming files from users, slow ish non-scalable (to a degree) GPU instances that do processing, and retrying if that processing fails or times out?

dave•1/13/23, 8:55 PM

(I ask because something similar to that is on my TODO list.)

Aaltryne GPU being slow, and single threaded, sometimes taking 20+ minutes to respond And...

dave•1/13/23, 9:05 PM

I still think it doesn't solve the core problem, but have you looked at using ephemeral/spot instances that can scale with requests?

Aaltryne GPU being slow, and single threaded, sometimes taking 20+ minutes to respond And...

Matt Silverlock•1/13/23, 9:27 PM

A “pull” based approach (the GPU calls to an endpoint when it’s ready) would make a lot more sense here. The GPU asks the queue for work when it’s idle/finished a job. Queues, SQS, Pub/Sub, Pulsar, etc - this “hang on for minutes or longer for a response” is not really reliable in any system.

MMatt Silverlock A “pull” based approach (the GPU calls to an endpoint when it’s ready) would mak...

Matt Silverlock•1/13/23, 9:28 PM

This isn’t something we support today, but just making clear that trying to block on the GPU will cause you issues no matter what “queue” you use.

Aaltryne Unless there's a way for me to consume messages from python code? or run a GPU p...

Matt Silverlock•1/13/23, 10:07 PM

No. In the future we will support polling via HTTP (HTTP GET to a /queue-name) and writing to a queue via HTTP POST but those are a little further out. Addressing consumer concurrency & partial batching are the two next up.

Aaltryne Yeah, makes sense Other queus (say Celery) the consumer can literally sit on th...

aarhus•1/14/23, 12:20 AM

How do you know the job has been completed by the GPU? If you can identity that, then you could use a DO - the queue (or directly) adds a job to the DO. The DO knows the status of the GPU tasks and only sends when the current one has finished. If you know roughly how long a task will take you can use a DO alarm to wake up and check the status

Aaltryne durable object?

Matt Silverlock•1/14/23, 12:21 AM

Yes.

Matt Silverlock•1/14/23, 12:21 AM

https://developers.cloudflare.com/workers/learning/using-durable-objects/

Using Durable Objects · Cloudflare Workers docs

Durable Objects provide low-latency coordination and consistent storage for the Workers platform through two features: global uniqueness and a transactional …

Matt Silverlock•1/14/23, 12:22 AM

Alarms: https://developers.cloudflare.com/workers/learning/using-durable-objects/#alarms-in-durable-objects

Using Durable Objects · Cloudflare Workers docs

Durable Objects provide low-latency coordination and consistent storage for the Workers platform through two features: global uniqueness and a transactional …

Aaarhus How do you know the job has been completed by the GPU? If you can identity that...

Matt Silverlock•1/14/23, 12:22 AM

Good idea.

MMatt Silverlock Good idea.

aarhus•1/14/23, 12:22 AM

it happens sometimes

aarhus•1/14/23, 3:57 PM

The DO will wake up.

I personally like the way AWS does it. By default all messages are assumed to be successfully proces

Similar Threads

Similar Threads

Similar Threads