Yes it's the whole batch, though you can tune your `max_batch_size` to be as small as you'd like. We

Yes it's the whole batch, though you can tune your

max_batch_size

max_batch_size

max_batch_size

max_batch_size to be as small as you'd like. We also allow you to configure how many times a given ~~batch~~ message can be retried, up to 100. https://developers.cloudflare.com/queues/limits/

Limits · Cloudflare Queues

1 Request adjustments to limits that conflict with your project goals by contacting Cloudflare. To increase a limit, complete the Limit Increase Request Form. 2 …

Aaltryne Is there way to catch the timeout somehow? Also, are the timeouts writted expli...

Matt Silverlock•1/13/23, 5:34 PM

Are you catching errors around the fetch() you are presumably making to your GPU machine? How are you otherwise seeing the JSON error? It could also be your end closing the condition if it’s idle too long as well. Holding a minutes long connection open and blocking on it is just going to be prone to failure, regardless of the message bus in-between.

kian•1/13/23, 5:39 PM

Are you calling

response.json()

response.json()

with your fetch()fetch() call that's timing out?

kian•1/13/23, 5:45 PM

Unexpected end of JSON inputUnexpected end of JSON input isn't the exact error I'd expect but any requests that are left waiting for a response for more than 100 seconds will just get a plain-text error code: 524error code: 524 response - or a pretty HTML page if the request headers indicate that's acceptable.

EEmilio Yes it's the whole batch, though you can tune your `max_batch_size` to be as sma...

Kai•1/13/23, 6:57 PM

OK, unfortunately nothing changed then. I'm using this to push a fluctuating load through a rate limited API, but in my experiments it was very inefficient due to constant requing

KKai OK, unfortunately nothing changed then. I'm using this to push a fluctuating loa...

EmilioOP•1/13/23, 7:31 PM

Would giving you control over the retry interval (min, max, backoff coefficient) help you with your usecase?

Kai•1/13/23, 7:39 PM

yes, very much!

dave•1/13/23, 7:49 PM

likewise here

Aaltryne Yeah looks like my response.json() was outside of the "response.ok" check. Than...

dave•1/13/23, 8:52 PM

Would Pub/Sub maybe work?

Aaltryne Yeah looks like my response.json() was outside of the "response.ok" check. Than...

dave•1/13/23, 8:55 PM

without giving out more info than you'd like, what's the gist of your problem? Large incoming files from users, slow ish non-scalable (to a degree) GPU instances that do processing, and retrying if that processing fails or times out?

dave•1/13/23, 8:55 PM

(I ask because something similar to that is on my TODO list.)

Aaltryne GPU being slow, and single threaded, sometimes taking 20+ minutes to respond And...

dave•1/13/23, 9:05 PM

I still think it doesn't solve the core problem, but have you looked at using ephemeral/spot instances that can scale with requests?

Aaltryne GPU being slow, and single threaded, sometimes taking 20+ minutes to respond And...

Matt Silverlock•1/13/23, 9:27 PM

A “pull” based approach (the GPU calls to an endpoint when it’s ready) would make a lot more sense here. The GPU asks the queue for work when it’s idle/finished a job. Queues, SQS, Pub/Sub, Pulsar, etc - this “hang on for minutes or longer for a response” is not really reliable in any system.

MMatt Silverlock A “pull” based approach (the GPU calls to an endpoint when it’s ready) would mak...

Matt Silverlock•1/13/23, 9:28 PM

This isn’t something we support today, but just making clear that trying to block on the GPU will cause you issues no matter what “queue” you use.

Aaltryne Unless there's a way for me to consume messages from python code? or run a GPU p...

Matt Silverlock•1/13/23, 10:07 PM

No. In the future we will support polling via HTTP (HTTP GET to a /queue-name) and writing to a queue via HTTP POST but those are a little further out. Addressing consumer concurrency & partial batching are the two next up.

Aaltryne Yeah, makes sense Other queus (say Celery) the consumer can literally sit on th...

aarhus•1/14/23, 12:20 AM

How do you know the job has been completed by the GPU? If you can identity that, then you could use a DO - the queue (or directly) adds a job to the DO. The DO knows the status of the GPU tasks and only sends when the current one has finished. If you know roughly how long a task will take you can use a DO alarm to wake up and check the status

Aaltryne durable object?

Matt Silverlock•1/14/23, 12:21 AM

Yes.

Matt Silverlock•1/14/23, 12:21 AM

https://developers.cloudflare.com/workers/learning/using-durable-objects/

Using Durable Objects · Cloudflare Workers docs

Durable Objects provide low-latency coordination and consistent storage for the Workers platform through two features: global uniqueness and a transactional …

Matt Silverlock•1/14/23, 12:22 AM

Alarms: https://developers.cloudflare.com/workers/learning/using-durable-objects/#alarms-in-durable-objects

Using Durable Objects · Cloudflare Workers docs

Durable Objects provide low-latency coordination and consistent storage for the Workers platform through two features: global uniqueness and a transactional …

Aaarhus How do you know the job has been completed by the GPU? If you can identity that...

Matt Silverlock•1/14/23, 12:22 AM

Good idea.

MMatt Silverlock Good idea.

aarhus•1/14/23, 12:22 AM

it happens sometimes

aarhus•1/14/23, 3:57 PM

The DO will wake up.

Unsmart•1/14/23, 4:20 PM

Afaik there arent any FIFO queues yet they are still working on standard queues. But setting the max batch size to 1 should in theory make it instantly call your processor yes

Unsmart•1/14/23, 7:18 PM

You can use DOs to create your own FIFO queues and you can have unlimited DOs that can be created on demand

Unsmart•1/14/23, 7:20 PM

DOs also give you a lot more power over how you process in general because you can control all the timings and retries

GGiggiux There is no way to automatically create queues based on some text-string, right?...

Matt Silverlock•1/15/23, 10:34 AM

Right. We’re discussing the concept of “dynamic bindings” but they are non-trivial and have a lot of IAM/security implications that we need to solve, too. Won’t be soon, but we think it’s important long term for Queues/D1/R2/etc

Unsmart•1/15/23, 9:38 PM

Dynamic bindings for things other than DOs would be super cool

UUnsmart Dynamic bindings for things other than DOs would be super cool 👀

DaniFoldi•1/16/23, 7:58 AM

That’s exactly what I was thinking - even if it’s a prototype/default-based submamespace of sorts, if per-namespace operations can be supported, it can solve the majority of use cases where any kind of dynamic binding was needed

Bbye Does Durable Objects also supports Queues?

Unsmart•1/17/23, 9:50 PM

As in sending a message to a queue from DOs? It should

John Spurlock•1/18/23, 1:13 AM

I would love to have a DO consumer - ie a single DO that processes as fast as it can against a queue - avoids trying to guess how much work will overload a single DO.

John Spurlock•1/18/23, 1:15 AM

having eyeballs as consumers is kind of strange since they are subject to the same short runtime limits as fetch handlers - any real work is sent down to DOs anyway, so it would be nice to cut out the middleman!

Burrito•1/18/23, 4:53 AM

Would the 100 batch size be increased later?

BBurrito Would the 100 batch size be increased later?

Matt Silverlock•1/18/23, 5:24 AM

After we ship partial batch acknowledgment (explicit ack/retry behavior) - otherwise any consumer failure on a large batch means the entire batch is retried. Likely to be closer to end of this quarter (subject to change)

BBurrito Would the 100 batch size be increased later?

Matt Silverlock•1/18/23, 5:24 AM

(What are you trying to do && what is an “increase” to you?)

Yes it's the whole batch, though you can tune your `max_batch_size` to be as small as you'd like. We

Similar Threads

Similar Threads

Similar Threads