A “pull” based approach (the GPU calls to an endpoint when it’s ready) would make a lot more sense h

A “pull” based approach (the GPU calls to an endpoint when it’s ready) would make a lot more sense here. The GPU asks the queue for work when it’s idle/finished a job. Queues, SQS, Pub/Sub, Pulsar, etc - this “hang on for minutes or longer for a response” is not really reliable in any system.

MMatt Silverlock A “pull” based approach (the GPU calls to an endpoint when it’s ready) would mak...

Matt SilverlockOP•1/13/23, 9:28 PM

This isn’t something we support today, but just making clear that trying to block on the GPU will cause you issues no matter what “queue” you use.

Aaltryne Unless there's a way for me to consume messages from python code? or run a GPU p...

Matt SilverlockOP•1/13/23, 10:07 PM

No. In the future we will support polling via HTTP (HTTP GET to a /queue-name) and writing to a queue via HTTP POST but those are a little further out. Addressing consumer concurrency & partial batching are the two next up.

Aaltryne Yeah, makes sense Other queus (say Celery) the consumer can literally sit on th...

aarhus•1/14/23, 12:20 AM

How do you know the job has been completed by the GPU? If you can identity that, then you could use a DO - the queue (or directly) adds a job to the DO. The DO knows the status of the GPU tasks and only sends when the current one has finished. If you know roughly how long a task will take you can use a DO alarm to wake up and check the status

Aaltryne durable object?

Matt SilverlockOP•1/14/23, 12:21 AM

Yes.

Matt SilverlockOP•1/14/23, 12:21 AM

https://developers.cloudflare.com/workers/learning/using-durable-objects/

Using Durable Objects · Cloudflare Workers docs

Durable Objects provide low-latency coordination and consistent storage for the Workers platform through two features: global uniqueness and a transactional …

Matt SilverlockOP•1/14/23, 12:22 AM

Alarms: https://developers.cloudflare.com/workers/learning/using-durable-objects/#alarms-in-durable-objects

Using Durable Objects · Cloudflare Workers docs

Durable Objects provide low-latency coordination and consistent storage for the Workers platform through two features: global uniqueness and a transactional …

Aaarhus How do you know the job has been completed by the GPU? If you can identity that...

Matt SilverlockOP•1/14/23, 12:22 AM

Good idea.

MMatt Silverlock Good idea.

aarhus•1/14/23, 12:22 AM

it happens sometimes

aarhus•1/14/23, 3:57 PM

The DO will wake up.

Unsmart•1/14/23, 4:20 PM

Afaik there arent any FIFO queues yet they are still working on standard queues. But setting the max batch size to 1 should in theory make it instantly call your processor yes

Unsmart•1/14/23, 7:18 PM

You can use DOs to create your own FIFO queues and you can have unlimited DOs that can be created on demand

Unsmart•1/14/23, 7:20 PM

DOs also give you a lot more power over how you process in general because you can control all the timings and retries

GGiggiux There is no way to automatically create queues based on some text-string, right?...

Matt SilverlockOP•1/15/23, 10:34 AM

Right. We’re discussing the concept of “dynamic bindings” but they are non-trivial and have a lot of IAM/security implications that we need to solve, too. Won’t be soon, but we think it’s important long term for Queues/D1/R2/etc

Unsmart•1/15/23, 9:38 PM

Dynamic bindings for things other than DOs would be super cool

UUnsmart Dynamic bindings for things other than DOs would be super cool 👀

DaniFoldi•1/16/23, 7:58 AM

That’s exactly what I was thinking - even if it’s a prototype/default-based submamespace of sorts, if per-namespace operations can be supported, it can solve the majority of use cases where any kind of dynamic binding was needed

Bbye Does Durable Objects also supports Queues?

Unsmart•1/17/23, 9:50 PM

As in sending a message to a queue from DOs? It should

John Spurlock•1/18/23, 1:13 AM

I would love to have a DO consumer - ie a single DO that processes as fast as it can against a queue - avoids trying to guess how much work will overload a single DO.

John Spurlock•1/18/23, 1:15 AM

having eyeballs as consumers is kind of strange since they are subject to the same short runtime limits as fetch handlers - any real work is sent down to DOs anyway, so it would be nice to cut out the middleman!

Burrito•1/18/23, 4:53 AM

Would the 100 batch size be increased later?

BBurrito Would the 100 batch size be increased later?

Matt SilverlockOP•1/18/23, 5:24 AM

After we ship partial batch acknowledgment (explicit ack/retry behavior) - otherwise any consumer failure on a large batch means the entire batch is retried. Likely to be closer to end of this quarter (subject to change)

BBurrito Would the 100 batch size be increased later?

Matt SilverlockOP•1/18/23, 5:24 AM

(What are you trying to do && what is an “increase” to you?)

Burrito•1/18/23, 5:33 AM

I'm building a storage like service with pretty low write volume, for reads I want it to be extremely fast and it's acceptable to show stale state for some seconds, while for writes as long as it shows up instantly on the colo it writes from (so that writer won't assume their write failed) then it can take time propagate to other colos.

Currently I'm thinking:
- R2 to store the data.
- Worker checks cache API for a match and return instantly; if there isn't a match, pull from R2 and store it with cache API as immutable cache that lasts forever.
- When a write happens, uses CF API to purge that cache for all colo.

Problem is that CF API has a rate limit of 1k per minute (or lower?), so I'm thinking batching them with queues. 100 per batch is probably enough for the write volume I need to handle, but just want to see if it would become a problem later down the line.

BBurrito I'm building a storage like service with pretty low write volume, for reads I wa...

Matt SilverlockOP•1/18/23, 6:54 AM

Sounds like a closer fit to KV? Writes are eventually consistent globally, but cached in the local colo for readers. Reads are fast (and much faster than R2).

MMatt Silverlock Sounds like a closer fit to KV? Writes are eventually consistent globally, but c...

Burrito•1/18/23, 6:59 AM

KV doesn't quite work, it's pull based so once the cache in a colo expires it needs to go fetch again; there's also no way to lower the cache below 60 seconds, and lowering that would also mean there would be more fetches.

BBurrito KV doesn't quite work, it's pull based so once the cache in a colo expires it ne...

Matt SilverlockOP•1/18/23, 7:00 AM

Right, I understand how KV works, but “take time to propagate to other colos” left your requirements unclear.

Burrito•1/18/23, 7:01 AM

Good point, yeah a few seconds for writes to show up is fine, but 60 seconds of KV is too long.

kian•1/18/23, 7:13 AM

No idea about timeframes, likely a long way away - but r2 might be of interest down the line with KV

Burrito•1/18/23, 7:14 AM

Yep keeping an eye on that too, if that becomes a thing than it would solve my problem nicely without having to hack together all these things.

kian•1/18/23, 7:15 AM

How big is the data you’re storing/caching?

Burrito•1/18/23, 7:16 AM

Vast majority should be a few MB, largest can still fit in KV.

kian•1/18/23, 7:17 AM

Shame - if it was small enough I’d say go full mad scientist and use a Worker itself as a data store. Would expect the usual provisioner lag to be good for your eventual consistency needs & be fairly quick reads since it exists in every colo without needing to be pulled from a central store like KV

kian•1/18/23, 7:18 AM

Buuut that only works with sub-1MB data - and would also hit script per account limits - and other bad things

kian•1/18/23, 7:18 AM

kian•1/18/23, 7:20 AM

The wording here is weird, since the Cache API “cache key” is just the URL, so what defines custom or not?

kian•1/18/23, 7:20 AM

But I’d recommend making sure the Purge Cache API fits your needs and actually purges the objects you put into the Cache API from a Worker

Burrito•1/18/23, 7:21 AM

Yeah I'll check that for sure.

kian•1/18/23, 7:22 AM

You can use up to 30 cache-tags in one API call and make up to 30,000 purge API calls in a 24-hour period.

kian•1/18/23, 7:22 AM

Naturally, these are your bounds too

Burrito•1/18/23, 7:23 AM

Ah, wasn't aware that's a thing.

Burrito•1/18/23, 7:23 AM

My problem really just boils down to needing a push mechanism to tell each colo "new data, go fetch again"

A “pull” based approach (the GPU calls to an endpoint when it’s ready) would make a lot more sense h

Similar Threads

Similar Threads

Similar Threads