I implemented a throttled queue to manage rate limits for a project in the past where I had to limit

I implemented a throttled queue to manage rate limits for a project in the past where I had to limit to 3rps to the Notion API: https://github.com/jahands/msc-utils/blob/main/src/notion.ts
Could probably use this when processing batches

Kai•11/28/22, 2:33 PM

So currently you could set some sort of semaphore using a DO.

Can you explain how this would work? I can sync the rate limit via one or multiple DOs but this is very complicated to do (especially since I have multiple "layered" rate limits) & DOs aren't really atomic iirc.

JJacob I implemented a throttled queue to manage rate limits for a project in the past ...

JacobOP•11/28/22, 2:35 PM

I could see this working pretty well in a Queue consumer, and it feels like a pretty common pattern that you might need to rate-limit processing of items in a batch
It's used like this:

await queue.add(myAsyncFunction)
await queue.clear() // Execute functions in a rate-limited fassion

await queue.add(myAsyncFunction)
await queue.clear() // Execute functions in a rate-limited fassion

p-throttle and p-queue on npm are really helpful packages for stuff like this

RRob At GA we envision something *like* (not the actual API surface - still under con...

Kai•11/28/22, 4:08 PM

Hmm, that would be nice. But wouldn't this be more expensive to the user then just handling the timeout themselves? From what I understand each retry incurs one operation, but consumers are executed as unbound workers, so sleeping for 20s doesn't incurr any charges.

SSananGuliyev I am on enterprise plan. But unfortunately I have no contact to account team.

Unsmart•11/28/22, 6:04 PM

Account team meaning your CSM. Are you the owner of the account? If not I would reach out to whoever at your company is to find out who the CSM is to contact. Or possibly create a support ticket to see if they can help you get access?

Kai•11/28/22, 7:00 PM

Yes, but I'd use retries to bounce off above-ratelimit requests in this scenario, right? It's not that unlikely to have a lot of requests above that rate limit especially during peak hours

Kai•11/28/22, 7:18 PM

that's actually very fair. Specifying larger durations may be viable. Not sure what batch size would do here though? Is there something I'm missing about how batches work?

dave•11/29/22, 11:09 PM

I keep getting removed.

dave•11/29/22, 11:09 PM

I haven’t even logged into my account today.

Ddave I haven’t even logged into my account today.

James•11/29/22, 11:59 PM

I think there are some billing changes happening with how Queues is provisioned on the backend. Hopefully @robsutter can provide more info - sorry if the emails are scary, I know it can be quite worrying when you receive an email about this when you might be using the product

JJames I think there are some billing changes happening with how Queues is provisioned ...

dave•11/30/22, 12:44 AM

oh sweet, yeah, I’m currently using Queues for feeding analytics. Not mission critical yet thankfully.

Erisa•11/30/22, 1:19 AM

Do you still have access to queues or did it remove it properly?

Erisa•11/30/22, 1:20 AM

Shouldn't have actually impacted anything, just a side effect of the billing changes

Kai•11/30/22, 5:38 PM

from workers-and-pages-discussions
Where are consumers placed? I'd be interested in getting a consumer be placed close to some specific region.

Unsmart•11/30/22, 6:49 PM

I could see setting region being useful to set it close to your database so it returns results faster

Unsmart•11/30/22, 6:50 PM

or close to whatever region a specific service/api you use is hosted in if they dont use edge services

Ddave I keep getting removed. 😐

Matt Silverlock•11/30/22, 7:15 PM

Sorry about this: this is a cleanup for some billing stuff. As long as you are on: a) a paid Workers plan and b) enable Queues, it should work. We don't immediately delete your queues.

RRob Long-term we may configure; would love to hear the use case/requirement. CC @Mat...

Kai•11/30/22, 8:02 PM

It's really just an optimization for queues/cron/pubsub, not really a requirement for anything I'm doing. Hinting that there will be outbound requests & to what region would be nice to have.
For my use case, I have a small product that exposes an API and pulls data to serve from external sites. It both does this on demand and on a schedule, to keep upstream data fresh.
It has some complicated internal routing to do based on the upstream resource requested, and one of those routings is what region data is requested from, ie European data is only available from Europe each routing step is another queue and this makes a small tree of queues. To improve latency it'd be great to have Europe specific queues to be attached to European data centers.
I can give more details once I've started actually porting part of the product to cloudflare queues. The current code base is very stateful so specifics don't really translate.

maguayo•12/1/22, 9:59 AM

Is it there a way to see how many messages are in a queue? I found nothing in the UI or Cloudflare API

Mmaguayo Is it there a way to see how many messages are in a queue? I found nothing in th...

John Spurlock•12/1/22, 2:43 PM

queues

MMatt Silverlock Sorry about this: this is a cleanup for some billing stuff. As long as you are o...

dave•12/1/22, 3:59 PM

thanks! I did actually run into an error when trying to redeploy something that was bound to a queue, but enabling Queues in the web UI did the trick.

Matt Silverlock•12/1/22, 4:01 PM

Yeah, you must use the UI for the first time to enable - we like to be explicit about the potential for extra $$

MMatt Silverlock Yeah, you _must_ use the UI for the first time to enable - we like to be explici...

dave•12/1/22, 5:14 PM

it's my third time enabling it though lol

Ddave it's my third time enabling it though lol

Matt Silverlock•12/1/22, 5:51 PM

Welcome to the beta

James•12/2/22, 5:27 PM

Deployed our first Workers queue into production today. Working really well, and currently chugging through about 10k tasks.

James•12/2/22, 5:45 PM

Out of interest, although metrics aren't available in the dash yet, are they being recorded so we'll see historic data at some point?

James•12/2/22, 5:45 PM

Or is that all WIP?

James•12/2/22, 5:47 PM

And also, what's the max_wait_time_msmax_wait_time_ms I see returned in the API. Is that mapped to max_batch_timeoutmax_batch_timeout, or something different?

RRob They are being recorded. The current focus is more on real-time metrics, but exp...

James•12/2/22, 5:47 PM

Awesome, ty for the info!

RRob Under `.consumers[.settings.max_wait_time_ms]` right? That's the same.

James•12/2/22, 5:49 PM

Yeah precisely, okay cool thank you!

I implemented a throttled queue to manage rate limits for a project in the past where I had to limit

Similar Threads

Similar Threads

Similar Threads