Cloudflare Developers•3y ago

I don't think queues would be any good for rate limiting unless you mean reducing throughput to exte

I don't think queues would be any good for rate limiting unless you mean reducing throughput to external apis

UUnsmart I don't think queues would be any good for rate limiting unless you mean reducin...

Jacob•3/14/23, 10:57 PM

That’s exactly what I’m doing

UUnsmart I don't think queues would be any good for rate limiting unless you mean reducin...

Kai•3/14/23, 11:54 PM

yeah that's also what I'm doing. I think (?) a fairly common need given cloudflare based products have no problem handing out request volumes that just kill vendors. Generally I've not seen that many external APIs disallow hitting the rate limit in general, but they'll get unhappy if it's too much / impacting other customers.

Kai•3/14/23, 11:55 PM

So at least taking some care to stop that from happening is important and queues seems like the best way to still ensure all requests make it eventually, but limit throughput. It's just a bit akward to implement efficiently right now

Erisa•3/14/23, 11:58 PM

you can also batch from a queue if the provider allows it, to avoid spending a request for each and better optimise the amount of data you send in each request (which i think jacob is doing?)

Jacob•3/15/23, 12:39 AM

Yes that’s what I’m doing to increase throughout - batching together data so I’m making fewer api calls

Jacob•3/15/23, 12:41 AM

One of my queues is saving batches of data to R2 which saves a ton of API calls (averages around 30 messages/write to R2)

Erisa•3/15/23, 12:42 AM

Thats awesome, reminds me of how logpush will batch requests in a similar way

Jacob•3/15/23, 12:46 AM

Nice! Yeah it’s super nice. I also have a Workers cron that will combine each hour of files into a single file so that it’s super easy to process later

$\ ឵឵឵$ \\ ឵឵឵Hey all, is it the plan to raise the retention limits from 4 days in future?

Matt Silverlock•3/16/23, 5:23 PM

Yes, once we start actively expiring messages that reach the retention limit, and build some controls around it. One of the nearer-term items on the roadmap.

$\ ឵឵឵$ \\ ឵឵឵Awesome. Do you guys have any timeline in mind for the delay and redrive feature...

Matt Silverlock•3/16/23, 6:31 PM

No timeline just yet. HTTP Pull is the big next thing on the roadmap, so that folks can consume from outside Workers and/or in cases where you have a pull-based workload (vs. pushing, as we do today)

kika•3/16/23, 9:59 PM

Hi, could please someone have a look at this thread and tell me is queues could be a culprit here? kv

Wwarpandas Is there any reason why I can't get the Authorization part of the request sent t...

Matt Silverlock•3/17/23, 1:35 PM

Can you expand? There’s nothing different about a Worker that calls .send() on a queue that would inhibit any Request headers.

Kkika Hi, could please someone have a look at this thread and tell me is queues could ...

Matt Silverlock•3/17/23, 2:03 PM

Can you:

1. Paste the exact error message you're seeing? Is it resources exceeded or is it something else?
2. Can you confirm your batch settings, as specified in your

wrangler.toml

wrangler.toml

- ?
3. Share your accountId with me?

MMatt Silverlock Can you: 1. Paste the exact error message you're seeing? Is it resources exceed...

kika•3/17/23, 2:27 PM

I'm on paid, unbound plan, so 600ms should not be my CPU limit.

kian•3/17/23, 2:32 PM

Exceeds Resource could also show if you run out of memory, which is 128MB

Kkian Exceeds Resource could also show if you run out of memory, which is 128MB

kika•3/17/23, 2:32 PM

at exactly 1000 items? never 999, never 1001

Kkika at exactly 1000 items? never 999, never 1001

kika•3/17/23, 2:35 PM

also, it's highly unlikely. a few hundred K file, HTML with inserted badly formed JSON. it's messy, but not large and not complex. I believe I measured the parser footprint early in the development, it was a couple M if I recall correctly.

kian•3/17/23, 2:53 PM

The only limit that’s specifically 1,000 is the amount of in-house subrequests per invocation, which afaik should just be a rejected promise on the subrequest rather than a resources exceeded warning. That warning should exclusively be CPU or memory.

Kkian The only limit that’s specifically 1,000 is the amount of in-house subrequests p...

kika•3/17/23, 2:57 PM

maybe my (or Rust's worker runtime's) error handling is leaking memory. dunno how to test for that, needless to say that the worker works okay locally. strange though, this code is WIP and littered with unwrap()unwrap() s which turn every error into a panic!()panic!()

kian•3/17/23, 3:08 PM

Do you see it in the headers log?

kian•3/17/23, 3:18 PM

How are you making the request to the Worker? It doesn’t sound like that header is getting there at all

kian•3/17/23, 3:20 PM

Lots of your fetch options seem un-necessary - but I’m pretty sure no-cors doesn’t allow you to send that header.

Wwarpandas ``` Access to fetch at 'https:// test-worker.sensay.workers.dev/' from origin 'h...

Chaika•3/18/23, 7:26 PM

It sounds like you are trying to fetch your Worker Script from your browser to add to a queue, on a webpage at http://localhost:8080

The point of CORS is to prevent other websites/origins from fetching your origin, preventing scripts from accessing resources that they shouldn't have access to.

"no-cors" isn't a magical escape hatch to get around this, it's a very restrictive mode to make your request safe. Stripping most headers from the request, restricting content-type, restricting http request type, and making you unable to read the response body/status/headers: https://evertpot.com/no-cors/

The answer is to change your worker to respond with the required CORS Headers, for example:
Access-Control-Allow-Origin: http://localhost:8080
Your real website won't be on localhost:8080, for local testing you could use a proxy that adds the CORS headers for you. For production use though, if you want to continue to have your script on a different website/origin then your website, you will need to add the required CORS Headers.
More info: https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS
This has nothing to do with Queues or even Workers themselves, just do to with Cross-origin requests, so if you have any more questions/etc, please use #coding-help or another channel that fits better

Matt Silverlock•3/20/23, 11:48 PM

Queue consumer concurrency is going to land in just under two weeks: https://developers.cloudflare.com/queues/changelog/#consumer-concurrency-upcoming

If you have a use-case where you want to limit concurrency, you’ll want to cap it via the dashboard or wrangler before we enable autoscaling.

Changelog · Cloudflare Queues

Queue consumers will soon automatically scale up concurrently as a queues’ backlog grows in order to keep overall message processing latency down. …

MMatt Silverlock Queue consumer concurrency is going to land in just under two weeks: https://dev...

Erisa•3/21/23, 8:18 AM

@Queues

zegevlier•3/21/23, 1:06 PM

Yup, that's totally fine

MMoretti Thinking about this use case: fetch handler receives request, process and needs...

Matt Silverlock•3/21/23, 4:30 PM

Are you trying to serve a real-time response back to a user? If so, how do you intend to do that?

Matt Silverlock•3/21/23, 4:31 PM

Queues is best at async work. If you're relying on returning a live response to a client, then it may not make sense, but you'll also need to manage your upstream API (in this case, OpenAI's) rate limits as well vs. your own user demand.

Kai•3/21/23, 5:27 PM

doing something like fetch -> upgrade WS -> push WS to DO & enqueue work item w/ DO Id -> ... queue ... -> eventual return via DO WS -> close WS works decently imo. Can reuse the WS / DO as needed & is decently efficient. It'll be a bit more expensive depending on how you implement it (DOs with WS are never evicted so you pay) but it's possible to do & even implement timeouts and that sort of thing.

MMoretti In our case I want to close the connection with the user and then reply async wi...

Matt Silverlock•3/21/23, 6:44 PM

How do you get that response back to the user? e.g. what medium?

MMatt Silverlock How do you get that response back to the user? e.g. what medium?

Matt Silverlock•3/21/23, 6:44 PM

(does the user/client poll/retry?)

I don't think queues would be any good for rate limiting unless you mean reducing throughput to exte

Similar Threads

Similar Threads

Similar Threads