Can I have multiple consumers in one Worker, each one processing different queues? Context: I'm r

Can I have multiple consumers in one Worker, each one processing different queues?

Context:

I'm reading this in the docs:

Best practice
Configure a single consumer per queue. This both logically separates your queues, and ensures that errors (failures) in processing messages from one queue do not impact your other queues.

Rrbravo Can I have multiple consumers in one Worker, each one processing different queue...

Ping for toast•8/1/24, 5:30 AM

It is my understanding that you can have a single worker reading ("consuming") multiple queues, but a single queue should only be read by one worker

Ping for toast•8/1/24, 5:33 AM

Avi•8/1/24, 9:06 AM

is there truly no way to read environment variables when my worker starts up? (i.e. before the initial fetch() comes in?)

Avi•8/1/24, 9:06 AM

would be fine with this even being limited to local dev...

Baby Beel•8/1/24, 9:58 AM

isn't scrypt in worker?

dom•8/1/24, 1:13 PM

might be a dumb question but which permissions should i use if i want to create an API token to update cron triggers for a worker? it doesn't seem like there's a separate scope for it

Ddom might be a dumb question but which permissions should i use if i want to create ...

Isaac McFadyen•8/1/24, 1:26 PM

I believe it should fall under Edit Worker permissions.

infinite cat gen•8/1/24, 5:14 PM

Hey, quick question

infinite cat gen•8/1/24, 5:15 PM

is it possible to open a worker to direct IP connections?

Iinfinite cat gen is it possible to open a worker to direct IP connections?

Chaika•8/1/24, 5:16 PM

You mean tcp/udp inbound, essentially? As it is right now workers fit firmly behind the CDN/normal proxy. They've talked about being able to have tcp/udp inbound in the past, but nothing yet

CChaika You mean tcp/udp inbound, essentially? As it is right now workers fit firmly beh...

infinite cat gen•8/1/24, 5:16 PM

I need to handle a few POST and GET requests

infinite cat gen•8/1/24, 5:16 PM

I can't change something in my programs code so I wrote an API that handles that for me

Chaika•8/1/24, 5:17 PM

If you mean just skipping the cdn or something then as said above they sit firmly behind the cdn/normal proxy. Workers do not have IPs assigned to them. They run everywhere on every machine

CChaika If you mean just skipping the cdn or something then as said above they sit firml...

infinite cat gen•8/1/24, 5:17 PM

ah alright

Iinfinite cat gen I need to handle a few POST and GET requests

Chaika•8/1/24, 5:17 PM

Workers can handle that

Chaika•8/1/24, 5:17 PM

unless you're talking about getting around the upload limit or something

CChaika unless you're talking about getting around the upload limit or something

infinite cat gen•8/1/24, 5:18 PM

No I don't need to get around that

infinite cat gen•8/1/24, 5:19 PM

I need to modify my hosts file to include the IP of the worker, because the hosts file can't handle URL to URL redirects

infinite cat gen•8/1/24, 5:19 PM

That's why I'm asking

Iinfinite cat gen I need to modify my hosts file to include the IP of the worker, because the host...

Chaika•8/1/24, 5:22 PM

yea I mean they're behind the normal cdn so you would have to pass the right host header/sni/url to reach them, and even then CDN IPs are not a good idea to hardcode as they change

infinite cat gen•8/1/24, 5:22 PM

yeah so no point in trying

infinite cat gen•8/1/24, 5:22 PM

thx for help

Chaika•8/1/24, 5:22 PM

could maybe use some other proxy server (like nginx on a vps) to proxy the requests to workers but that'd be another step

CChaika could maybe use some other proxy server (like nginx on a vps) to proxy the reque...

infinite cat gen•8/1/24, 5:23 PM

maybe ill be able to do that

infinite cat gen•8/1/24, 5:23 PM

lemme see

infinite cat gen•8/1/24, 5:26 PM

nah i think ill just try something different

Jonathan Silva•8/1/24, 6:10 PM

Good afternoon
How to resolve this error in request.json() ?

JJonathan Silva Good afternoon How to resolve this error in request.json() ?

Idle•8/1/24, 6:11 PM

most likely by catching it or making sure that the request body has a valid json format?

Frank•8/1/24, 9:49 PM

is there a limit on how many custom cache namespace we can create?

Hyper•8/1/24, 10:14 PM

Hey guys. I want to create a notification system but im not sure where to start. my original idea was using durable objects but i think the limit of 32,000 active connections makes it not scalable, and if i go for a 1:1 architcture then it would be difficult to communicate with other DOs. My next idea is SSE using the event source API. What I want to achieve is essentially when a request is made to my API (which runs on workers), some requests will trigger notifications to reach certain clients. Id like to communicate the data to those specific clients. Any ideas or suggestions?

For example, user A sends a friend request to user B by calling an API endpoint on my worker. I want to push the notification to user B. Any help would be appreciated

Brett Willis•8/2/24, 12:25 AM

Coming from Node.JS, the event loop is well understood. Particularly that Promise callbacks operate on the microtask queue and I/O is not processed in between microtasks. Unless I've missed something in my searches, very little is described about the event loop in Workers and how/when the event loop processes other requests (does it use the "default" V8 event loop implementation, and how does that work?).

The reason I ask is is: how to break up long-running CPU-intensive calculations to allow other requests to be processed in between? E.g. calculating password hashes using

noble-hashes

noble-hashes

. That library provides an async implementation which just uses promise callbacks (microtask queue) to break up the calculation: https://github.com/paulmillr/noble-hashes/blob/ae060daa6252f3ff2aa2f84e887de0aab491281d/src/utils.ts#L103-L119.

If the workers event loop is similar to Node.JS, I/O and other requests will not be processed between microtasks. So the async implementation would only be less efficient and not have the desired effect of breaking up yielding the CPU to other requests.

Can someone teach me how the Workers event loop is implemented and how to yield to other requests?

BBrett Willis Coming from Node.JS, the event loop is well understood. Particularly that Promis...

kian•8/2/24, 12:41 AM

The first part about microtask queues blocking I/O sounds like the same scenario as described in https://github.com/cloudflare/workerd/blob/ce15e0b82e6408c49ec3e980f23640bd25d6c675/src/workerd/io/io-context.c%2B%2B#L1044-L1065. Unfortunately, looking at the runtime is probably the only 'documentation' about details like this at the moment.

Kkian The first part about microtask queues blocking I/O sounds like the same scenario...

Brett Willis•8/2/24, 1:10 AM

Thanks for pointing me to that code! I won't pretend to understand the code but yeah from reading that comment it seems to suggest that it behaves the same as Node.JS in that microtasks callbacks do not yield to process I/O (and I'm assuming that "I/O" includes other requests).
No benefit to using the async

noble-hashes

noble-hashes

versions then, just the significant overhead of enqueueing/dequeueing the stack onto the microtask queue.
It seems you also answered the other question: it looks like Workers has it's own event loop implementation.

Brett Willis•8/2/24, 1:16 AM

Another question related to calculating password hashes. The hashing algorithms usually intentionally consume no small amount of memory. While a single computation may fit within the 128MB limit, two or more computations may not, and therefore all in progress requests would get aborted if the worker hits the memory limit. Does the Workers runtime look at the available memory and/or average memory per request when deciding whether so spin up a new instance vs. hit the same instance and potentially hit the limit?
Perhaps this is a good reason to block the thread and not allow other requests to interleave during hashing calculation...

Mehanika•8/2/24, 10:28 AM

I have a project that has a page and worker. Every new github branch for the app spins up a testing environment which is great. I am not finding any resources how to (or if even possible) to do the same with the api/worker. Anyone has experience with this?

jdruwe•8/2/24, 11:53 AM

I added an issues for the next-on-pages project at: https://github.com/cloudflare/next-on-pages/issues/844, I noticed it to be related to the selected worker runtime version. On older versions the code works. Not sure who from the cloudflare team is able to resolve this.

GitHub

Issues · cloudflare/next-on-pages

CLI to build and develop Next.js apps for Cloudflare Pages - Issues · cloudflare/next-on-pages

konedi•8/2/24, 1:17 PM

Hi all, anyway to speed up workers ai through rest api? responses are taking up to 30s, while I have a local llama3.1 running on a rtx3080 and it only takes like 6s?

AngusMa•8/2/24, 3:22 PM

workers-and-pages-helpEmbedding worker in Notion

sia•8/2/24, 4:33 PM

I think I'm missing some basic understanding of how functions and workers can work together. I've binded my worker to my function as mentioned here https://developers.cloudflare.com/pages/functions/bindings/#service-bindings. But the only usage example is:

return context.env.SERVICE.fetch(context.request);

return context.env.SERVICE.fetch(context.request);

But, I don't want to just forward the request sent to the function. I want to send the worker JSON data for it to send an email, and in the function I redirect to a confirmation page. I'm unclear how to interact with the worker from the function

yat•8/2/24, 5:23 PM

does the free plan have cold starts ?

konedi•8/2/24, 6:11 PM

does anyone know why cloudflare worker ai llama 3.1 is 3x slower than local llama 3.1 running on rtx3080? is there no way to speed this up? 30-40 seconds for text generation is insane. I get that it is free credits but damn that is kinda slow

Kkonedi does anyone know why cloudflare worker ai llama 3.1 is 3x slower than local lla...

Chaika•8/2/24, 6:46 PM

?channel-crossposting -> #workers-ai

Flare•8/2/24, 6:46 PM

Please do not post your question in multiple channels/post it multiple times per the rules at ⁠#

welcome-and-rules. It creates confusion for people trying to help you and doesn't get your issue or question solved any faster.

Chaika•8/2/24, 6:46 PM

you have the best chance of being answered there anyway

konedi•8/2/24, 6:47 PM

got it. thanks. sorry, it just seems like all channels are somewhat dead so I needd to see which is the one that is best for this

Chaika•8/2/24, 6:50 PM

people respond/post questions in these channels and don't talk otherwise

does anyone know why cloudflare worker ai llama 3.1 is 3x slower than local llama 3.1 running on rtx3080? is there no way to speed this up? 30-40 seconds for text generation is insane. I get that it is free credits but damn that is kinda slow

AI team explain more in their channel, I'm no AI guy and don't know 100% their setup, but comparing local vs remote seems a bit silly. Workers AI is powered by a ton of shared GPUs, vs your one unshared gpu, and they've got lots of magic in front of it with request routing/etc to try to scale/shard requests. There's lots of different ways to run models too is my understanding, each with different quirks

kabocha•8/2/24, 10:17 PM

does anyone know how long a request to increase subrequest limit takes to process?

also is it possible to increase the subrequest limit on a free project, if the use case requires it?