is it possible to open a worker to direct IP connections?

Iinfinite cat gen is it possible to open a worker to direct IP connections?

Chaika•8/1/24, 5:16 PM

You mean tcp/udp inbound, essentially? As it is right now workers fit firmly behind the CDN/normal proxy. They've talked about being able to have tcp/udp inbound in the past, but nothing yet

CChaika You mean tcp/udp inbound, essentially? As it is right now workers fit firmly beh...

infinite cat genOP•8/1/24, 5:16 PM

I need to handle a few POST and GET requests

infinite cat genOP•8/1/24, 5:16 PM

I can't change something in my programs code so I wrote an API that handles that for me

Chaika•8/1/24, 5:17 PM

If you mean just skipping the cdn or something then as said above they sit firmly behind the cdn/normal proxy. Workers do not have IPs assigned to them. They run everywhere on every machine

CChaika If you mean just skipping the cdn or something then as said above they sit firml...

infinite cat genOP•8/1/24, 5:17 PM

ah alright

Iinfinite cat gen I need to handle a few POST and GET requests

Chaika•8/1/24, 5:17 PM

Workers can handle that

Chaika•8/1/24, 5:17 PM

unless you're talking about getting around the upload limit or something

CChaika unless you're talking about getting around the upload limit or something

infinite cat genOP•8/1/24, 5:18 PM

No I don't need to get around that

infinite cat genOP•8/1/24, 5:19 PM

I need to modify my hosts file to include the IP of the worker, because the hosts file can't handle URL to URL redirects

infinite cat genOP•8/1/24, 5:19 PM

That's why I'm asking

Iinfinite cat gen I need to modify my hosts file to include the IP of the worker, because the host...

Chaika•8/1/24, 5:22 PM

yea I mean they're behind the normal cdn so you would have to pass the right host header/sni/url to reach them, and even then CDN IPs are not a good idea to hardcode as they change

infinite cat genOP•8/1/24, 5:22 PM

yeah so no point in trying

infinite cat genOP•8/1/24, 5:22 PM

thx for help

Chaika•8/1/24, 5:22 PM

could maybe use some other proxy server (like nginx on a vps) to proxy the requests to workers but that'd be another step

CChaika could maybe use some other proxy server (like nginx on a vps) to proxy the reque...

infinite cat genOP•8/1/24, 5:23 PM

maybe ill be able to do that

infinite cat genOP•8/1/24, 5:23 PM

lemme see

infinite cat genOP•8/1/24, 5:26 PM

nah i think ill just try something different

Jonathan Silva•8/1/24, 6:10 PM

Good afternoon
How to resolve this error in request.json() ?

JJonathan Silva Good afternoon How to resolve this error in request.json() ?

Idle•8/1/24, 6:11 PM

most likely by catching it or making sure that the request body has a valid json format?

Frank•8/1/24, 9:49 PM

is there a limit on how many custom cache namespace we can create?

Hyper•8/1/24, 10:14 PM

Hey guys. I want to create a notification system but im not sure where to start. my original idea was using durable objects but i think the limit of 32,000 active connections makes it not scalable, and if i go for a 1:1 architcture then it would be difficult to communicate with other DOs. My next idea is SSE using the event source API. What I want to achieve is essentially when a request is made to my API (which runs on workers), some requests will trigger notifications to reach certain clients. Id like to communicate the data to those specific clients. Any ideas or suggestions?

For example, user A sends a friend request to user B by calling an API endpoint on my worker. I want to push the notification to user B. Any help would be appreciated

Brett Willis•8/2/24, 12:25 AM

Coming from Node.JS, the event loop is well understood. Particularly that Promise callbacks operate on the microtask queue and I/O is not processed in between microtasks. Unless I've missed something in my searches, very little is described about the event loop in Workers and how/when the event loop processes other requests (does it use the "default" V8 event loop implementation, and how does that work?).

The reason I ask is is: how to break up long-running CPU-intensive calculations to allow other requests to be processed in between? E.g. calculating password hashes using noble-hashesnoble-hashes. That library provides an async implementation which just uses promise callbacks (microtask queue) to break up the calculation: https://github.com/paulmillr/noble-hashes/blob/ae060daa6252f3ff2aa2f84e887de0aab491281d/src/utils.ts#L103-L119.

If the workers event loop is similar to Node.JS, I/O and other requests will not be processed between microtasks. So the async implementation would only be less efficient and not have the desired effect of breaking up yielding the CPU to other requests.

Can someone teach me how the Workers event loop is implemented and how to yield to other requests?

BBrett Willis Coming from Node.JS, the event loop is well understood. Particularly that Promis...

kian•8/2/24, 12:41 AM

The first part about microtask queues blocking I/O sounds like the same scenario as described in https://github.com/cloudflare/workerd/blob/ce15e0b82e6408c49ec3e980f23640bd25d6c675/src/workerd/io/io-context.c%2B%2B#L1044-L1065. Unfortunately, looking at the runtime is probably the only 'documentation' about details like this at the moment.

Kkian The first part about microtask queues blocking I/O sounds like the same scenario...

Brett Willis•8/2/24, 1:10 AM

Thanks for pointing me to that code! I won't pretend to understand the code but yeah from reading that comment it seems to suggest that it behaves the same as Node.JS in that microtasks callbacks do not yield to process I/O (and I'm assuming that "I/O" includes other requests).
No benefit to using the async noble-hashesnoble-hashes versions then, just the significant overhead of enqueueing/dequeueing the stack onto the microtask queue.
It seems you also answered the other question: it looks like Workers has it's own event loop implementation.

Brett Willis•8/2/24, 1:16 AM

Another question related to calculating password hashes. The hashing algorithms usually intentionally consume no small amount of memory. While a single computation may fit within the 128MB limit, two or more computations may not, and therefore all in progress requests would get aborted if the worker hits the memory limit. Does the Workers runtime look at the available memory and/or average memory per request when deciding whether so spin up a new instance vs. hit the same instance and potentially hit the limit?
Perhaps this is a good reason to block the thread and not allow other requests to interleave during hashing calculation...

Mehanika•8/2/24, 10:28 AM

I have a project that has a page and worker. Every new github branch for the app spins up a testing environment which is great. I am not finding any resources how to (or if even possible) to do the same with the api/worker. Anyone has experience with this?

jdruwe•8/2/24, 11:53 AM

I added an issues for the next-on-pages project at: https://github.com/cloudflare/next-on-pages/issues/844, I noticed it to be related to the selected worker runtime version. On older versions the code works. Not sure who from the cloudflare team is able to resolve this.

GitHub

Issues · cloudflare/next-on-pages

CLI to build and develop Next.js apps for Cloudflare Pages - Issues · cloudflare/next-on-pages

konedi•8/2/24, 1:17 PM

Hi all, anyway to speed up workers ai through rest api? responses are taking up to 30s, while I have a local llama3.1 running on a rtx3080 and it only takes like 6s?

AngusMa•8/2/24, 3:22 PM

workers-and-pages-helpEmbedding worker in Notion

sia•8/2/24, 4:33 PM

I think I'm missing some basic understanding of how functions and workers can work together. I've binded my worker to my function as mentioned here https://developers.cloudflare.com/pages/functions/bindings/#service-bindings. But the only usage example is:

return context.env.SERVICE.fetch(context.request);

return context.env.SERVICE.fetch(context.request);

But, I don't want to just forward the request sent to the function. I want to send the worker JSON data for it to send an email, and in the function I redirect to a confirmation page. I'm unclear how to interact with the worker from the function

yat•8/2/24, 5:23 PM

does the free plan have cold starts ?

konedi•8/2/24, 6:11 PM

does anyone know why cloudflare worker ai llama 3.1 is 3x slower than local llama 3.1 running on rtx3080? is there no way to speed this up? 30-40 seconds for text generation is insane. I get that it is free credits but damn that is kinda slow

Kkonedi does anyone know why cloudflare worker ai llama 3.1 is 3x slower than local lla...

Chaika•8/2/24, 6:46 PM

?channel-crossposting -> #workers-ai

Flare•8/2/24, 6:46 PM

Please do not post your question in multiple channels/post it multiple times per the rules at ⁠#

welcome-and-rules. It creates confusion for people trying to help you and doesn't get your issue or question solved any faster.

Chaika•8/2/24, 6:46 PM

you have the best chance of being answered there anyway

konedi•8/2/24, 6:47 PM

got it. thanks. sorry, it just seems like all channels are somewhat dead so I needd to see which is the one that is best for this

Chaika•8/2/24, 6:50 PM

people respond/post questions in these channels and don't talk otherwise

does anyone know why cloudflare worker ai llama 3.1 is 3x slower than local llama 3.1 running on rtx3080? is there no way to speed this up? 30-40 seconds for text generation is insane. I get that it is free credits but damn that is kinda slow

AI team explain more in their channel, I'm no AI guy and don't know 100% their setup, but comparing local vs remote seems a bit silly. Workers AI is powered by a ton of shared GPUs, vs your one unshared gpu, and they've got lots of magic in front of it with request routing/etc to try to scale/shard requests. There's lots of different ways to run models too is my understanding, each with different quirks

kabocha•8/2/24, 10:17 PM

does anyone know how long a request to increase subrequest limit takes to process?

also is it possible to increase the subrequest limit on a free project, if the use case requires it?

DanGamble•8/3/24, 12:24 PM

Has anyone been able to get Remix + Vite + Workers working with queues in dev? Feel like you can only pass load-contextload-context to Vite which will just contain

env

env

stuff. The worker/server.tsworker/server.ts is never actually hit so it can't consume the queue

KKashall Is there any way for a worker to tell what region it is running?

Hello, I’m Allie!•8/3/24, 1:16 PM

When handling a fetch event, you can check request.cf.colorequest.cf.colo to get the IATA code

HHello, I’m Allie!When handling a fetch event, you can check `request.cf.colo` to get the IATA cod...

Brett Willis•8/4/24, 12:44 AM

Unless I'm mistaken, that is the client's closest datacentre, not necessarily where the worker is running if smart placement is enabled?

BBrett Willis Unless I'm mistaken, that is the client's closest datacentre, not necessarily wh...

Chaika•8/4/24, 1:16 AM

If smart placement is enabled then request.cf.colo would indeed just be the entry data center and not the worker is running in. You'd have to fetch /cdn-cgi/trace to get the colo then within the worker or look at the cf-placement response header (which you can't get/see within the worker).
In other cases request.cf.colo would be accurate though and it's way easier then those other options

Brett Willis•8/4/24, 2:32 AM

Would /cdn-cgi/trace/cdn-cgi/trace be against any Cloudflare "orange clouded" hostname?

BBrett Willis Would `/cdn-cgi/trace` be against any Cloudflare "orange clouded" hostname?

Chaika•8/4/24, 2:36 AM

You can just fetch https://cloudflare.com/cdn-cgi/tracehttps://cloudflare.com/cdn-cgi/trace

Chaika•8/4/24, 2:37 AM

any proxied/orange cloud hostname would work though yea, although you wouldn't want to depend on something that could change in the future. It's a pretty cheap subrequest as it should be handled by same machine/same location

Brett Willis•8/4/24, 2:37 AM

Ok understood

Barry_Based_Benson•8/4/24, 7:13 PM

What's the correct way to skip a prod worker when the locally-running version of that same worker also needs to access the same route so that it doesn't conflict with the prod worker's code?

Barry_Based_Benson•8/4/24, 7:19 PM

Currently I send a key in a header to skip if the env is prod but it seems a bit messy. Not sure if there's a more official way to do it

AngusMa•8/5/24, 5:36 AM

https://i.imgur.com/iAIi2Z7.png

Imgur

is it possible to open a worker to direct IP connections?

Similar Threads

Similar Threads

Similar Threads