could maybe use some other proxy server (like nginx on a vps) to proxy the requests to workers but t

could maybe use some other proxy server (like nginx on a vps) to proxy the requests to workers but that'd be another step

CChaika could maybe use some other proxy server (like nginx on a vps) to proxy the reque...

infinite cat gen•8/1/24, 5:23 PM

maybe ill be able to do that

infinite cat gen•8/1/24, 5:23 PM

lemme see

infinite cat gen•8/1/24, 5:26 PM

nah i think ill just try something different

Jonathan Silva•8/1/24, 6:10 PM

Good afternoon
How to resolve this error in request.json() ?

JJonathan Silva Good afternoon How to resolve this error in request.json() ?

Idle•8/1/24, 6:11 PM

most likely by catching it or making sure that the request body has a valid json format?

Frank•8/1/24, 9:49 PM

is there a limit on how many custom cache namespace we can create?

Hyper•8/1/24, 10:14 PM

Hey guys. I want to create a notification system but im not sure where to start. my original idea was using durable objects but i think the limit of 32,000 active connections makes it not scalable, and if i go for a 1:1 architcture then it would be difficult to communicate with other DOs. My next idea is SSE using the event source API. What I want to achieve is essentially when a request is made to my API (which runs on workers), some requests will trigger notifications to reach certain clients. Id like to communicate the data to those specific clients. Any ideas or suggestions?

For example, user A sends a friend request to user B by calling an API endpoint on my worker. I want to push the notification to user B. Any help would be appreciated

Brett Willis•8/2/24, 12:25 AM

Coming from Node.JS, the event loop is well understood. Particularly that Promise callbacks operate on the microtask queue and I/O is not processed in between microtasks. Unless I've missed something in my searches, very little is described about the event loop in Workers and how/when the event loop processes other requests (does it use the "default" V8 event loop implementation, and how does that work?).

The reason I ask is is: how to break up long-running CPU-intensive calculations to allow other requests to be processed in between? E.g. calculating password hashes using

noble-hashes

noble-hashes

. That library provides an async implementation which just uses promise callbacks (microtask queue) to break up the calculation: https://github.com/paulmillr/noble-hashes/blob/ae060daa6252f3ff2aa2f84e887de0aab491281d/src/utils.ts#L103-L119.

If the workers event loop is similar to Node.JS, I/O and other requests will not be processed between microtasks. So the async implementation would only be less efficient and not have the desired effect of breaking up yielding the CPU to other requests.

Can someone teach me how the Workers event loop is implemented and how to yield to other requests?

BBrett Willis Coming from Node.JS, the event loop is well understood. Particularly that Promis...

kian•8/2/24, 12:41 AM

The first part about microtask queues blocking I/O sounds like the same scenario as described in https://github.com/cloudflare/workerd/blob/ce15e0b82e6408c49ec3e980f23640bd25d6c675/src/workerd/io/io-context.c%2B%2B#L1044-L1065. Unfortunately, looking at the runtime is probably the only 'documentation' about details like this at the moment.

Kkian The first part about microtask queues blocking I/O sounds like the same scenario...

Brett Willis•8/2/24, 1:10 AM

Thanks for pointing me to that code! I won't pretend to understand the code but yeah from reading that comment it seems to suggest that it behaves the same as Node.JS in that microtasks callbacks do not yield to process I/O (and I'm assuming that "I/O" includes other requests).
No benefit to using the async

noble-hashes

noble-hashes

versions then, just the significant overhead of enqueueing/dequeueing the stack onto the microtask queue.
It seems you also answered the other question: it looks like Workers has it's own event loop implementation.

Brett Willis•8/2/24, 1:16 AM

Another question related to calculating password hashes. The hashing algorithms usually intentionally consume no small amount of memory. While a single computation may fit within the 128MB limit, two or more computations may not, and therefore all in progress requests would get aborted if the worker hits the memory limit. Does the Workers runtime look at the available memory and/or average memory per request when deciding whether so spin up a new instance vs. hit the same instance and potentially hit the limit?
Perhaps this is a good reason to block the thread and not allow other requests to interleave during hashing calculation...

Mehanika•8/2/24, 10:28 AM

I have a project that has a page and worker. Every new github branch for the app spins up a testing environment which is great. I am not finding any resources how to (or if even possible) to do the same with the api/worker. Anyone has experience with this?

jdruwe•8/2/24, 11:53 AM

I added an issues for the next-on-pages project at: https://github.com/cloudflare/next-on-pages/issues/844, I noticed it to be related to the selected worker runtime version. On older versions the code works. Not sure who from the cloudflare team is able to resolve this.

GitHub

Issues · cloudflare/next-on-pages

CLI to build and develop Next.js apps for Cloudflare Pages - Issues · cloudflare/next-on-pages

konedi•8/2/24, 1:17 PM

Hi all, anyway to speed up workers ai through rest api? responses are taking up to 30s, while I have a local llama3.1 running on a rtx3080 and it only takes like 6s?

AngusMa•8/2/24, 3:22 PM

workers-and-pages-helpEmbedding worker in Notion

sia•8/2/24, 4:33 PM

I think I'm missing some basic understanding of how functions and workers can work together. I've binded my worker to my function as mentioned here https://developers.cloudflare.com/pages/functions/bindings/#service-bindings. But the only usage example is:

return context.env.SERVICE.fetch(context.request);

return context.env.SERVICE.fetch(context.request);

But, I don't want to just forward the request sent to the function. I want to send the worker JSON data for it to send an email, and in the function I redirect to a confirmation page. I'm unclear how to interact with the worker from the function

yat•8/2/24, 5:23 PM

does the free plan have cold starts ?

konedi•8/2/24, 6:11 PM

does anyone know why cloudflare worker ai llama 3.1 is 3x slower than local llama 3.1 running on rtx3080? is there no way to speed this up? 30-40 seconds for text generation is insane. I get that it is free credits but damn that is kinda slow

Kkonedi does anyone know why cloudflare worker ai llama 3.1 is 3x slower than local lla...

ChaikaOP•8/2/24, 6:46 PM

?channel-crossposting -> #workers-ai

Flare•8/2/24, 6:46 PM

Please do not post your question in multiple channels/post it multiple times per the rules at ⁠#

welcome-and-rules. It creates confusion for people trying to help you and doesn't get your issue or question solved any faster.

ChaikaOP•8/2/24, 6:46 PM

you have the best chance of being answered there anyway

konedi•8/2/24, 6:47 PM

got it. thanks. sorry, it just seems like all channels are somewhat dead so I needd to see which is the one that is best for this

ChaikaOP•8/2/24, 6:50 PM

people respond/post questions in these channels and don't talk otherwise

does anyone know why cloudflare worker ai llama 3.1 is 3x slower than local llama 3.1 running on rtx3080? is there no way to speed this up? 30-40 seconds for text generation is insane. I get that it is free credits but damn that is kinda slow

AI team explain more in their channel, I'm no AI guy and don't know 100% their setup, but comparing local vs remote seems a bit silly. Workers AI is powered by a ton of shared GPUs, vs your one unshared gpu, and they've got lots of magic in front of it with request routing/etc to try to scale/shard requests. There's lots of different ways to run models too is my understanding, each with different quirks

kabocha•8/2/24, 10:17 PM

does anyone know how long a request to increase subrequest limit takes to process?

also is it possible to increase the subrequest limit on a free project, if the use case requires it?

DanGamble•8/3/24, 12:24 PM

Has anyone been able to get Remix + Vite + Workers working with queues in dev? Feel like you can only pass

load-context

load-context

to Vite which will just contain

env

env

stuff. The

worker/server.ts

worker/server.ts

is never actually hit so it can't consume the queue

KKashall Is there any way for a worker to tell what region it is running?

Hello, I’m Allie!•8/3/24, 1:16 PM

When handling a fetch event, you can check

request.cf.colo

request.cf.colo

to get the IATA code

HHello, I’m Allie!When handling a fetch event, you can check `request.cf.colo` to get the IATA cod...

Brett Willis•8/4/24, 12:44 AM

Unless I'm mistaken, that is the client's closest datacentre, not necessarily where the worker is running if smart placement is enabled?

BBrett Willis Unless I'm mistaken, that is the client's closest datacentre, not necessarily wh...

ChaikaOP•8/4/24, 1:16 AM

If smart placement is enabled then request.cf.colo would indeed just be the entry data center and not the worker is running in. You'd have to fetch /cdn-cgi/trace to get the colo then within the worker or look at the cf-placement response header (which you can't get/see within the worker).
In other cases request.cf.colo would be accurate though and it's way easier then those other options

Brett Willis•8/4/24, 2:32 AM

Would

/cdn-cgi/trace

/cdn-cgi/trace

be against any Cloudflare "orange clouded" hostname?

BBrett Willis Would `/cdn-cgi/trace` be against any Cloudflare "orange clouded" hostname?

ChaikaOP•8/4/24, 2:36 AM

You can just fetch

https://cloudflare.com/cdn-cgi/trace

https://cloudflare.com/cdn-cgi/trace

ChaikaOP•8/4/24, 2:37 AM

any proxied/orange cloud hostname would work though yea, although you wouldn't want to depend on something that could change in the future. It's a pretty cheap subrequest as it should be handled by same machine/same location

Brett Willis•8/4/24, 2:37 AM

Ok understood

Barry_Based_Benson•8/4/24, 7:13 PM

What's the correct way to skip a prod worker when the locally-running version of that same worker also needs to access the same route so that it doesn't conflict with the prod worker's code?

Barry_Based_Benson•8/4/24, 7:19 PM

Currently I send a key in a header to skip if the env is prod but it seems a bit messy. Not sure if there's a more official way to do it

AngusMa•8/5/24, 5:36 AM

https://i.imgur.com/iAIi2Z7.png

Imgur

AngusMa•8/5/24, 5:36 AM

The devtools on the dashboard is blocked by Microsoft Edge

ItsWendell•8/5/24, 8:09 AM

I've spotted TraceMetrics while I was working on my tail worker in the typescript types for

@clouflare/workers-types

@clouflare/workers-types

export interface TraceMetrics {
  readonly cpuTime: number;
  readonly wallTime: number;
}
export interface UnsafeTraceMetrics {
  fromTrace(item: TraceItem): TraceMetrics;
}

export interface TraceMetrics {
  readonly cpuTime: number;
  readonly wallTime: number;
}
export interface UnsafeTraceMetrics {
  fromTrace(item: TraceItem): TraceMetrics;
}

Is this already available under a flag?

IItsWendell I've spotted TraceMetrics while I was working on my tail worker in the typescrip...

Isaac McFadyen•8/5/24, 2:22 PM

Usually the types marked unsafe are either for internal use only or only in the non-production (i.e. selfhosted)

workerd

workerd

runtime.

IIsaac McFadyen Usually the types marked unsafe are either for internal use only or only in the ...

ItsWendell•8/5/24, 2:24 PM

Yeah I already suspected that, I even tried to see if the production global scope of a tail worker had these available but sadly no, would be great to have it available in a tail / trace worker to better debug CPU spikes

Rohan•8/5/24, 2:26 PM

Hey guys im new to cloudflare so bear with me
my question is whether I can run a tensorflow.js program in worker
its basically a model to convert chess images to FEN(notation)
from what i searched the answer is kinda vague

Rohan•8/5/24, 2:27 PM

I dont think it can run in python worker from what i searched since its pure python

Rohan•8/5/24, 2:29 PM

https://github.com/cloudflare/tensorflow-nata (look at the end of readme file)
they kinda something similar here but also used a gpu server so idk

GitHub

GitHub - cloudflare/tensorflow-nata: Our model uses a convolutional...

Our model uses a convolutional neural network and TensorFlow to infer if an image is a Pastel de Nata or not. We trained it with thousands of Portuguese egg custard tart (Pasteis de Nata) images an...

RRohan Hey guys im new to cloudflare so bear with me my question is whether I can run ...

Isaac McFadyen•8/5/24, 2:30 PM

That's correct (re Python workers). You might be able to run via Tensorflow.js assuming the combination of runtime and model is light enough - under 10MB combined.

Isaac McFadyen•8/5/24, 2:30 PM

I'm assuming tensorflow.js uses WASM which is supported on Workers with the caveat of the 10MB size limit total.

Rohan•8/5/24, 2:32 PM

Okay thank you so much

0xKIBO•8/5/24, 3:32 PM

How can I import hosted JS code in Cloudflare workers

00xKIBO How can I import hosted JS code in Cloudflare workers

Hard@Work•8/5/24, 3:36 PM

You cannot. All code you wish to run must be already deployed to Cloudflare

kunal•8/5/24, 9:34 PM

I couldn't tell, how is the Workers Rate Limiting functionality billed?
https://developers.cloudflare.com/workers/runtime-apis/bindings/rate-limit/

kunal•8/5/24, 9:36 PM

I see it was free during the beta period, but curious how it will be charged after it graduates

could maybe use some other proxy server (like nginx on a vps) to proxy the requests to workers but t

Similar Threads

Similar Threads

Similar Threads