Cloudflare Developers•3y ago

Do you control the client software at all?

Ddave Do you control the client software at all?

McSneaky•7/5/23, 10:30 PM

Jup, but it's huge thing, getting changes thro there is rather long and tedious process

McSneaky•7/5/23, 10:32 PM

But I think for proof-of-concept demo, I'll go cacheless and if things go well, then before production can hopefully figure out and do some changes to client software that would allow direct updating from WS connection

McSneaky•7/5/23, 10:38 PM

It's basically single .js file Vue application, that needs to take data from parent page, that gets it's data from live feed. Tricky part is, that so far all the data has been baked into that JS file at delivery time, so it's huge change architecturally to change it now

McSneaky•7/5/23, 10:40 PM

^ Tool for ad delivery networks and agencies

daveOP•7/5/23, 10:40 PM

Why is serving a 5-10 second out of date file so bad?

Erisa•7/5/23, 10:47 PM

Ordinarily you would see the .js file keyed by a hash in the filename (e.g. script.abcdef.js) and then the html page specifies which one to load, so when you purge cache it updates the html page to point to the new js but the old js stays around for those on the old html

Ddave Why is serving a 5-10 second out of date file so bad?

McSneaky•7/5/23, 10:47 PM

Update needs to happen within some seconds max. File itself could be older actually. But since right now only way to get data in is update file, that needs to be updated
From business side, it's bad if your betting data is 10sec late and decisions are made based on that

Erisa•7/5/23, 10:48 PM

For live updates you should 100% be using a websocket

EErisa Ordinarily you would see the .js file keyed by a hash in the filename (e.g. scri...

McSneaky•7/5/23, 10:48 PM

Aah, that's actually interesting idea. Currently hashes are rather random. But could make them kinda guessable. So script within HTML can try to "guess" next hash and ping that every x time

MMcSneaky Update needs to happen within some seconds max. File itself could be older actua...

daveOP•7/5/23, 10:48 PM

first question, are you using Workers already?

Ddave first question, are you using Workers already?

McSneaky•7/5/23, 10:49 PM

Jup, but not modifying files. Thought about that, but modifying them on that scale at delivery time would be too costly sadly

MMcSneaky Aah, that's actually interesting idea. Currently hashes are rather random. But c...

Erisa•7/5/23, 10:49 PM

Not quite what I had in mind but for your use case that.. might work? Maybe?

EErisa For live updates you should 100% be using a websocket

McSneaky•7/5/23, 10:49 PM

Yeah.. Dealing with legacy stuff.. Trying to figure out faster workarounds

EErisa Not quite what I had in mind but for your use case that.. might work? Maybe?

McSneaky•7/5/23, 10:50 PM

So instead of random hashes I'd have like myfile.v1.jsmyfile.v1.js, myfile.v2.jsmyfile.v2.js, myfile.v3.jsmyfile.v3.js etc

MMcSneaky So instead of random hashes I'd have like `myfile.v1.js`, `myfile.v2.js`, `myfil...

daveOP•7/5/23, 10:50 PM

have

myfile.latest.js

myfile.latest.js

that does a 303 redirect to whatever the current version is

Ddave have `myfile.latest.js` that does a 303 redirect to whatever the current version...

McSneaky•7/5/23, 10:51 PM

Aah! That's real good idea!

MMcSneaky Aah! That's real good idea!

daveOP•7/5/23, 10:51 PM

it's not an original one, I can't take credit

MMcSneaky So instead of random hashes I'd have like `myfile.v1.js`, `myfile.v2.js`, `myfil...

Erisa•7/5/23, 10:51 PM

I see, yeah what dave said

Erisa•7/5/23, 10:51 PM

I've seen that done with some success

McSneaky•7/5/23, 10:52 PM

Noice. Thnx for discussions!
I can do redirects thro API all good

And that will work in production long enuf until I (or someone else) figure out how to update client side legacy code to support WS directly

Erisa•7/5/23, 10:53 PM

websockets are OP for live updates, its how your discord client is showing all our messages in realtime

daveOP•7/5/23, 10:54 PM

This is a guess:

Put these headers on the redirected

myfile.latest.js

myfile.latest.js

cache-control: private, no-transform, no-cache, no-store, max-age=0, s-maxage=0

cache-control: private, no-transform, no-cache, no-store, max-age=0, s-maxage=0

And these headers on

myfile.v*.js

myfile.v*.js

cache-control: private, immutable, no-transform, max-age=31536000, s-maxage=31536000

cache-control: private, immutable, no-transform, max-age=31536000, s-maxage=31536000

You should change

private

private

public

public

if these files aren't custom per-user.

daveOP•7/5/23, 10:56 PM

personally I would recommend using this for the redirect:

https://cdn.example.com/js/@teamname/productname@latest/index.js

https://cdn.example.com/js/@teamname/productname@latest/index.js

And like this for the versioned files:

https://cdn.example.com/js/@teamname/productname@1.2.3/index.js

https://cdn.example.com/js/@teamname/productname@1.2.3/index.js

daveOP•7/5/23, 10:56 PM

that's how https://cdn.jsdelivr.net/npm/@stoplight/elements@latest/ does it iirc

@stoplight/elements CDN by jsDelivr - A free, fast, and reliable Op...

@stoplight/elements CDN by jsDelivr - A free, fast, and reliable Open Source CDN for npm and GitHub

EErisa websockets are OP for live updates, its how your discord client is showing all o...

McSneaky•7/5/23, 10:59 PM

We were using WS, that was many years ago. We weren't using Cloudflare nor did Cloudflare support websockets. Then at some point we got 250k new WS connections per second, peaking at about 10-15m. Ofc our servers died under that (each connection had to deal with quite some DB reads and writes etc). We were estimating scale of about 300 concurrent connections..

So I got the idea of bundling it all into JS files, since we really didn't need that much of an realtime
Now times have changed and we need realtime again.. But team who made move from WS to bundled JS isn't around anymore, so it's rather interesting digging.. I kinda know what should be there in codebase, but not too sure. It's been over 3y since I worked on that codebase. For all that time it had 5 people on it full-time, so it's been totally rewritten about 2-3 times for sure

Erisa•7/5/23, 11:00 PM

nor did Cloudflare support websockets

dang, that must have been quite some time ago

Erisa•7/5/23, 11:01 PM

blog says they were supported in 2014

McSneaky•7/5/23, 11:02 PM

For our use-case we needed (and still need) connection to go to workers, but workers weren't a thing then. Now with durable objects it became possible to move it all to CF

EErisa > nor did Cloudflare support websockets dang, that must have been quite some tim...

daveOP•7/5/23, 11:04 PM

It was enterprise only, at least in June 2015 when I asked.

MMcSneaky For our use-case we needed (and still need) connection to go to workers, but wor...

McSneaky•7/5/23, 11:06 PM

It's in dev plan, when we get to it is other thing..
Other thing in dev plan is getting really good deal from CF for 10b worker requests to run some AI models on the edge

Erisa•7/5/23, 11:08 PM

by my rough maths, 10 bil workers bundled reqs on paygo plan would be like $5,000 or so

daveOP•7/5/23, 11:09 PM

that's less than payroll for most folks (in the US anyway)

Erisa•7/5/23, 11:10 PM

indeed

McSneaky•7/5/23, 11:12 PM

Most likely as enterprise customer we'd get off from there, but money is tight for business currently. Since market went interesting

McSneaky•7/5/23, 11:12 PM

So it hasn't been priority

McSneaky•7/5/23, 11:13 PM

I hope once we kill our own small bare-metal server farm, that keeps super legacy stuff running we can redirect that cash flow to fancy CF stuff

Erisa•7/5/23, 11:15 PM

I'm sure our sales team would love a chat when the time comes, if the paygo plan isnt enticing

McSneaky•7/5/23, 11:16 PM

You have quite some good fellas in sales team
Can't remember the name of fella who was our sales contact. But he always got us real sweet deals
He also got us hooked in more cool stuff than we had time to implement

MMcSneaky It's in dev plan, when we get to it is other thing.. Other thing in dev plan is ...

daveOP•7/5/23, 11:16 PM

hopefully you're making money on those 10 billion requests?

Ddave hopefully you're making money on those 10 billion requests?

McSneaky•7/5/23, 11:18 PM

We currently make money already based on them. So adding edge-ai on top of that would be just cost at the beginning, until we finish tuning our models
After that can sell it to clients as additional feature for extra cost

erichileman•7/5/23, 11:18 PM

Anyone know where the cloudlfare blog article / how to is for chatgpt, workers, and using kv for the vector db storage? It was an example loading in cf's own documentation. I could have sworn I read one a few weeks back but now can't find it...

McSneaky•7/5/23, 11:18 PM

For how long it would be just a cost, is unknown. That's why business isn't too keen to jump that bandwagon yet + we are in eastern Europe. For that money you can get 2 devs in here

Eerichileman Anyone know where the cloudlfare blog article / how to is for chatgpt, workers, ...

McSneaky•7/5/23, 11:20 PM

Hmm, someone at work shared similar article some time ago. Lemme see if can find it from history somewhere

Eerichileman Anyone know where the cloudlfare blog article / how to is for chatgpt, workers, ...

McSneaky•7/5/23, 11:21 PM

https://blog.cloudflare.com/langchain-and-cloudflare/
That one?

The Cloudflare Blog

Using LangChainJS and Cloudflare Workers together

We are incredibly stoked that our friends at LangChain have announced LangChainJS Support for Cloudflare Workers! In this post we'll show you how to build a sample application.

MMcSneaky https://blog.cloudflare.com/langchain-and-cloudflare/ That one?

erichileman•7/5/23, 11:25 PM

Thank you, but I don't think it was that one. The one I'm thinking of was specifically loading cf's own knowledgebase in. This one is using a wikipedia article on Brooklyn.

Eerichileman Thank you, but I don't think it was that one. The one I'm thinking of was speci...