The DO storage is generally based on input order right?

ItsWendellOP•8/12/21, 9:47 PM

Any tips regarding this? I implemented some sort of chat history in https://edgechat-graphql.vercel.app which basically writes the chat messages in the DO 'storage' before emitting to the chatroom, but the output once I actually fetch the messages is this

ItsWendellOP•8/12/21, 9:47 PM

IItsWendell The DO storage is generally based on input order right?

John Spurlock•8/12/21, 9:50 PM

Nope, it's based on the lexicographical ordering of the key

ItsWendellOP•8/12/21, 9:53 PM

Oh okay that makes a lot of sense, time to move from nanoid to ulids

ItsWendellOP•8/12/21, 9:54 PM

That's also actually quite nice since now I don't need to actually 'wait' before it's stored to emit it back into the chatroom

John Spurlock•8/12/21, 9:56 PM

uilds would work - sometimes its useful to do your own key scheme in case you ever need to use

prefix

prefix

filtering as an optimization

JJohn Spurlock uilds would work - sometimes its useful to do your own key scheme in case you ev...

ItsWendellOP•8/12/21, 9:58 PM

What do you mean? I do entity:IDentity:ID now as a key-scheme.

John Spurlock•8/12/21, 9:59 PM

ah then you already are doing it : ) entity sounds like your

prefix

prefix

filter

John Spurlock•8/12/21, 10:00 PM

totally application-specific tho

ItsWendellOP•8/12/21, 10:00 PM

Yeah exactly haha, yeah in my DO for the chatrooms I do message:IDmessage:ID, probably have to purge the current storages somehow though now

John Spurlock•8/12/21, 10:04 PM

or you could do message2:newidmessage2:newid if you want to keep the old data around for migration - but it looks like you are building something new so probably nothing major to keep

ItsWendellOP•8/12/21, 10:05 PM

No nothing major, any tips on clearing that store?

John Spurlock•8/12/21, 10:06 PM

deleteAll

deleteAll

https://developers.cloudflare.com/workers/runtime-apis/durable-objects

John Spurlock•8/12/21, 10:06 PM

but that will nuke everything for that DO instance

John Spurlock•8/12/21, 10:07 PM

if you just want to clear message:message: you will need to

list

list

with a

prefix

prefix

then make delete(keys)delete(keys) calls

ItsWendellOP•8/12/21, 10:09 PM

Yeah maybe I'll just use another prefix

John Spurlock•8/12/21, 10:11 PM

announcing Durable Object Stored Procedures

ItsWendellOP•8/12/21, 10:13 PM

That will probably work depending on the size of your Twitch viewers. The thing is that the DO will be created closest to the request initiating the DO, and Twitch will probably output all these messages from only 1 specific location.

ItsWendellOP•8/12/21, 10:16 PM

So if DO's are the right use-case? Maybe not. There is a runtime memory limit of only 128MB, and since you'll only be able to utilize a 'few' DOs depending on how you structure this it might run out quick depending on the heaviness of the game server and the amount of users.

ItsWendellOP•8/12/21, 10:19 PM

It might be worth trying though, since Twitch will probably be the 'only' one connected to your DO, since the messages probably come from their API right?

ItsWendellOP•8/12/21, 10:21 PM

Apparently Twitch supports Pubsub on certain events through Websockets, which you can connect to in your Durable Object. (https://dev.twitch.tv/docs/pubsub#topics)

ItsWendellOP•8/12/21, 10:42 PM

You'll need the durable objects to 'save' the game state, take input and communicate some output back to workers or other clients.

ItsWendellOP•8/12/21, 10:43 PM

Having a DO for each character will make that 'harder' since you wan to know where that character is.

IItsWendell No nothing major, any tips on clearing that store?

Erwin•8/13/21, 3:40 AM

@john.spurlock's

deleteAll()

deleteAll()

will delete everything for that DO instance.. if you want to delete all the data in every DO, things get a bit more complex. If I remember correctly deleting the namespace is the way to go (but I can't remember if you also needed to wait for DOs to be evicted)..

If you wanted to start from scratch, I would probably just delete the old binding and start over with a new binding name.

FFuglen Thank you, that helped me a lot to understand this more 🙂 Yes, I will use the ...

Erwin•8/13/21, 3:47 AM

We have the absolutely best community here..

But first things first.. how many people do you think would be playing the game at any one time? Which you can translate into how many commands would be going into the DO. Some internal tests we were doing on one of our own projects showed that a single DO could handle 200+ requests/second with relatively little latency if the time it takes to process that message is small enough.

And the other limitation would be the 128MB of memory, but I don't think that necessarily has to be a big issue, assuming that a lot of the rendering logic will be done in the browser app and the DO is mostly used for storing state and passing messages from the chat to the browser?

Erwin•8/13/21, 5:46 AM

What do you mean work with Workers Sites?

berkant•8/13/21, 6:42 AM

Is there a way to know the latency between the edge PoP serving user requests and the DO colo?

Erwin•8/13/21, 7:27 AM

Sort of roughly yes, you can do a

const start = Date.now()
const response = await stub.fetch(request)
const durationMs = Date.now() - start

const start = Date.now()
const response = await stub.fetch(request)
const durationMs = Date.now() - start

Erwin•8/13/21, 7:28 AM

It isn't perfect, because the Date.now() isn't measured from that point, but the last time there was any IO.. so it won't be accurate to the millisecond, but it should give you an order of magnitude measurement.

Erwin•8/13/21, 7:29 AM

Also note that that latency is very sensitive to where the user Worker is. Right now the DO coverage is less than ideal in places like South America and Australia

TTill Do DO work with Worker Sites, yet? I asked 6+ months ago and it wasn’t ready.

GregBrimble•8/13/21, 10:14 AM

Not quite yet, I don't think. @James maybe knows a bit more, but this issue (https://github.com/cloudflare/kv-asset-handler/issues/174) and open PR (https://github.com/cloudflare/kv-asset-handler/pull/200) suggest that Workers Sites aren't yet possible (at least without some hacks) with the ES Modules format (which is required for Durable Objects).

(Sorry for all the parenthesis)

GitHub

Not compatible with the new ES6 module worker format · Issue #174 ·...

It seems that this package still has the assumption of the old worker format (e.g. global environment variables, FetchEvent, etc) and does not work properly with the new ES6 module format. I would ...

GitHub

Support ES6 Module Syntax by DrewRidley · Pull Request #200 · clou...

Here I introduce a possible solution for handling the retrieval of assets from the KV when using the module syntax in your workers. You can see in my PR that instead of FetchEvent, the method takes...

GGregBrimble Not quite yet, I don't think. @James maybe knows a bit more, but this issue (htt...

GregBrimble•8/13/21, 10:14 AM

cc. @evanderkoogh

Erwin•8/13/21, 10:22 AM

Ah cheers @gregbrimble.. There is also little priority to put a lot of work into Worker sites (pun intended).. as Pages is the way forward for those..

ItsWendellOP•8/13/21, 10:57 AM

How do you have it setup?

ItsWendellOP•8/13/21, 12:38 PM

You could try setting up some kind of ping system from your DO to the client where you ping the sockets every x seconds, I personally haven't experienced this I am also building a similar project that's a little bit more advanced than workers chat demo.

GGregBrimble Not quite yet, I don't think. @James maybe knows a bit more, but this issue (htt...

James•8/13/21, 1:31 PM

Yeah, it’s a bit confusing right now. I personally think the best idea is to wait for ESM and DO to leave beta, and then release a new version of kv-asset-handler with breaking changes, and a new API for ESM. Maybe a version with support for both in the interim, but either way https://github.com/cloudflare/wrangler/issues/1938 will need to be resolved first.

GitHub

__STATIC_CONTENT_MANIFEST missing in env object when using ES modul...

Bug report Describe the bug When using the new ES module syntax and rollup instead of webpack as a bundler, I cannot access the workers-site binding __STATIC_CONTENT_MANIFEST neither through the ...

James•8/13/21, 1:33 PM

I love Pages, but there’s definitely still a valid use case for Sites imo, and anyone doing anything complex with workers and assets.