uhh, this is probably a dumb question... but do Durable Objects have to be serializable? or could th

uhh, this is probably a dumb question... but do Durable Objects have to be serializable? or could they contain something like an RxJS Observable that you can push to between incoming requests?

davidbarrattOP•8/12/21, 5:55 PM

basically I want to receive "purge" requests from an Amazon SNS and group the tags together and send a request to Cloudflare's API whenever I get up to 30 or every 3 seconds (whichever comes sooner) so that way we are not spamming their API.

Ddavidbarratt uhh, this is probably a dumb question... but do Durable Objects have to be seria...

John Spurlock•8/12/21, 6:00 PM

they do not need to use at all if that's what you're asking, can do stuff only in memory. The classic websocket chat server for example can be done without storage if necessary. DOs can be just a nice centralized coordination point

davidbarrattOP•8/12/21, 6:00 PM

interesting, and can you use within a worker?

davidbarrattOP•8/12/21, 6:01 PM

errr

davidbarrattOP•8/12/21, 6:01 PM

within the Durable Object

Ddavidbarratt interesting, and can you use `setTimeout()` within a worker?

John Spurlock•8/12/21, 6:01 PM

yep

davidbarrattOP•8/12/21, 6:01 PM

nice!

davidbarrattOP•8/12/21, 6:04 PM

I think this is going to make my life a lot easier.

davidbarrattOP•8/12/21, 6:06 PM

another question... is it possible to force a specific location rather than just wherever it is hit first?

Ddavidbarratt another question... is it possible to force a specific location rather than just...

John Spurlock•8/12/21, 6:09 PM

a specific colo? no - they are kind of like normal workers in that respect. but once they are instantiated (i believe they try to do the first instantiation close to the normal worker that spawned it), they pretty much stay put

davidbarrattOP•8/12/21, 6:12 PM

right, I'm trying to reduce the latency from our AWS region (Ashburn, VA) and I was wondering if I could force it in that colo rather than wherever someone else hits it

Ddavidbarratt right, I'm trying to reduce the latency from our AWS region (Ashburn, VA) and I ...

John Spurlock•8/12/21, 6:14 PM

you could vpn to north va or hit it first from ec2 or lambda or something inside us-east-1

John Spurlock•8/12/21, 6:14 PM

but there's no guarantee

davidbarrattOP•8/12/21, 6:14 PM

right

John Spurlock•8/12/21, 6:14 PM

could always check the colo once created and keep trying, using a different id or idFromName in the stub call

John Spurlock•8/12/21, 6:16 PM

but none of this behavior is guaranteed (they might be doing a server migration e.g.) and might change in the future etc

davidbarrattOP•8/12/21, 6:17 PM

interesting ok, I guess that is a feature request for the workers team... kind of like "get me as close to X location as you can" kind of thing

John Spurlock•8/12/21, 6:20 PM

i would love this as well - right now you can specify the "eu jurisdictional area" but that's the only supported physical constraint you can set

davidbarrattOP•8/12/21, 6:25 PM

Is the Durable Object smart enough to know the setTimeout is still running or do you have to put that in the waitUntil ?

John Spurlock•8/12/21, 6:27 PM

i believe it's smart enough, iirc they said recently waitUntil has no benefit in DO workers - I've used setTimeout outside of waitUntil and it seems to work - maybe one of the cloudflare folks can confirm this is still the case - things change quickly in DO land : )

John Spurlock•8/12/21, 6:28 PM

lots of good stuff in the scrollback for this channel

davidbarrattOP•8/12/21, 6:34 PM

well this is all great news because I think I can do what I want in a Worker without having to make/deploy a Node.js service.

ItsWendell•8/12/21, 9:38 PM

The DO storage is generally based on input order right?

ItsWendell•8/12/21, 9:47 PM

Any tips regarding this? I implemented some sort of chat history in https://edgechat-graphql.vercel.app which basically writes the chat messages in the DO 'storage' before emitting to the chatroom, but the output once I actually fetch the messages is this

ItsWendell•8/12/21, 9:47 PM

IItsWendell The DO storage is generally based on input order right?

John Spurlock•8/12/21, 9:50 PM

Nope, it's based on the lexicographical ordering of the key

ItsWendell•8/12/21, 9:53 PM

Oh okay that makes a lot of sense, time to move from nanoid to ulids

ItsWendell•8/12/21, 9:54 PM

That's also actually quite nice since now I don't need to actually 'wait' before it's stored to emit it back into the chatroom

John Spurlock•8/12/21, 9:56 PM

uilds would work - sometimes its useful to do your own key scheme in case you ever need to use filtering as an optimization

JJohn Spurlock uilds would work - sometimes its useful to do your own key scheme in case you ev...

ItsWendell•8/12/21, 9:58 PM

What do you mean? I do now as a key-scheme.

John Spurlock•8/12/21, 9:59 PM

ah then you already are doing it : ) entity sounds like your filter

John Spurlock•8/12/21, 10:00 PM

totally application-specific tho

ItsWendell•8/12/21, 10:00 PM

Yeah exactly haha, yeah in my DO for the chatrooms I do , probably have to purge the current storages somehow though now

John Spurlock•8/12/21, 10:04 PM

or you could do if you want to keep the old data around for migration - but it looks like you are building something new so probably nothing major to keep

ItsWendell•8/12/21, 10:05 PM

No nothing major, any tips on clearing that store?

John Spurlock•8/12/21, 10:06 PM

https://developers.cloudflare.com/workers/runtime-apis/durable-objects

John Spurlock•8/12/21, 10:06 PM

but that will nuke everything for that DO instance

John Spurlock•8/12/21, 10:07 PM

if you just want to clear you will need to

list

list

with a then make calls

ItsWendell•8/12/21, 10:09 PM

Yeah maybe I'll just use another prefix

John Spurlock•8/12/21, 10:11 PM

announcing Durable Object Stored Procedures

ItsWendell•8/12/21, 10:13 PM

That will probably work depending on the size of your Twitch viewers. The thing is that the DO will be created closest to the request initiating the DO, and Twitch will probably output all these messages from only 1 specific location.

ItsWendell•8/12/21, 10:16 PM

So if DO's are the right use-case? Maybe not. There is a runtime memory limit of only 128MB, and since you'll only be able to utilize a 'few' DOs depending on how you structure this it might run out quick depending on the heaviness of the game server and the amount of users.

ItsWendell•8/12/21, 10:19 PM

It might be worth trying though, since Twitch will probably be the 'only' one connected to your DO, since the messages probably come from their API right?

ItsWendell•8/12/21, 10:21 PM

Apparently Twitch supports Pubsub on certain events through Websockets, which you can connect to in your Durable Object. (https://dev.twitch.tv/docs/pubsub#topics)

ItsWendell•8/12/21, 10:42 PM

You'll need the durable objects to 'save' the game state, take input and communicate some output back to workers or other clients.