DO storage uses the structured clone algorithm. Do you know if there are any data protocol libraries

DO storage uses the structured clone algorithm. Do you know if there are any data protocol libraries available that can send that same data over the wire to a client application? protobuf or messagepack or anything?

Unsmart•3/7/23, 10:20 PM

Would be neat if DOs took in smart placement but a little bit more complex rather than just latency to external fetches took in the latency to requester locations and tried to place it in the best location latency wise for all of request latency and external fetch latency. I assume that would be very complex though

Jacob WrightOP•3/7/23, 10:20 PM

I'd like to create a data console to inspect the data in my durable objects and am deciding if I should just support JSON or all the types

Jacob WrightOP•3/7/23, 10:21 PM

I suppose I should have asked first if there are any GUI/UI consoles to manage DO storage. That isn't something Cloudflare is working on already, is it?

UUnsmart Would be neat if DOs took in smart placement but a little bit more complex rathe...

Jacob WrightOP•3/7/23, 10:24 PM

You mean, it would try and sit somewhere between the user and backend servers it has to make requests to instead of as close to the user as possible?

Unsmart•3/7/23, 10:25 PM

It would do both. Try to be as close to the users and data as possible latency wise.

Jacob WrightOP•3/7/23, 10:25 PM

That would get complicated.

Jacob WrightOP•3/7/23, 10:25 PM

But it would be cool

Unsmart•3/7/23, 10:25 PM

I am well aware thats why I said it would probably be really complex

Jacob WrightOP•3/7/23, 10:25 PM

Yeah

JJacob Wright DO storage uses the structured clone algorithm. Do you know if there are any dat...

Jacob WrightOP•3/7/23, 10:26 PM

It appears https://github.com/kriszyp/msgpackr can support structured cloning

Unsmart•3/7/23, 10:27 PM

I dont really do external fetches on DOs much so in that case it would basically choose the lowest RTT of all users who request the specific DO

Jacob WrightOP•3/7/23, 10:48 PM

Feature request: add

name

name

field to returned result of https://api.cloudflare.com/#durable-objects-namespace-list-objects for objects that were created with idFromName(name)idFromName(name). Thanks!

JJacob Wright **Feature request:** add `name` field to returned result of https://api.cloudfla...

Jacob WrightOP•3/7/23, 11:10 PM

Actually, I probably need more flexibility with storing/loading my DOs, so I'll probably use D1 or something else to keep track of them rather than using the API.

JJacob Wright It appears https://github.com/kriszyp/msgpackr can support structured cloning

Larry•3/8/23, 1:30 AM

I use https://github.com/kriszyp/cbor-x by the same author. CBOR is essentially msgpack but with an international standard. It's fast and easy to work with.

GitHub

GitHub - kriszyp/cbor-x: Ultra-fast CBOR encoder/decoder with exten...

Ultra-fast CBOR encoder/decoder with extensions for records and structural cloning - GitHub - kriszyp/cbor-x: Ultra-fast CBOR encoder/decoder with extensions for records and structural cloning

ashutosh•3/8/23, 6:39 AM

Anyone using DO for any production use-case as a replacement for Web-sockets? If yes, what is your monthly bills for the same on Cloudflare?

I ran some tests and within few tests the dashboard was showing 100s GB-s (something similar, not sure) but at that point I abandoned the idea.

My existing implementation use sockets via Pusher and I am on a $49/month plan.
I can check the pricing examples here: https://developers.cloudflare.com/workers/platform/pricing/#durable-objects-billing-examples but that does not contain any real examples so it would be great if someone here can help us figure out the estimated costs.

Aashutosh Anyone using DO for any production use-case as a replacement for Web-sockets? If...

Hello, I’m Allie!•3/8/23, 7:32 AM

Running a DO non-stop costs about $4 a month, and a single DO can handle multiple WS connections, so you could probably get pretty far there, depending on what you are doing

Hello, I’m Allie!•3/8/23, 7:32 AM

Also note, if you can fit it all in a single DO, then it qualifies for the “free” tier

LLarry I use https://github.com/kriszyp/cbor-x by the same author. CBOR is essentially ...

Jacob WrightOP•3/8/23, 2:48 PM

Thank you!

DanTheGoodman•3/8/23, 7:32 PM

Is there a quick way to purge all DOs for a service without deleting the service? Doing a wrangler delete && wrangler publishwrangler delete && wrangler publish doesn't actually seem to delete the objects.

DDanTheGoodman Is there a quick way to purge all DOs for a service without deleting the service...

zehawk•3/9/23, 8:27 AM

Search for "deleteall" - you'll get lots of previous discussions

Zzehawk Search for "deleteall" - you'll get lots of previous discussions

DanTheGoodman•3/9/23, 12:36 PM

Yeah unfortunately I have to keep track of all the DO instance IDs then somewhere which is annoying

DDanTheGoodman Yeah unfortunately I have to keep track of all the DO instance IDs then somewher...

Hello, I’m Allie!•3/9/23, 12:39 PM

You can use the API for that

HHello, I’m Allie!You can use the API for that

DanTheGoodman•3/9/23, 12:40 PM

oh right! forgot about that, thanks

DanTheGoodman•3/9/23, 12:40 PM

but I have to implement my own deleete right? looking at the docs I don't see a way to kill it

Aandheller Is there a way to know if/why a DO fails? I am merging Yjs updates and saving to...

Larry•3/9/23, 12:40 PM

Would a try/catch approach when you do an operation in a DO on a document work? Then you could retry triggering your different way.

DDanTheGoodman but I have to implement my own deleete right? looking at the docs I don't see a ...

Hello, I’m Allie!•3/9/23, 12:42 PM

Yes

DanTheGoodman•3/9/23, 12:42 PM

gotcha thanks!

DDanTheGoodman but I have to implement my own deleete right? looking at the docs I don't see a ...

Hello, I’m Allie!•3/9/23, 12:46 PM

Note too that just because you run deleteAll doesn’t mean that the DO itself is deleted. It(and its ID) will continue to exist(without any storage) for about a minute or two

HHello, I’m Allie!Note too that just because you run deleteAll doesn’t mean that the DO itself is ...

DanTheGoodman•3/9/23, 12:47 PM

thanks for the heads up!

johtso•3/9/23, 1:57 PM

Is there an easy way to achieve self destructive ephemeral DOs? I'd love to have my user sessions backed by a DO but have it destroyed if no requests are made for a certain period of time

Jjohtso Is there an easy way to achieve self destructive ephemeral DOs? I'd love to have...

DanTheGoodman•3/9/23, 1:58 PM

Set an alarm for some time after every request (say new Date().getTime() + 5000new Date().getTime() + 5000 for a 5 second delay) and in the alarm handler just call .deleteAll().deleteAll()

johtso•3/9/23, 1:59 PM

And I guess replace the previous alarm each time

DanTheGoodman•3/9/23, 1:59 PM

SetAlarm will upsert it

DanTheGoodman•3/9/23, 1:59 PM

so it will replace if it already exists, or set if it does not

johtso•3/9/23, 2:00 PM

Another question, what's the go to way to get a type safe interface between my DO and my worker? Something like trpc?

johtso•3/9/23, 2:17 PM

Does itty router help with this?

johtso•3/9/23, 2:40 PM

Does anyone know if itty-durable gives type safety?

Jjohtso Another question, what's the go to way to get a type safe interface between my D...

Larry•3/9/23, 2:43 PM

There might be a way to use native TypeScript types that I don't know of, but I use vanilla JavaScript so I've settled on @cfworker/json-schema

johtso•3/9/23, 2:56 PM

@Kevin W does itty-durable "just work" when it comes to types?

Jjohtso Another question, what's the go to way to get a type safe interface between my D...

DanTheGoodman•3/9/23, 3:23 PM

they are bound together, and can share types in the codebase

DanTheGoodman•3/9/23, 3:23 PM

Do you mean runtime-safe? It's better to validate inputs and assume they are as expected once they enter the system rather than checking everywhere

DanTheGoodman•3/9/23, 3:24 PM

If you need runtime type checking you can use something like zod

DanTheGoodman•3/9/23, 3:24 PM

Hono has better typing across the board, I've used both itty-router and hono. I prefer to stay away from itty-durable but that would have better typing. I stay away because I don't like how differnet it is from making normal DOs

johtso•3/9/23, 3:25 PM

@danthegoodman this looks promising https://www.npmjs.com/package/trpc-durable-objects

npm

trpc-durable-objects

Testing out trpc for durable objects. Latest version: 1.0.4, last published: 19 days ago. Start using trpc-durable-objects in your project by running npm i trpc-durable-objectsnpm i trpc-durable-objects. There are no other projects in the npm registry using trpc-durable-objects.

johtso•3/9/23, 3:25 PM

Just looking to reduce boilerplate

DanTheGoodman•3/9/23, 3:25 PM

Are you just trying to get strong type checking?

DanTheGoodman•3/9/23, 3:26 PM

just type check when requests come it, then you can assume they are safe once they are in the system

DO storage uses the structured clone algorithm. Do you know if there are any data protocol libraries

Similar Threads