Would D1 w/ Smart Placement’s latency be comparable to SQLite on DO’s latency? I’m going to be doing

Would D1 w/ Smart Placement’s latency be comparable to SQLite on DO’s latency?
I’m going to be doing a little test soon but I’m asking just to know what to expect with my results

Oo7 Would D1 w/ Smart Placement’s latency be comparable to SQLite on DO’s latency? I...

kenton•9/27/24, 6:20 PM

Smart Placement with D1 will reduce your latency down to a few milliseconds, but SQLite on DO has latency of a few microseconds. Still a big difference, but may or may not matter depending on how you use it.

Ashish Rawat•9/27/24, 6:40 PM

I want to ask how all DO will work together. For eg: Let's say I want to query across all DO on sqlite Because DO has 1 GB limit, I'm still not getting how to infra my app according to it, if I want to migrate from mongodb.

Ashish Rawat•9/27/24, 6:41 PM

There must be a fullstack tutorial for that with DO on sqlite3 with sharding

Original message was deleted

kenton•9/27/24, 7:25 PM

Yes, if you're on the sqlite storage backend, the KV API still works and is actually backed by a hidden table in SQLite, so you get all the benefits.

AAshish Rawat I want to ask how all DO will work together. For eg: Let's say I want to query a...

kenton•9/27/24, 7:27 PM

Generally you want to split your database into logical "shards" of related data, trying to make these shards as fine-grained as reasonably makes sense. Remember that it's totally fine to have millions of DOs -- the system is built for that.

kenton•9/27/24, 7:27 PM

one common strategy is to put each user's data in a separate DO. But the right split depends on the app.

shoo•9/27/24, 7:45 PM

Hey guys
I'm not sure if this is possible or if its a good idea

but because mongodb long connections are not possible for workers and the documentation suggests that we use mongodb realm.

is it possible to use durable objects to store the connection details and get that everytime a new worker is invoked?

Kkenton Yes, if you're on the sqlite storage backend, the KV API still works and is actu...

geg•9/27/24, 9:13 PM

Can I ask why is the table hidden?

Ggeg Can I ask why is the table hidden?

kenton•9/27/24, 9:17 PM

It contains V8-serialized values. We don't want people trying to decode the V8 serialization as it sometimes breaks in backwards-incompatible ways (which we can deal with as long as we control all the deserializers). We are also not completely sure that the parser is resilient against malicious input. Chrome similarly uses V8 serialization but never lets apps see nor modify the serialized bytes.

Kkenton It contains V8-serialized values. We don't want people trying to decode the V8 s...

geg•9/27/24, 9:23 PM

Thank you for the answer! I was particularly interested in the serialization/deserialization, that clears up why there's no functions for that exposed.

Wallacy•9/27/24, 11:48 PM

Still plans to put DOs on more regions? Sucks very hard here on South America. And i get the problem of the storage, but in many cases a DO whiout any storage backend will be just fine. For handle Websockets will be very helpfull, today i need to pay the latency to call a DO across the atlantic just make a websocket point. Sometimes i use a DO just to keep my in memory graphql cache, etc. The point is: Every point that runs a worker should be able to run a DO without storage. (more storage locations will be good also);

Vy•9/28/24, 12:21 AM

FYI we've identified an issue with alarms in the beta for SQLite in Durable Objects. While our team implements a fix, we are temporarily disabling alarms for only Durable Objects with SQLite storage backend.

Durable Objects with the default, key-value storage are unaffected and alarms are functioning normally

Pending docs:https://github.com/cloudflare/cloudflare-docs/pull/17185

GitHub

Disable alarms for SQLite in DO by vy-ton · Pull Request #17185 · c...

Disable alarms in SQLite in DO temporarily while a fix is implemented

Chaika•9/28/24, 12:29 AM

aw hopefully doesn't take too long for fix, was going to play around with alarms and sqlite

kchro3•9/28/24, 12:56 AM

hope it's ok to solicit feedback on a use-case here.

i'm interested in building a personalized email scheduler, and i want to see if DO would be a good architecture for this & if it would scale to millions of users. the exact use case is that i have a ranking algo where for a given user & set of emails, i'll predict which email to send, and now i want to predict the optimal send time too.

the way i've seen this work at prev companies has been to have a daily cron job that computes the optimal send time for every user & stores them in a database. then a separate cron job would run like hourly to fetch the users eligible to receive an email or notification and then apply the content ranking algorithm. however, that usually means that you'll need at least two cron jobs, a database, and a pubsub topic.

what i'd like to do is to use a durable object per user to represent their send time preferences. in an alarm routine, it would wake up => fetch & rank the user's eligible emails => send an email => set its next alarm. a separate worker would receive webhook events about when an email was sent / opened / clicked / etc, so it would update the durable object per user. if the email is opened, the worker could update the user's preferences & reschedule the alarm.

if this sounds like a bad idea (too overengineered, etc), would love to know before investing too much. i'd be looking at supporting millions of users & emails per week

Kkenton Generally you want to split your database into logical "shards" of related data,...

Ashish Rawat•9/28/24, 6:23 AM

Is there any way to create durable objects bindings on the fly with api or sdk for each users or maybe I'm missing something. A basic tutorial on scalibility would be appreciated if possible.

AAshish Rawat Is there any way to create durable objects bindings on the fly with api or sdk f...

Ashley•9/28/24, 9:15 AM

See https://developers.cloudflare.com/durable-objects/best-practices/access-durable-objects-from-a-worker/, you would have a binding for a given DO (implemented by a class in your code) and then create DOs on the fly using whatever mechanism works for you (e.g. create a DO using a userId with idFromName). So the DO creation is dynamic, it’s not like other bindings where you create a resource and then you interact with one resource with an ID, with DOs you bind to a class + then generate millions of resources (if you need)

John Spurlock•9/29/24, 2:47 PM

important metrics for sqlite-backed DOs already in graphql, thanks!

JVictorV•9/29/24, 4:52 PM

Does anyone have a good way to uniquely identify websocket connections? I have a simple map of id -> websocket that i want to keep in sync even when the ws hibernates or the DO is evicted from memory

Matt Silverlock•9/29/24, 5:13 PM

Set tags when you accept the WebSockets: https://developers.cloudflare.com/durable-objects/api/websockets/#acceptwebsocket

Cloudflare Docs

WebSockets | Cloudflare Durable Objects docs

WebSockets are long-lived TCP connections that enable bi-directional, real-time communication between client and server.

JJikyu Would it be possible to use drizzle / run drizzle migrations against the new DO ...

zwily•9/29/24, 9:17 PM

same question here! drizzle may need to make another adapter for it

1.1.1.1APP•9/29/24, 10:01 PM

DNS over Discord: Invite

Invite DNS over Discord to your server with https://dns-over-discord.v4.wtf/invite

Or, use the bot anywhere by adding it to your account with https://dns-over-discord.v4.wtf/invite/user

Nick•9/30/24, 6:07 AM

Is there any way to get notified when a new DO is created or updated? I guess you can build it yourself with Alarms, but wondering if something built in

NNick Is there any way to get notified when a new DO is created or updated? I guess yo...

Hello, I’m Allie!•9/30/24, 7:27 AM

There is not

HHello, I’m Allie!There is not

Nick•9/30/24, 7:28 AM

cool that's what i thought, thanks

Nick•9/30/24, 7:30 AM

sorry, another question, does the new workers logs stuff work on Durable Object logs?

NNick sorry, another question, does the new workers logs stuff work on Durable Object ...

Hello, I’m Allie!•9/30/24, 7:31 AM

Not sure, I would try it and see?

Nick•9/30/24, 7:32 AM

alright, i'll give it a bash at the weekend

Nick•9/30/24, 7:45 AM

actually just did it now. It seeeems to be coming through.

Nick•9/30/24, 7:45 AM

Hello was said comes from the DO

Yota•9/30/24, 8:13 AM

Even though I performed a single INSERT operation, 27 Storage Operations were counted. Why is that? How is this calculated?

Yota•9/30/24, 8:29 AM

no, but there are only 5 records

Yota•9/30/24, 8:30 AM

will try index

Yota•9/30/24, 8:38 AM

I tried it, but I couldn't observe any difference

Yota•9/30/24, 9:24 AM

Okay. Thank you for the clarification

YYota Even though I performed a single INSERT operation, 27 Storage Operations were co...

Yota•9/30/24, 9:26 AM

So the original issue still isn’t resolved…

YYota Even though I performed a single INSERT operation, 27 Storage Operations were co...

lambrospetrou•9/30/24, 1:25 PM

What was the INSERT doing? You are billed for "rows" read/written by your query, not operations. Assuming you are referring to the SQLite DO.

Llambrospetrou What was the INSERT doing? You are billed for "rows" read/written by your query,...

Yota•9/30/24, 1:43 PM

this.

Yota•9/30/24, 1:43 PM

also, only this consume 26 read operation.

Yota•9/30/24, 1:44 PM

I tested several times in different condition and figured out that if the DO have multiple tables(8 tables in total) the number increased.
If i removed other tables, the read count reduced but it still count 6 read.

Noah•9/30/24, 3:24 PM

Hey, I can't make a new durable object :

Cannot create binding for class 'Trainer' because it is not currently configured to implement durable objects. Did you forget to apply a --new-class migration to it? [code: 10061]

Cannot create binding for class 'Trainer' because it is not currently configured to implement durable objects. Did you forget to apply a --new-class migration to it? [code: 10061]

But if I try to add it

npx wrangler deploy --new-class Trainer
✘ [ERROR] Unknown arguments: new-class, newClass

npx wrangler deploy --new-class Trainer
✘ [ERROR] Unknown arguments: new-class, newClass

In my wrangler I have

[[durable_objects.bindings]]
name = "DURABLE_TRAINER"
class_name = "Trainer"
[[migrations]]
tag = "v1"
new_classes = ["Trainer"]

[[durable_objects.bindings]]
name = "DURABLE_TRAINER"
class_name = "Trainer"
[[migrations]]
tag = "v1"
new_classes = ["Trainer"]

I already did it once and then I deleted my durable object via dashboard, how to remake one ?

NNoah Hey, I can't make a new durable object : ``` Cannot create binding for class 'T...

Hello, I’m Allie!•9/30/24, 3:27 PM

Try just running

wrangler deploy

wrangler deploy

Noah•9/30/24, 3:28 PM

Yeah that's what gives me

  Cannot create binding for class 'Trainer' because it is not currently configured to implement
  durable objects. Did you forget to apply a --new-class migration to it? [code: 10061]

  Cannot create binding for class 'Trainer' because it is not currently configured to implement
  durable objects. Did you forget to apply a --new-class migration to it? [code: 10061]

Noah•9/30/24, 3:28 PM

and I tried with different wrangler version, currently i'm on 3.78.12

NNoah Yeah that's what gives me ``` Cannot create binding for class 'Trainer' becaus...

cloudkite•9/30/24, 5:05 PM

Use a different tag, for example tag = “v1.1”

Noah•9/30/24, 6:19 PM

Well I finally resolved the issue by deleting the existing worker, made a new worker with different name, then migrate my page on the new worker

Noah•9/30/24, 6:20 PM

But ty! I'll try

Original message was deleted

Yota•9/30/24, 6:22 PM

I tired: durable-objects but the number was not changed.

YYota also, only this consume 26 read operation.

lambrospetrou•9/30/24, 7:29 PM

Hey, I suggest taking a look at the examples in https://developers.cloudflare.com/durable-objects/best-practices/access-durable-objects-storage/#examples

Aside from your billing question about the rows, you seem to be using promises when the API is synchronous, and also one()one() does not return null, it will throw when it doesn't find a result.

YYota I tested several times in different condition and figured out that if the DO hav...

lambrospetrou•9/30/24, 7:30 PM

To clarify something again, the

rowsRead

rowsRead

you get is not the number of results. It's the number of rows read by SQLite to satisfy your query. So, if you do joins, scans, etc, and only end up with 1 row, you are not just charged for 1 row, but all the rows that had to be read to answer your query.
Having correct and good indices on your tables to answer your queries without scanning the whole table is important.

Would D1 w/ Smart Placement’s latency be comparable to SQLite on DO’s latency? I’m going to be doing

Similar Threads

Similar Threads

Similar Threads