Cloudflare Developers•12mo ago

so your approach is to have a db lookup first then to invoke a DO?

James•1/25/25, 5:47 PM

The db is just a set of calls in the DO. You have to be in the DO to access it.

Aalgads When you say after the first hop, its cached in the local datacenter, is it cach...

lambrospetrou•1/26/25, 12:36 PM

There are too many variables in what you describe. How you instantiate DOs, how long that takes, is it serially or paralle, and many other things.
In general, any work you can parallelize from your worker the better. You should not pre-optimize DO stub creation until you actually measure it and see that it's the bottleneck in your code. Performance optimizations just based on intuition doesn't work. Measure and then optimize accordingly.
Start with simple code doing what you need and take it from there.

森森@VerseLink (RW)so your approach is to have a db lookup first then to invoke a DO?

lambrospetrou•1/26/25, 12:39 PM

My mental model for Durable Objects constructor is different than most.
As the step 3 in https://developers.cloudflare.com/reference-architecture/diagrams/storage/durable-object-control-data-plane-pattern/ describes, I prefer doing the expensive DO initialization in a dedicated

init()

init()

call that is part of a resource creation flow. Including any initial SQL table creation and setup. After the init() completes, any other SQL migration is done only when necessary (at the point of access, not the constructor, and it's very fast/noop majority of the time when done properly: https://github.com/lambrospetrou/durable-utils?tab=readme-ov-file#sqlite-schema-migrations ).

I don't consider the constructor to be the place to do expensive initialization. The DO constructor as of today should be fast, and I only use it to initialize the "in-memory state of the DO". Loading anything from the storage into memory if necessary or just not do anything expensive.
It's different from how most folks think about the DO constructor, but in practice this works much better, especially if your DOs are not receiving enough traffic to stay warm and you have many cold starts, so you need to keep your constructor lightweight.

Llambrospetrou My mental model for Durable Objects constructor is different than most. As the s...

森

森@VerseLink (RW)OP•1/26/25, 2:26 PM

Thanks, yeh, I'm using Drizzle to manage migration of our database structure (just that, still write raw SQL most of the time), and the migration code it provided was less ideal so I had to incorporate your suggestion into it.

I guess premature optimization is evil, but I'm still curious about how I can approach DO init a bit better. I sort of understood how DO creates objects that were previously created (say via idFromName), but I'm still uncertain how DO creates when it's a new name id that it had never seen before. Does it ask every data center if that name id has existed before creating the object? or does it create it first anyways, and only sync at storage level?

If it was the latter, if two people decides to create the same name id DO at the same time on the opposite side of the earth, they would not see the same in memory status as the other? (or is there more memory syncing magic involved)

森森@VerseLink (RW)Thanks, yeh, I'm using Drizzle to manage migration of our database structure (ju...

lambrospetrou•1/27/25, 10:37 AM

Does it ask every data center if that name id has existed before creating the object? or does it create it first anyways, and only sync at storage level?

This is my personal suggestion, but you shouldn't design your applications based on internal implementation.
If DOs internally ask every datacenter, or some of them, or use a central database, should not impact your designs. The end result for you and your application is how long would this process take. So, do that, measure it, and only if it's slow for your use-case start thinking about optimizations. Internal implementations can and will change over time.

If it was the latter, if two people decides to create the same name id DO at the same time on the opposite side of the earth, they would not see the same in memory status as the other? (or is there more memory syncing magic involved)

Read https://developers.cloudflare.com/durable-objects/platform/known-issues/
The uniqueness guarantee is strictly enforced when you access storage.

lemmih•1/27/25, 10:39 AM

Can I emit log messages from a Durable Object?

geg•1/28/25, 8:11 AM

Hi, can it be that runtime updates break hibernated websockets? My DO can still send messages out but incoming messages stop arriving at random times. I noticed that it doesn't happen over the weekends and as soon as the weekend is over it starts happening almost every day, sometimes every couple of hours on a given day. My DO code couldn't be simpler, message in -> message out. I strongly believe it's runtime updates at this point. Reconnecting the "writing" websocket fixes it.

Ggeg Hi, can it be that runtime updates break hibernated websockets? My DO can still ...

lambrospetrou•1/28/25, 8:26 AM

Not sure about this specific issue, but you should be doing reconnection anyway, since servers can go down for many reasons, therefore your clients should attempt to reconnect (with usual jitter+backoff) if the connection fails.

Llambrospetrou Not sure about this specific issue, but you should be doing reconnection anyway,...

geg•1/28/25, 8:29 AM

What I meant with "messages stop arriving" is that there is a connection with no errors, yet messages don't arrive.

geg•1/28/25, 8:31 AM

When this issue happens to me, reconnecting doesn't work instantly also, the websocket connects fine but messages don't arrive still. I have to give it like 10 seconds before reconnecting, then magically it starts working again..

geg•1/28/25, 8:35 AM

When I deploy new code, the websocket disconnects and my "writer/sender" gets a close frame and reconnects fine.

nejc•1/28/25, 8:46 AM

hi all, one thing that I always prevent me from using DO is observability. Each object is isolated via ID, so can I get a whole picutre of what is happening. Let's say that I build a chat app which uses DO and each session ID is it's own DO ID. Now I would like to get real time overview of how many sessions I have and how many users are active. Is there an easy way to do this inside DO, or do I need D1 / Analytics engine in parallel, where DO pushes this data there?

iLearn•1/28/25, 9:25 AM

Pulling this Q out of thread (thread : Creating Replica for a DO), for kind attention and response :

Use-Case :
I have DO per user, accessed through unique_id (= email of user).
Now, a user requests to change email.
Hence, need to "replicate" complete DO(old_email) to DO(new_email)

My Q :
Q#1 : Is my request to "replicate" feasible ?
Q#2 : If yes, how to achieve this ? (a sample code will help)

Thanks.

IiLearn Pulling this Q out of thread (thread : Creating Replica for a DO), for kind atte...

lambrospetrou•1/28/25, 10:14 AM

There is no built in method now to duplicate the storage of a DO to a different one. You need to do the copy of the data in your code. Reading rows/KV items and writing them in the new one.

In my opinion though, if users are meant to be able to change their emails in your application, then those emails are not a good key to use for your DOs. The DO keys should be immutable identifiers for the data hosted in that DO. Otherwise it will become very expensive and slow to duplicate DOs all the time, and unless you block access to them while that's done you will also have consistency issues.

Ggeg What I meant with "messages stop arriving" is that there is a connection with no...

1984 Ford Laser•1/28/25, 12:17 PM

Are you broadcasting the message to all connected clients, or only the one that sent the message to the server?

1984 Ford Laser•1/28/25, 12:23 PM

If you're sending it directly, I'd be curious to see if the

ws

ws

can be found in the

this.state.getWebSockets()

this.state.getWebSockets()

set

11984 Ford Laser Are you broadcasting the message to all connected clients, or only the one that ...

geg•1/28/25, 12:25 PM

webSocketMessage(source: WebSocket, data: string) {
  for (let ws of this.ctx.getWebSockets()) {
    if (ws == source) {
      continue
    }
    
    ws.send(data)
  }
}

webSocketMessage(source: WebSocket, data: string) {
  for (let ws of this.ctx.getWebSockets()) {
    if (ws == source) {
      continue
    }
    
    ws.send(data)
  }
}

Just like this

Llambrospetrou Of course! https://developers.cloudflare.com/durable-objects/api/storage-api/#tr...

Burrito•1/28/25, 12:35 PM

So if

transaction

transaction

can be used over

transactionSync

transactionSync

even in SQLite DO, does that mean the purpose of

transactionSync

transactionSync

is merely to provide a sync API?

Burrito•1/28/25, 12:38 PM

Cool.

Burrito•1/28/25, 12:40 PM

It'd be nice if the KV API also had sync equivalents in SQLite DO.

Llambrospetrou There is no built in method now to duplicate the storage of a DO to a different ...

iLearn•1/28/25, 4:12 PM

Thanks for the detailed explanation.

Ggeg ```typescript webSocketMessage(source: WebSocket, data: string) { for (let ws ...

1984 Ford Laser•1/28/25, 8:38 PM

Yeah okay that disproves my theory. I was wondering if the client had closed the connection unexpectedly after sending a message, meaning it wouldn't be in

getWebSockets()

getWebSockets()

, but your code proves it always would be for the server to respond. Interesting issue

iLearn•1/28/25, 10:14 PM

QQ : Can "Storage per Durable Object" for "key-value storage backend" store 200GB of data ? .. Limits section says "Unlimited", but still want this to be confirmed .

peachneo•1/28/25, 11:19 PM

What happens in terms of retries if you have an alarm that is triggered and that alarm handler also sets an alarm, folloowed by a failure:

eg:

async alarm() {
  await setAlarm(someTimestampFarIntoTheFuture)
  await somethingThatWillFail()
}

async alarm() {
  await setAlarm(someTimestampFarIntoTheFuture)
  await somethingThatWillFail()
}

do the retries get blown away by the new alarm time?

codefrau•1/28/25, 11:22 PM

It looks like the

durable_objects.namespaces.objects

durable_objects.namespaces.objects

API always returns

hasStoredData: false

hasStoredData: false

for SQL-backed DOs, even if they do have data. Is this intentional or a bug?

Ppeachneo What happens in terms of retries if you have an alarm that is triggered and that...

森

森@VerseLink (RW)OP•1/28/25, 11:28 PM

no i don't think so, you can image that there's a try catch call around

alarm()

alarm()

, so whatever happens within the

alarm()

alarm()

doesn't really affect how the original

alarm()

alarm()

is invoked

森森@VerseLink (RW)no i don't think so, you can image that there's a try catch call around `alarm()...

peachneo•1/28/25, 11:37 PM

Ok thanks, so effectively the alarm would run for N retries + the additional alarms set in the handler I guess?

Ppeachneo Ok thanks, so effectively the alarm would run for N retries + the additional ala...

lucidplot•1/28/25, 11:39 PM

no you can only have 1 alarm time set. If your alarm handler throws, it will retry as if you haven’t made any changes to stored state (ie forgetting any storage calls, including setAlarm)

Llucidplot no you can only have 1 alarm time set. If your alarm handler throws, it will ret...

peachneo•1/28/25, 11:39 PM

Oh ok.. that makes sense

Will•1/29/25, 2:32 AM

Does anyone have a resource on testing Durable object methods? I am using DO SqlStorage in my DO and hono to manage my worker.

I would be happy to just test the DO methods atm

IiLearn QQ : Can "Storage per Durable Object" for "key-value storage backend" store 200G...

1984 Ford Laser•1/29/25, 3:36 AM

This is the limit for total account storage (across all instances of all DOs) with Durable Objects, but you can contact CF to ask for a raise in that limit

11984 Ford Laser This is the limit for total account storage (across all instances of all DOs) wi...

iLearn•1/29/25, 6:29 AM

Thanks.
Can i have same luxury of Workers KV in terms of Unlimited Storage ? OR i need to regularly keep tap on storage of DO-KV and dropping mail to increase ?

WWill Does anyone have a resource on testing Durable object methods? I am using DO Sql...

lambrospetrou•1/29/25, 10:40 AM

You can use the vitest integration and test at the DO stub level directly: https://github.com/cloudflare/workers-sdk/blob/main/fixtures/vitest-pool-workers-examples/durable-objects/test/direct-access.test.ts

Llambrospetrou You can use the vitest integration and test at the DO stub level directly: https...

Will•1/30/25, 2:03 AM

Thank you!

Fernando Barroso•1/30/25, 2:26 AM

Hello all! Do you know when the SQLite storage will become GA? Or when the storage limit will be increased from 1GB? Will be a big decision point for us to use it in prod given that we do expect to have >1GB of data in our DOs

FFernando Barroso Hello all! Do you know when the SQLite storage will become GA? Or when the stora...

森

森@VerseLink (RW)OP•1/30/25, 4:27 AM

i think if you ask for it they'll provide you regardless of GA, just GA garuantees everyone has 10GB to work with + 50GB all

Bbrett Our vision is that you don’t need a traditional DB in addition to DOs, but we do...

Eric•1/30/25, 5:26 AM

Hey, any update on this? As in, is there any data viewer for the SQL backend? Unfortunately this is going to be a critical business decision as to whether we use DO's or not

1984 Ford Laser•1/30/25, 6:41 AM

Is something broken with DOs? I have made requests to multiple DO classes today and haven't incurred any duration or storage operation charges according to the dashboard, where normally I would be incurring plenty of both for what I was doing

1984 Ford Laser•1/30/25, 6:42 AM

Last week vs last 24 hours

EEric Hey, any update on this? As in, is there any data viewer for the SQL backend? Un...

lambrospetrou•1/30/25, 10:24 AM

Not built-in yet. There are several open source libraries adding a simple HTTP API allowing you to get some stats or execute SQL on-demand, though.

Llambrospetrou Not built-in yet. There are several open source libraries adding a simple HTTP A...

scruffy•1/30/25, 10:41 AM

Do you have any names/links for examples? Have tried a few simple searches, but all results come back as Cloudflare docs

Sscruffy Do you have any names/links for examples? Have tried a few simple searches, but ...

lambrospetrou•1/30/25, 11:43 AM

StarbaseDB was mentioned a few times in the channel, although I never used it, so cannot vouch for it. Others (including myself) just add an admin HTTP endpoint to expose some stats and functionality from DOs. That should be easy in most cases (make sure to add a simple secret auth check).

Llambrospetrou StarbaseDB was mentioned a few times in the channel, although I never used it, s...

scruffy•1/30/25, 11:44 AM

excellent, thanks for the tips

mihaaai•1/30/25, 12:11 PM

Is there any outgoing Websocket Message Size limit? Only the incoming (from clients to the DO server is mentioned in the docs) but I'd need to also know if there are limits for outgoing messages, either targeted sent or broadcast from the DO server to clients.

Sscruffy excellent, thanks for the tips

Brayden•1/30/25, 1:52 PM

It's worth mentioning too that if you do use StarbaseDB (which is a DO with some more code on top of it - https://github.com/outerbase/starbasedb) you can connect it to either Outerbase (www.outerbase.com) for a visual interface, or the StarbaseDB instance you deploy itself comes with a built in GUI if you append

/studio

/studio

to the deployed instance URL.

森森@VerseLink (RW)i think if you ask for it they'll provide you regardless of GA, just GA garuante...

Fernando Barroso•1/30/25, 2:13 PM

is this just by submitting a support ticket? Or how is this requested?

FFernando Barroso is this just by submitting a support ticket? Or how is this requested?

boop•1/30/25, 6:10 PM

https://docs.google.com/forms/d/e/1FAIpQLSd_fwAVOboH9SlutMonzbhCxuuuOmiU1L_I5O2CFbXf_XXMRg/viewform

boop•1/30/25, 6:10 PM

This looks super sketchy lol, it's from here: https://developers.cloudflare.com/pages/platform/limits/

so your approach is to have a db lookup first then to invoke a DO?

Similar Threads

Similar Threads

Similar Threads