QQ : Can "Storage per Durable Object" for "key-value storage backend" store 200GB of data ? .. Limit

QQ : Can "Storage per Durable Object" for "key-value storage backend" store 200GB of data ? .. Limits section says "Unlimited", but still want this to be confirmed .

peachneo•1/28/25, 11:19 PM

What happens in terms of retries if you have an alarm that is triggered and that alarm handler also sets an alarm, folloowed by a failure:

eg:

async alarm() {
  await setAlarm(someTimestampFarIntoTheFuture)
  await somethingThatWillFail()
}

async alarm() {
  await setAlarm(someTimestampFarIntoTheFuture)
  await somethingThatWillFail()
}

do the retries get blown away by the new alarm time?

codefrau•1/28/25, 11:22 PM

It looks like the durable_objects.namespaces.objectsdurable_objects.namespaces.objects API always returns hasStoredData: falsehasStoredData: false for SQL-backed DOs, even if they do have data. Is this intentional or a bug?

Ppeachneo What happens in terms of retries if you have an alarm that is triggered and that...

森

森@VerseLink (RW)•1/28/25, 11:28 PM

no i don't think so, you can image that there's a try catch call around alarm()alarm(), so whatever happens within the alarm()alarm() doesn't really affect how the original alarm()alarm() is invoked

森森@VerseLink (RW)no i don't think so, you can image that there's a try catch call around `alarm()...

peachneo•1/28/25, 11:37 PM

Ok thanks, so effectively the alarm would run for N retries + the additional alarms set in the handler I guess?

Ppeachneo Ok thanks, so effectively the alarm would run for N retries + the additional ala...

lucidplot•1/28/25, 11:39 PM

no you can only have 1 alarm time set. If your alarm handler throws, it will retry as if you haven’t made any changes to stored state (ie forgetting any storage calls, including setAlarm)

Llucidplot no you can only have 1 alarm time set. If your alarm handler throws, it will ret...

peachneo•1/28/25, 11:39 PM

Oh ok.. that makes sense

Will•1/29/25, 2:32 AM

Does anyone have a resource on testing Durable object methods? I am using DO SqlStorage in my DO and hono to manage my worker.

I would be happy to just test the DO methods atm

IiLearn QQ : Can "Storage per Durable Object" for "key-value storage backend" store 200G...

1984 Ford Laser•1/29/25, 3:36 AM

This is the limit for total account storage (across all instances of all DOs) with Durable Objects, but you can contact CF to ask for a raise in that limit

11984 Ford Laser This is the limit for total account storage (across all instances of all DOs) wi...

iLearnOP•1/29/25, 6:29 AM

Thanks.
Can i have same luxury of Workers KV in terms of Unlimited Storage ? OR i need to regularly keep tap on storage of DO-KV and dropping mail to increase ?

WWill Does anyone have a resource on testing Durable object methods? I am using DO Sql...

lambrospetrou•1/29/25, 10:40 AM

You can use the vitest integration and test at the DO stub level directly: https://github.com/cloudflare/workers-sdk/blob/main/fixtures/vitest-pool-workers-examples/durable-objects/test/direct-access.test.ts

Llambrospetrou You can use the vitest integration and test at the DO stub level directly: https...

Will•1/30/25, 2:03 AM

Thank you!

Fernando Barroso•1/30/25, 2:26 AM

Hello all! Do you know when the SQLite storage will become GA? Or when the storage limit will be increased from 1GB? Will be a big decision point for us to use it in prod given that we do expect to have >1GB of data in our DOs

FFernando Barroso Hello all! Do you know when the SQLite storage will become GA? Or when the stora...

森

森@VerseLink (RW)•1/30/25, 4:27 AM

i think if you ask for it they'll provide you regardless of GA, just GA garuantees everyone has 10GB to work with + 50GB all

Bbrett Our vision is that you don’t need a traditional DB in addition to DOs, but we do...

Eric•1/30/25, 5:26 AM

Hey, any update on this? As in, is there any data viewer for the SQL backend? Unfortunately this is going to be a critical business decision as to whether we use DO's or not

1984 Ford Laser•1/30/25, 6:41 AM

Is something broken with DOs? I have made requests to multiple DO classes today and haven't incurred any duration or storage operation charges according to the dashboard, where normally I would be incurring plenty of both for what I was doing

1984 Ford Laser•1/30/25, 6:42 AM

Last week vs last 24 hours

EEric Hey, any update on this? As in, is there any data viewer for the SQL backend? Un...

lambrospetrou•1/30/25, 10:24 AM

Not built-in yet. There are several open source libraries adding a simple HTTP API allowing you to get some stats or execute SQL on-demand, though.

Llambrospetrou Not built-in yet. There are several open source libraries adding a simple HTTP A...

scruffy•1/30/25, 10:41 AM

Do you have any names/links for examples? Have tried a few simple searches, but all results come back as Cloudflare docs

Sscruffy Do you have any names/links for examples? Have tried a few simple searches, but ...

lambrospetrou•1/30/25, 11:43 AM

StarbaseDB was mentioned a few times in the channel, although I never used it, so cannot vouch for it. Others (including myself) just add an admin HTTP endpoint to expose some stats and functionality from DOs. That should be easy in most cases (make sure to add a simple secret auth check).

Llambrospetrou StarbaseDB was mentioned a few times in the channel, although I never used it, s...

scruffy•1/30/25, 11:44 AM

excellent, thanks for the tips

mihaaai•1/30/25, 12:11 PM

Is there any outgoing Websocket Message Size limit? Only the incoming (from clients to the DO server is mentioned in the docs) but I'd need to also know if there are limits for outgoing messages, either targeted sent or broadcast from the DO server to clients.

Sscruffy excellent, thanks for the tips

Brayden•1/30/25, 1:52 PM

It's worth mentioning too that if you do use StarbaseDB (which is a DO with some more code on top of it - https://github.com/outerbase/starbasedb) you can connect it to either Outerbase (www.outerbase.com) for a visual interface, or the StarbaseDB instance you deploy itself comes with a built in GUI if you append

/studio

/studio

to the deployed instance URL.

森森@VerseLink (RW)i think if you ask for it they'll provide you regardless of GA, just GA garuante...

Fernando Barroso•1/30/25, 2:13 PM

is this just by submitting a support ticket? Or how is this requested?

FFernando Barroso is this just by submitting a support ticket? Or how is this requested?

boop•1/30/25, 6:10 PM

https://docs.google.com/forms/d/e/1FAIpQLSd_fwAVOboH9SlutMonzbhCxuuuOmiU1L_I5O2CFbXf_XXMRg/viewform

boop•1/30/25, 6:10 PM

This looks super sketchy lol, it's from here: https://developers.cloudflare.com/pages/platform/limits/

boop•1/30/25, 6:11 PM

Mmihaaai Is there any outgoing Websocket Message Size limit? Only the incoming (from clie...

Milan•1/30/25, 7:18 PM

Not that I can remember. I think we just build multiple WebSocket message frames for you under the hood, so as long as your peer that is receiving can stitch the frames together you should be fine.

BBrayden It's worth mentioning too that if you do use StarbaseDB (which is a DO with some...

scruffy•1/31/25, 7:25 AM

Cool, thanks! Do the UIs in either StarbaseDB or Outerbase show the sqlite-backed KV data table?

lambrospetrou•1/31/25, 11:16 AM

Hey folks, we just launched a new page explaining

What are Durable Objects

What are Durable Objects

: https://developers.cloudflare.com/durable-objects/what-are-durable-objects/

Let us know what you think, and if it's helping you understand DOs better, especially for newcomers to the product.

Cloudflare Docs

What are Durable Objects? · Cloudflare Durable Objects docs

A Durable Object is a special kind of Cloudflare Worker which uniquely combines compute with storage. Like a Worker, a Durable Object is automatically provisioned geographically close to where it is first requested, starts up quickly when needed, and shuts down when idle. You can have millions of them around the world. However, unlike regular Wo...

Llambrospetrou Hey folks, we just launched a new page explaining `What are Durable Objects`: ht...

lucidplot•1/31/25, 11:23 AM

looks great (although I am already familiar with DOs). typo near the end “These blog posts may also server as inspiration” (think you mean “serve”)

Llucidplot looks great (although I am already familiar with DOs). typo near the end “These ...

lambrospetrou•1/31/25, 11:28 AM

Oh thanks

rehan678•1/31/25, 11:29 AM

@lambrospetrou

Sscruffy Cool, thanks! Do the UIs in either StarbaseDB or Outerbase show the sqlite-backe...

Brayden•1/31/25, 12:47 PM

Unfortunately they only show what’s in the SQLite storage at the moment.

BBrayden Unfortunately they only show what’s in the SQLite storage at the moment.

scruffy•1/31/25, 12:49 PM

Had a hunch that might be the case... is it something the team's looking at or is it something that's impossible (or somewhere in between)?

Sscruffy Had a hunch that might be the case... is it something the team's looking at or i...

lambrospetrou•1/31/25, 1:31 PM

It's not impossible, since you can list all your KV items too if needed. https://developers.cloudflare.com/durable-objects/api/storage-api/#list

森

森@VerseLink (RW)•1/31/25, 5:34 PM

@lambrospetrou When calling transaction (asynchronous version), the DO deadlocks itself via blockConcurrencyWhile if the callback include a remote RPC stub that calls itself.

For example a function called “WriteData”, and an worker wants to coordinate two DOs and ensure both write the value at the same time, so both DO exposes a transaction method to allow this to happen. Result is that the DO deadlocks itself when the WriteData RPC stub is invoked inside the transaction.

I understand why this is the result, but I feel like if the caller initiated a transaction, they should also be able to do I/O too (and intuitively it should work this way too). Would you consider this a bug, or do you suggest a better way to approach this?

Best workaround would for the first DO to enter a transaction itself and call second DO to write and also enter a transaction itself, but this could require DO to expose too many functions that is only used for these transaction calls, and certainly more hard to catch bug (if you accidentally call the wrong function for example)

森森@VerseLink (RW)@lambrospetrou When calling transaction (asynchronous version), the DO deadlocks...

lambrospetrou•1/31/25, 5:42 PM

General guideline when you post, don't tag individuals please.

What you describe is the definition deadlock, so it makes sense that it deadblocks.

blockConcurrencyWhile

blockConcurrencyWhile

is like a lock, nothing can get into your DO while that runs.

If you want to implement distributed transactions you need something like 2-phase commit, without blocking the DO itself (don't abuse use of

blockConcurrencyWhile

blockConcurrencyWhile

). Store something in your state/storage that you are doing a transaction and reject anything if that's true.

I am working on a different transaction approach, for a personal project, but it's not ready for public yet, maybe in the future that could help you as well. But for now, distributed transactions need the standard 2-phase approach (or other techniques used by traditional systems).

In general, I don't like

blockConcurrencyWhile

blockConcurrencyWhile

personally and try to avoid it.

Llambrospetrou General guideline when you post, don't tag individuals please. What you describ...

森

森@VerseLink (RW)•1/31/25, 5:51 PM

Oh sorry I’ll refrain from pinging.

With 2 phase commit approach, how do you rollback if one transaction fails on one DO, but success on another?

森森@VerseLink (RW)Oh sorry I’ll refrain from pinging. With 2 phase commit approach, how do you ro...

lambrospetrou•1/31/25, 5:54 PM

For 2PC to work there is one coordinator that controls that: https://en.wikipedia.org/wiki/Two-phase_commit_protocol#Commit_(or_completion)_phase

Eitherway, unless absolutely unavoidable, I suggest not doing distributed transactions unless you really get into these protocols. It will just slow down everything A LOT, and worst case lock you down.

If you need cross-DO coordination often, then maybe you need a different architectural design for your application.

Llambrospetrou For 2PC to work there is one coordinator that controls that: https://en.wikipedi...

森

森@VerseLink (RW)•1/31/25, 5:59 PM

Yeah…. I imagined this is what would need to be done, and I agree with not doing a lot of distributed transactions (but sometimes it could be unavoidable), I guess I was hoping for some alternative approach that I’m didn’t see.

Btw, transaction on KV api still invokes block concurrency through storage.transaction if I read the source code correctly right? Seems painful when the only async part is the KV storage api.

Isaac McFadyen•1/31/25, 10:28 PM

Asking here because it's probably the most related, do RPCs share subrequests with the caller?

Isaac McFadyen•1/31/25, 10:29 PM

As in, I'm wondering about both the 6 concurrent request limit and the 1000 request total limit.

geg•1/31/25, 11:37 PM

Can it be that hibernated websockets just stop working if there's no activity for a very long time?

geg•1/31/25, 11:38 PM

I keep running in to issues with people where the DO doesn't do anything even though websockets are connected...

Ggeg Can it be that hibernated websockets just stop working if there's no activity fo...

Milan•2/1/25, 12:10 AM

Do you have evidence of this happening? If so, can you give us code that reliably reproduces this behavior so we can verify it for ourselves?

MMilan Do you have evidence of this happening? If so, can you give us code that reliabl...

geg•2/1/25, 12:28 AM

Will let you know!

Ggeg I keep running in to issues with people where the DO doesn't do anything even th...

1984 Ford Laser•2/1/25, 5:23 AM

Are you sending directly to a single client, or multiple at once from server to client/s?

Gagan Suie•2/1/25, 5:45 AM

We have a userSession DO that handles websockets and chatRoom DO that handles DoSqlite. Is using fetch the only way to have them communicate? Wouldn't that cause micro letency every fetch?

1984 Ford Laser•2/1/25, 5:57 AM

Nah RPC will be your best bet for worker to worker/DO to DO invocations. The docs can be a bit lacking in describing this step, especially because the implementation can depend on how many calls you're making, which would make reusing your stubs more necessary (WorkerEntrypoint class vs RpcTarget class)

QQ : Can "Storage per Durable Object" for "key-value storage backend" store 200GB of data ? .. Limit

Similar Threads

Similar Threads

Similar Threads