Cloudflare Developers•7mo ago

TIL about the SQL storage of Durable Objects... thanks!

Ddavidbarratt TIL about the SQL storage of Durable Objects... thanks!

1984 Ford Laser•6/7/25, 1:02 AM

You'll probably need a bit more boilerplate than with D1 but its a far better solution overall

Brayden•6/7/25, 1:23 AM

https://github.com/outerbase/starbasedb <- A project I made before joining Cloudflare but helps with the boilerplate on top of DO's for a D1 like experience.

BBrayden https://github.com/outerbase/starbasedb <- A project I made before joining Cloud...

dsds10•6/7/25, 8:44 AM

hmm that's interesting. somehow I had the "sqlite in do" stored as "ephemeral" in my mind, but checking the docs that's not the case. I also read that the max size in DO was 1GB, but this doesn't seem the case anymore (or never was)

what are the pro/cons of using sqlite in DO vs using D1?

bkyerv•6/7/25, 9:01 AM

why does wrangler d1 create [name] not work with oauth token (results in auth error) and requires api token? is it by design? or is it a known bug?

Ddsds10 hmm that's interesting. somehow I had the "sqlite in do" stored as "ephemeral" i...

Brayden•6/7/25, 11:04 AM

D1 is actually built on top of DO's as an interesting note. D1 uses the SQLite storage layer of DO's but as part of D1 there are also a bunch of boilerplate and helper functionality built on top to make it easier to interact with.

I also read that the max size in DO was 1GB

They did use to be 1GB not too long ago, but not it is 10GB

what are the pro/cons of using sqlite in DO vs using D1

Pros of DO: you have a lot more flexibility to do what you want. Compute & storage all at the same time. Cons of DO: you have to build the boilerplate interfacing systems to inspect them at the moment (but there are libraries that do this). From a storage layer perspective.. they are both DO SQLite!

Ddsds10 hmm that's interesting. somehow I had the "sqlite in do" stored as "ephemeral" i...

Larry•6/7/25, 11:12 AM

Pros: dynamic binding, hibernating WebSockets, ACID with stored procedures written in JavaScript

Cons: mental model shift from traditional 3 tier architecure

LLarry Pros: dynamic binding, hibernating WebSockets, ACID with stored procedures writt...

1984 Ford Laser•6/7/25, 11:56 AM

That mental model shift can be really hard for some people

dsds10•6/7/25, 1:26 PM

@Brayden thanks! what libraries are you referring to? Just made some tests to auto create instances of my backend system on demand and it works surprisingly well! kinda feels like making a cloud self service in a day

Am I right to assume read/writes are not counted like in D1 as it's effectively querying it "locally"? I think just the requests to the DO and the duration it lives is billed afaik.

@Larry seems like everything has it's pro's and con's

only scratched the surface of DO's so far, but it works great

dsds10•6/7/25, 2:37 PM

works like a charm! in case anyone curious, here is an example: https://example.bknd.run/ (ACL enabled)
you can create your own instance by changing the subdomain to whatever you like

really cool what D1 and DO enables us to create :cloudflare:

thipperz•6/7/25, 3:55 PM

My D1 is dead has been 10 hours

Apparently this happens if you try to export a remote database that has fts5 tables on it.

I would appreciate if someone from Cloudflare could take a look at workers-and-pages-helpD1 ERROR: Currently processing a long-running export

Ddsds10 works like a charm! in case anyone curious, here is an example: https://example....

1984 Ford Laser•6/8/25, 12:46 AM

Love to see when people 'figure out' DOs and realise how powerful and interesting they are
Just a note that you are still charged for SQL reads/writes in DOs, but the read pricing is very reasonable

Ddsds10 works like a charm! in case anyone curious, here is an example: https://example....

bkyerv•6/8/25, 4:31 AM

what?

gabrielstuff•6/9/25, 7:21 AM

Hello, are D1 database compatible with RGPD ?

Any DPA link or documentation about this CF product ?

Thanks

Tthipperz 🔴 My D1 is dead has been 10 hours 😭 Apparently this happens if you try to ex...

thipperz•6/9/25, 10:03 AM

My D1 and therefore my whole project is still down.

Its been down for more than 50 hours right now, hopefully I can get a fix today, PLEASE

Tthipperz My D1 and therefore my whole project is still down. Its been down for more than...

lambrospetrou•6/9/25, 10:57 AM

I rebooted your database now, so the stuck export should be gone. Sorry for the annoying behavior, it's been one of the things we need to prioritize. Please let me know if it looks fine on your side now too.

Llambrospetrou I rebooted your database now, so the stuck export should be gone. Sorry for the ...

thipperz•6/9/25, 10:59 AM

It is back now, thank you!!

parisholley•6/9/25, 5:50 PM

how much downtime can i expect if turning replication off on D1? i enabled for a portion of our customers and it just isn't realiable. at least once a week i feel like there are 10-20min outages

Pparisholley how much downtime can i expect if turning replication off on D1? i enabled for a...

lambrospetrou•6/9/25, 6:22 PM

What do you mean downtime? There shouldn't be a downtime. The replica information is cached in the locations where it has been used once, so it needs a few hours for them to completely stop being used, though.

at least once a week i feel like there are 10-20min outages

Can you give me details about this? What kind of outages? Was it due to replicas? In all regions, or specific ones? I don't recall some past message here. Feel free to DM me the database and account IDs and will check it.

parisholley•6/9/25, 6:26 PM

i'm not using any replica functionality, i just turned it on for some of our smaller clients as a test. the clients where we have turned it on keep reporting worker errors due to outages

Pparisholley i'm not using any replica functionality, i just turned it on for some of our sma...

lambrospetrou•6/9/25, 6:28 PM

the clients where we have turned it on keep reporting worker errors due to outages

What kind of errors? Can you provide me with some timeranges when these happened?

parisholley•6/9/25, 6:29 PM

sure, i'll dm you

Llambrospetrou > Also is there any way to prevent this replication? Like when does that happen?...

planet•6/10/25, 7:04 AM

just seen this. any news on EU jurisdiction yet - for gdpr?

Pplanet just seen this. any news on EU jurisdiction yet - for gdpr?

lambrospetrou•6/10/25, 7:18 AM

It's on the roadmap for next quarter, but no exact release date yet, depending on priority changes.

C•6/10/25, 12:37 PM

Hi, anyone have experiences on this?

D1 [ERROR] Currently processing a long-running export.

Database stucks during export, can not view from Cloudflare dashboard as well. (API error 500)

Now my website crashed due to this.
https://community.cloudflare.com/t/d1-error-currently-processing-a-long-running-export/806216

scotto•6/10/25, 4:36 PM

how do use d1 dynamically from a worker? for example passing a string with the id instead of the normal binding? for context, in a multi tenant where each tenant has a d1 database

CC Hi, anyone have experiences on this? D1 [ERROR] Currently processing a long-run...

lambrospetrou•6/10/25, 4:57 PM

Can you send me your account ID and database ID and I can restart the DB for you, if it's still stuck?
Is this the first time you tried to export and faced this issue or did this happen again another time for the specific database?
I am trying to pinpoint some common cases where exports take much longer than usual, hence why I am asking.

Sscotto how do use d1 dynamically from a worker? for example passing a string with the i...

dsds10•6/10/25, 8:57 PM

Afaik you have 3 ways to achieve this:

update worker bindings via the API to add an additional d1 database (docs states that you can theoretically bind up to 5k dbs, env limits)
use d1 over http (but queries are way slower)
use durable objects with sql storage (downside is that you can‘t use the tooling d1 offers)

Ddsds10 Afaik you have 3 ways to achieve this: 1. update worker bindings via the API to...

scotto•6/10/25, 8:59 PM

thanks, regarding DO with sql storage, it's actually possible to use N sql storages each one with the 1 gb limit?

Sscotto thanks, regarding DO with sql storage, it's actually possible to use N sql stora...

dsds10•6/10/25, 9:06 PM

Actually it‘s now 10gb, I just found out a few days ago that this limit was lifted. Afaik you can create unlimited DO‘s but each can only have a single sql storage

Llambrospetrou Can you send me your account ID and database ID and I can restart the DB for you...

C•6/10/25, 11:19 PM

My database back to normal now after 8 hours. It has around 400k rows and total size is 37 MB. Thank you!

FYI My account ID
32c307ab568d8c7e7f64f6694bce4ee4

Database ID
c932e1d8-5c3f-4c4b-b8d4-cd9352b58a85

Brayden•6/11/25, 3:12 PM

Released an update today that removes the "Tables" tab from your D1 instance and replaces it with a new "Explore Data" button at the top right to take you into our more fully featured SQL Studio to interact with your data.

CC My database back to normal now after 8 hours. It has around 400k rows and total ...

C•6/11/25, 3:35 PM

I triggered the issue again.

The first time I tried to export:
"X [ERROR] cell.replace is not a function"

The second time I tried to export:
"X [ERROR] Currently processing a long-running export. Cannot start a new export until that completes or times out."

Now the database stucks, seems I need to wait until tomorrow again.

beingmudit [WePlayDOS]•6/11/25, 3:37 PM

Hey CF team, is there any outage going on with D1 at the moment? I am experiencing 500 due to D1 not responding.

Bbeingmudit [WePlayDOS]Hey CF team, is there any outage going on with D1 at the moment? I am experienci...

C•6/11/25, 3:39 PM

Did you run "wrangler d1 export " using wrangler 4.19.1 or other version?
I met the same problem due to this action.
Now my DB went 500 and website down.

beingmudit [WePlayDOS]•6/11/25, 3:40 PM

I haven't touched it TBH

beingmudit [WePlayDOS]•6/11/25, 3:40 PM

It was working fine yesterday night

beingmudit [WePlayDOS]•6/11/25, 3:40 PM

And today intermittent 500

beingmudit [WePlayDOS]•6/11/25, 3:42 PM

Errors like this

Error: D1_ERROR: internal error; reference = 2h8oi9k493ebe4vtjriu3s41

C•6/11/25, 3:44 PM

Sorry to hear that, seems not the same issue with mine.
Yesterday my DB crash, I wait 8 hours until it back online automatically.

Bbeingmudit [WePlayDOS]Hey CF team, is there any outage going on with D1 at the moment? I am experienci...

lambrospetrou•6/11/25, 4:01 PM

Not widespread, no. There were some sporadic issues in APAC with a handful of databases having elevated errors but not active issues at the moment.

jiulongw•6/11/25, 5:51 PM

Hi, if I have two D1 databases in my account, one for production and one for staging. Is there anyway to limit access to production database to some of the account members? Say we have compliance on who can access customer's data. It seems there is no resource based permission management for members in the account. Is there a solution other than creating a separate Cloudflare account for production environment? Thanks!

Jjiulongw Hi, if I have two D1 databases in my account, one for production and one for sta...

lambrospetrou•6/11/25, 5:54 PM

Unfortunately there aren't any resource level permissions at the moment. You can restrict writing to databases but as long as they have d1:readd1:read permissions they would be able to read all DBs.

Llambrospetrou Unfortunately there aren't any resource level permissions at the moment. You can...

jiulongw•6/11/25, 5:58 PM

Thanks! Looks like a new account is the way to go.

beingmudit [WePlayDOS]•6/11/25, 6:35 PM

Yeah my D1 is in APAC only. Do you see anything with this trace - 2h8oi9k493ebe4vtjriu3s41

beingmudit [WePlayDOS]•6/11/25, 6:37 PM

Thanks for updating on the APAC issue.

CC I triggered the issue again. The first time I tried to export: "X [ERROR] cell....

C•6/12/25, 1:59 AM

Tried to use Wrangler 4.19.2, the issue still exists.
Unable to export remote Database now.
cell.replace is not a function

C•6/12/25, 1:59 AM

SaM•6/12/25, 6:10 AM

Hi everyone! I have a question about Cloudflare D1 replication and latency:

My D1 database’s primary region is APAC (I’m in India).

If I call my Worker API without read replicas, my responses in India are around 300ms.

If I enable read replicas, my response time in India goes up to 500–700ms.

For a user in the UK, with read replicas enabled, the response comes from the WEUR region and is under 100ms (which is great!).

But if I turn off read replicas, UK users get responses from APAC and latency is over 700ms.

My question:
Is there any way to make my responses in India faster with D1 replication enabled?
Why is enabling read replicas increasing my latency locally, and what’s the best way to optimize for both my region (India) and global users?

Any tips or best practices would be greatly appreciated!

SSaM Hi everyone! I have a question about Cloudflare D1 replication and latency: My ...

lambrospetrou•6/12/25, 8:30 AM

Without database/account ID details I cannot know for sure, but I can see what you describe happening if the replica you are hitting from India is farther from your main database depending on which region you get routed since India doesn't have a region yet.
There is a plan for optimizing the choice of the replica to be the closest but now it's just selecting the replica of your closest region.

Llambrospetrou Without database/account ID details I cannot know for sure, but I can see what y...

SaM•6/12/25, 8:32 AM

45688789-5491-40fc-9532-191a5d7056db -> Database ID

TIL about the SQL storage of Durable Objects... thanks!

Similar Threads

Similar Threads

Similar Threads