yeah i don't really see it being feasible for my use case mentioned earlier in the thread; i want to

yeah i don't really see it being feasible for my use case mentioned earlier in the thread; i want to export it and use it in duckdb to power a few analytics heavy queries. i don't need real time, but something like per hour snapshots is what i'm looking to do. exporting by manually reading every row on every table i have is sure possible but a) costly and b) can't achieve fresh enough analytics (as i suspect the exports in this way would take hour+ when applied to a 10gb size db)

really eager to get some guidance from anyone at CF to see if this is something actually in the works for the remainder of this year. multiple docs mention there's but this d1 channel seems abandoned from PM perspective compared to the other channels

gruntlord6•7/28/25, 3:47 PM

Isn't it the same amount of reads to dump the entire database?

gruntlord6•7/28/25, 3:49 PM

D1 isn't abandoned, they just launched read replication the other month. Both d1 and durable objects are actively developed and share a lot of backend technology with each other (d1 is based on durable objects and durable objects use SQL storage based on d1)

dandoenOP•7/28/25, 4:35 PM

ah i mean this channel, not the product

depends on the implementation on the backend i suppose. in the way i understand how the sqlite instances are stored in a DO (largely based on this), i imagine it would be easier to save the d1 row in its entirety to "cold" storage (which essentially r2 would be) than to first read it to in memory, then iterate over each row in the db, export it etc.

Ddandoen yeah i don't really see it being feasible for my use case mentioned earlier in t...

lambrospetrou•7/28/25, 4:59 PM

Exports and some form of cloning a D1/DO DB's data into a separate D1/DO are on our roadmap. Other priorities unfortunately pushed them out of the past two quarters, but I am hoping that we will get to them by end of the year.
This is not a promised ETA, just to be clear, but it's been on our roadmap since the start of the year, just pushed down due to other things.

Llambrospetrou Exports and some form of cloning a D1/DO DB's data into a separate D1/DO are on ...

James•7/28/25, 5:56 PM

This is definitey something i'm highly interested in for DOs.

my strategy up to now is to send key events to an analytics store outside CF for cross DO analytics, but it would be a big plus to run the heavy stuff against an offline store.

JJames This is definitey something i'm highly interested in for DOs. my strategy up to...

dandoenOP•7/28/25, 10:04 PM

understood, thx @lambrospetrou . helps plan ahead architecturally knowing this is tracked and will make it in soon enough. thx!

Ddandoen ah i mean this channel, not the product 🙂 depends on the implementation on the ...

gruntlord6•7/28/25, 10:30 PM

That's actually the approach I take right now in my app that needs to backup user durable objects. We have a daily alarm that each durable object uses to dump itself to r2

Mmrbarletta Can't even delete the table using the UI 😐

lambrospetrou•7/30/25, 8:55 AM

Where did you get that error? From the REST API, from the Worker binding, through some other library/framework? That doesn't seem like something D1 throws. There wasn't an incident or service issue in D1 either.

Dean•7/30/25, 9:58 AM

is d1 replication relatively stable now? i've put off trying it after a few had some issues originally

DDean is d1 replication relatively stable now? i've put off trying it after a few had ...

lambrospetrou•7/30/25, 11:07 AM

There have been several bugfixes shipped over the past few months since the initial release, so more folks stress testing it would be great. Apart from improvements we have on the roadmap there isn't any active issue right now that should scare you off.
As always with any major feature, my suggestion is to first roll it out in your staging/test environment to make sure your use of the Sessions API is correct and you see replica usage, and then to production. Since, simply enabling replication should still work without using Sessions API, but you will start to be proxied by the nearest replica.

Please send us feedback or details if you find something not working as expected.

Llambrospetrou There have been several bugfixes shipped over the past few months since the init...

Dean•7/30/25, 11:08 AM

Will do! Thanks

Dean•7/30/25, 11:13 AM

Does it work on pages or is it only workers? I haven’t migrated everything over to workers yet

Noobie☺•7/30/25, 11:38 AM

hi. i need help with monorepo setup(https://github.com/waptik/woym-template). i have a packages/dbpackages/db where a wrangler.jsoncwrangler.jsonc with d1 bindings is present(without app name), and a apps/serverapps/server with another wrangler.jsoncwrangler.jsonc with d1 bindings and app name.
I use the db package to perform operations in packages/apipackages/api(orpc) which is attached to the server app. In my web app, i was trying to perform some queries(todos and authentication with better-auth) but was getting internal error code with some logs saying the table could not be found which was weird. after days of debugging, i just found out that a local db file also being created inside apps/server/.wranglerapps/server/.wrangler. so wrangler was using the local db inside the server directory instead of the one in db.

How do i make wrangler in server use the local db from packages/dbpackages/db instead?

UPDATE: i figured it by using --persist-to--persist-to flag like this "dev": "wrangler dev --persist-to ../../packages/db/.wrangler/state""dev": "wrangler dev --persist-to ../../packages/db/.wrangler/state"

DDean Does it work on pages or is it only workers? I haven’t migrated everything over ...

lambrospetrou•7/30/25, 12:20 PM

As long as you use the D1 Binding from your worker, or Pages function, it should work.

Reed Hewitt•7/30/25, 7:10 PM

Hello, I have run into an issue with tables that reference each other. I need to change column constraint on table X, which happens to have foreign key constraints pointing to tables Y and Z. Y and Z also have constraints pointing to table X. I used to be able to create a new version of the table named X_new, copy the data over from the original X, rename the original X to X_old and rename X_new to X. That would all happen in one transaction with PRAGMA defer_foreign_keys = truePRAGMA defer_foreign_keys = true. I could then confirm that all data was identical between X and X_old, then drop X_old. But now this doesn't work and I get the error FOREIGN KEY constraint failed: SQLITE_CONSTRAINT [code: 7500]FOREIGN KEY constraint failed: SQLITE_CONSTRAINT [code: 7500]. I see that D1 now updates the references on Y and Z so that they point at X_old instead of the newly created version of X. If I do the same copy/rename trick on Y and Z, then my new X ends up referencing Y_old and Z_old. I can't figure out how to get a clean reset with the new X, Y, and Z pointing to each other and nothing pointing to the old tables so that they can be deleted. Is there a way to disable the auto-renaming of foreign key constraints so that the previous behavior would work?

Reed Hewitt•7/30/25, 7:29 PM

Ok, I figured it out... I have to create new versions of the tables that reference each other by their new names (X_new, Y_new, Z_new). After creating all the new tables, then I do all the data copying. Then rename everything (old and new). The new versions of the tables will follow the renaming of each other. That allows the old versions of the tables to be dropped, because none of the new tables hold on to renamed foreign key constraints pointing to them.

James•7/30/25, 8:47 PM

sounds like a modeling problem more than a database constraint problem

WTDawson9•7/31/25, 12:30 PM

I wish this was called D2-database instead so we could have R2D2

WTDawson9•7/31/25, 12:31 PM

https://tenor.com/view/scream-gif-14892509196082257073

Tenor

dvddvd300•7/31/25, 6:58 PM

hi, i have multiple d1 DB, but 1 primary, with +5GB of data, in the list of dbs shows as 1gb, and also in billing, but in the actual stats for the db shows as 5.3GB, i don't know what number to believe i hope its the 5.3gb and the billing is giving free +4gb, but the 1gb number scares me if some data was lost

Robert•8/1/25, 3:27 PM

Is there a way to enforce my Worker function has read-only access to data stored in D1 on an API/network level?
I don't think there is, but wanted to confirm

RRobert Is there a way to enforce my Worker function has read-only access to data stored...

Larry•8/1/25, 5:30 PM

You are correct. No way to do it directly. You could proxy it through another worker where you did that or you could use a DO and save a network hop compared to the Worker only proxy

RRobert Is there a way to enforce my Worker function has read-only access to data stored...

Larry•8/1/25, 5:33 PM

D1 is implemented in DO so that approach would be just as efficient but it’s more work for you and you lose read replication

Llambrospetrou There have been several bugfixes shipped over the past few months since the init...

Zack•8/1/25, 9:59 PM

Oh yes, I should probably give you some feedback, since I turned on replication again a few weeks back we have now torn down our individual databases for each region and exclusively use the replicas now

Zack•8/1/25, 10:02 PM

Now that it bypasses the replica for the first request if the replica isn't active, we've had no issues except seemingly the occasional issue with APAC where the network latency spikes

ZZack Oh yes, I should probably give you some feedback, since I turned on replication ...

lambrospetrou•8/2/25, 6:27 AM

Thanks for getting back to us

If you notice anything, please reach out.

Llambrospetrou There have been several bugfixes shipped over the past few months since the init...

gruntlord6•8/2/25, 8:04 PM

So even without the API we can benefit from read replicas now?

onewug•8/2/25, 10:11 PM

hi! is this example really working in vitest watch mode for you? it fails after a second run with error code SQLITE_CANTOPEN https://github.com/cloudflare/workers-sdk/tree/main/fixtures/vitest-pool-workers-examples/d1

GitHub

workers-sdk/fixtures/vitest-pool-workers-examples/d1 at main · clo...

Home to Wrangler, the CLI for Cloudflare Workers® - cloudflare/workers-sdk

workers-sdk/fixtures/vitest-pool-workers-examples/d1 at main · clo...

onewug•8/2/25, 10:24 PM

created an issue: https://github.com/cloudflare/workers-sdk/issues/10195

GitHub

vitest-pool-workers-examples/d1 fails in vitest watch mode · Issue...

What versions & operating system are you using? System: OS: macOS 15.5 CPU: (8) arm64 Apple M2 Memory: 188.55 MB / 24.00 GB Shell: 4.0.2 - /opt/homebrew/bin/fish Binaries: Node: 22.17.1 - ~/.lo...

gruntlord6•8/3/25, 5:14 AM

Yea I feel like they are saying there's still a benefit which would be big even if you don't use the api

gruntlord6•8/3/25, 5:18 AM

Yea they just worded it in a way I felt like should ask

gruntlord6•8/3/25, 5:18 AM

Not breaking things is of course good lol

Ggruntlord6 So even without the API we can benefit from read replicas now?

lambrospetrou•8/3/25, 7:11 AM

If by "API" you mean the Sessions API, then you still have to use it to take advantage of replicas, otherwise there is no way to guarantee your queries are consistent. Without using the Sessions API, only the primary executes your queries even if they flow through replicas. If by API you mean the REST API, read replication is not yet exposed in the REST API.

Pepper•8/3/25, 9:37 PM

small suggestion: there should be a pop up warning if you try to leave d1 studio with unsaved edits

zsueo•8/4/25, 10:10 AM

hey is anyone else having issues with the wrangler d1 cli?

zsueo•8/4/25, 10:10 AM

i keep running into internal errors when trying to dump the db from the cli

zsueo•8/4/25, 10:10 AM

and the dashboard is being oddly slow

zsueo•8/4/25, 10:12 AM

ah thanks

PPepper small suggestion: there should be a pop up warning if you try to leave d1 studio...

Visal .In•8/4/25, 12:24 PM

I will work on it. Thanks for feedback

Matthew•8/4/25, 12:34 PM

I've created a D1 database in ENAM, enabled read replication, am querying it from a worker running in Manchester GB, but all my queries are going to the primary (meta.served_by_primarymeta.served_by_primary = true). The Cloudflare D1 dashboard says ENAM is the only region. Is there a minimum amount of time, or number of queries, before replicas are created? Have I missed a step?

Matthew•8/4/25, 12:48 PM

(to add, I'm querying with DB_REPLICATED.withSession("first-unconstrained")DB_REPLICATED.withSession("first-unconstrained"))

MMatthew I've created a D1 database in ENAM, enabled read replication, am querying it fro...

lambrospetrou•8/4/25, 2:47 PM

There is no minimum time or anything. The first time just needs a few requests so that the information propagates to the location creating the requests, and then just the time for the replica in that region to boot (you won't be blocked on this time).

But, if you only send a request every few minutes, then all of them will end up in the primary.

How many requests are you sending? Do you still not see replicas processing your queries?

I'm querying with DB_REPLICATED.withSession("first-unconstrained")

Just to confirm, you are doing reads only right?

Llambrospetrou There is no minimum time or anything. The first time just needs a few requests s...

Matthew•8/4/25, 3:02 PM

~400 queries over the last few hours, all reporting served by primary. Query is a SELECT.

MMatthew ~400 queries over the last few hours, all reporting served by primary. Query is ...

lambrospetrou•8/4/25, 3:03 PM

DM me your database ID and account ID and I can take a look.

Llambrospetrou DM me your database ID and account ID and I can take a look.

lambrospetrou•8/4/25, 3:53 PM

Closing the loop here, the issue was transient during the replication toggle originally when the triggered it not actually persisting the replica enablement. Toggling it on/off again fixed it. We will look into the original error and improve it so that you don't have to toggle it manually.

Abakir•8/4/25, 6:07 PM

This might be a dumb question if a 1000 users try to write to the database at the same time or make a query it won’t fault correct.

I know there are limits of d1 database

AAbakir This might be a dumb question if a 1000 users try to write to the database at th...

James•8/4/25, 6:49 PM

single threaded db, you got 1000ms in a second, gotta work out your usage constraints based on that

yeah i don't really see it being feasible for my use case mentioned earlier in the thread; i want to

Similar Threads

Similar Threads

Similar Threads