I'm still more than simply curious to understand how an impossible thing happened. Can you run, `PRA

I'm still more than simply curious to understand how an impossible thing happened. Can you run,

PRAGMA table_info(table_name)

PRAGMA table_info(table_name)

PRAGMA table_info(table_name)

PRAGMA table_info(table_name) and show us the output? I'm wondering if the table was originally created with a different primary key and changed your code to something like,

CREATE TABLE IF NOT EXISTS...

CREATE TABLE IF NOT EXISTS...

CREATE TABLE IF NOT EXISTS...

CREATE TABLE IF NOT EXISTS... with the primary key that you now think is the primary key but it actually isn't.

LLarry I'm still more than simply curious to understand how an impossible thing happene...

James•7/26/25, 5:19 PM

definitle not default behavior, messing w the key / rowid / pragma settings could get you there

LLarry I'm still more than simply curious to understand how an impossible thing happene...

Pepper•7/26/25, 6:27 PM

Here is the output (num is what I expected the primary key to be, in my earlier message I wrote primary_key just for the example). It originally happened when I used d1 manager to add rows to the table but didn't enter any values. But I was able to replicate this by doing:
INSERT INTO queue DEFAULT VALUES;
What that query does for me is insert everything as null except for the primary key, which it sets to be equal to the largest primary key in my database.

LarryOP•7/26/25, 7:11 PM

Ohh, I remember that bug in SQLite that allows null primary key values unless you create the table with that column set to

NOT NULL

NOT NULL

. https://sqlite.org/quirks.html#primary_keys_can_sometimes_contain_nulls

LarryOP•7/26/25, 7:20 PM

Wait, I'm still confused...

The value of an INTEGER PRIMARY KEY column must always be a non-NULL integer because the INTEGER PRIMARY KEY is an alias for the ROWID. If you try to insert a NULL into an INTEGER PRIMARY KEY column, SQLite automatically converts the NULL into a unique integer.

Ddandoen if only there was a way – as a work around – to do point in time recovery of a d...

dandoen•7/28/25, 8:33 AM

hi all, are there any cloudflare PMs that can shed some light whether this is something on the roadmap for this year? feels fairly important to be able to export a d1 db without blocking all reads/writes.

Original message was deleted

dandoen•7/28/25, 11:12 AM

thanks nora, just worried to go down this path w/ some of my d1 instances being fairly large (and the upperl imit being 10gb)

Ddandoen thanks nora, just worried to go down this path w/ some of my d1 instances being ...

gruntlord6•7/28/25, 3:39 PM

Reading your data is pretty straightforward so you shouldn't really have any issues with checking the integrity of your copy etc. I prefer the d1 dump but the file lock is a big downside for production data

Ggruntlord6 Reading your data is pretty straightforward so you shouldn't really have any iss...

dandoen•7/28/25, 3:43 PM

yeah i don't really see it being feasible for my use case mentioned earlier in the thread; i want to export it and use it in duckdb to power a few analytics heavy queries. i don't need real time, but something like per hour snapshots is what i'm looking to do. exporting by manually reading every row on every table i have is sure possible but a) costly and b) can't achieve fresh enough analytics (as i suspect the exports in this way would take hour+ when applied to a 10gb size db)

really eager to get some guidance from anyone at CF to see if this is something actually in the works for the remainder of this year. multiple docs mention there's but this d1 channel seems abandoned from PM perspective compared to the other channels

gruntlord6•7/28/25, 3:47 PM

Isn't it the same amount of reads to dump the entire database?

gruntlord6•7/28/25, 3:49 PM

D1 isn't abandoned, they just launched read replication the other month. Both d1 and durable objects are actively developed and share a lot of backend technology with each other (d1 is based on durable objects and durable objects use SQL storage based on d1)

dandoen•7/28/25, 4:35 PM

ah i mean this channel, not the product

depends on the implementation on the backend i suppose. in the way i understand how the sqlite instances are stored in a DO (largely based on this), i imagine it would be easier to save the d1 row in its entirety to "cold" storage (which essentially r2 would be) than to first read it to in memory, then iterate over each row in the db, export it etc.

Ddandoen yeah i don't really see it being feasible for my use case mentioned earlier in t...

lambrospetrou•7/28/25, 4:59 PM

Exports and some form of cloning a D1/DO DB's data into a separate D1/DO are on our roadmap. Other priorities unfortunately pushed them out of the past two quarters, but I am hoping that we will get to them by end of the year.
This is not a promised ETA, just to be clear, but it's been on our roadmap since the start of the year, just pushed down due to other things.

Llambrospetrou Exports and some form of cloning a D1/DO DB's data into a separate D1/DO are on ...

James•7/28/25, 5:56 PM

This is definitey something i'm highly interested in for DOs.

my strategy up to now is to send key events to an analytics store outside CF for cross DO analytics, but it would be a big plus to run the heavy stuff against an offline store.

JJames This is definitey something i'm highly interested in for DOs. my strategy up to...

dandoen•7/28/25, 10:04 PM

understood, thx @lambrospetrou . helps plan ahead architecturally knowing this is tracked and will make it in soon enough. thx!

Ddandoen ah i mean this channel, not the product 🙂 depends on the implementation on the ...

gruntlord6•7/28/25, 10:30 PM

That's actually the approach I take right now in my app that needs to backup user durable objects. We have a daily alarm that each durable object uses to dump itself to r2

Mmrbarletta Can't even delete the table using the UI 😐

lambrospetrou•7/30/25, 8:55 AM

Where did you get that error? From the REST API, from the Worker binding, through some other library/framework? That doesn't seem like something D1 throws. There wasn't an incident or service issue in D1 either.

Dean•7/30/25, 9:58 AM

is d1 replication relatively stable now? i've put off trying it after a few had some issues originally

DDean is d1 replication relatively stable now? i've put off trying it after a few had ...

lambrospetrou•7/30/25, 11:07 AM

There have been several bugfixes shipped over the past few months since the initial release, so more folks stress testing it would be great. Apart from improvements we have on the roadmap there isn't any active issue right now that should scare you off.
As always with any major feature, my suggestion is to first roll it out in your staging/test environment to make sure your use of the Sessions API is correct and you see replica usage, and then to production. Since, simply enabling replication should still work without using Sessions API, but you will start to be proxied by the nearest replica.

Please send us feedback or details if you find something not working as expected.

Llambrospetrou There have been several bugfixes shipped over the past few months since the init...

Dean•7/30/25, 11:08 AM

Will do! Thanks

Dean•7/30/25, 11:13 AM

Does it work on pages or is it only workers? I haven’t migrated everything over to workers yet

Noobie☺•7/30/25, 11:38 AM

hi. i need help with monorepo setup(https://github.com/waptik/woym-template). i have a packages/dbpackages/db where a wrangler.jsoncwrangler.jsonc with d1 bindings is present(without app name), and a apps/serverapps/server with another wrangler.jsoncwrangler.jsonc with d1 bindings and app name.
I use the db package to perform operations in packages/apipackages/api(orpc) which is attached to the server app. In my web app, i was trying to perform some queries(todos and authentication with better-auth) but was getting internal error code with some logs saying the table could not be found which was weird. after days of debugging, i just found out that a local db file also being created inside apps/server/.wranglerapps/server/.wrangler. so wrangler was using the local db inside the server directory instead of the one in db.

How do i make wrangler in server use the local db from packages/dbpackages/db instead?

UPDATE: i figured it by using

--persist-to

--persist-to

flag like this "dev": "wrangler dev --persist-to ../../packages/db/.wrangler/state""dev": "wrangler dev --persist-to ../../packages/db/.wrangler/state"

DDean Does it work on pages or is it only workers? I haven’t migrated everything over ...

lambrospetrou•7/30/25, 12:20 PM

As long as you use the D1 Binding from your worker, or Pages function, it should work.

Reed Hewitt•7/30/25, 7:10 PM

Hello, I have run into an issue with tables that reference each other. I need to change column constraint on table X, which happens to have foreign key constraints pointing to tables Y and Z. Y and Z also have constraints pointing to table X. I used to be able to create a new version of the table named X_new, copy the data over from the original X, rename the original X to X_old and rename X_new to X. That would all happen in one transaction with PRAGMA defer_foreign_keys = truePRAGMA defer_foreign_keys = true. I could then confirm that all data was identical between X and X_old, then drop X_old. But now this doesn't work and I get the error FOREIGN KEY constraint failed: SQLITE_CONSTRAINT [code: 7500]FOREIGN KEY constraint failed: SQLITE_CONSTRAINT [code: 7500]. I see that D1 now updates the references on Y and Z so that they point at X_old instead of the newly created version of X. If I do the same copy/rename trick on Y and Z, then my new X ends up referencing Y_old and Z_old. I can't figure out how to get a clean reset with the new X, Y, and Z pointing to each other and nothing pointing to the old tables so that they can be deleted. Is there a way to disable the auto-renaming of foreign key constraints so that the previous behavior would work?

Reed Hewitt•7/30/25, 7:29 PM

Ok, I figured it out... I have to create new versions of the tables that reference each other by their new names (X_new, Y_new, Z_new). After creating all the new tables, then I do all the data copying. Then rename everything (old and new). The new versions of the tables will follow the renaming of each other. That allows the old versions of the tables to be dropped, because none of the new tables hold on to renamed foreign key constraints pointing to them.

James•7/30/25, 8:47 PM

sounds like a modeling problem more than a database constraint problem

WTDawson9•7/31/25, 12:30 PM

I wish this was called D2-database instead so we could have R2D2

WTDawson9•7/31/25, 12:31 PM

https://tenor.com/view/scream-gif-14892509196082257073

Tenor

dvddvd300•7/31/25, 6:58 PM

hi, i have multiple d1 DB, but 1 primary, with +5GB of data, in the list of dbs shows as 1gb, and also in billing, but in the actual stats for the db shows as 5.3GB, i don't know what number to believe i hope its the 5.3gb and the billing is giving free +4gb, but the 1gb number scares me if some data was lost

Robert•8/1/25, 3:27 PM

Is there a way to enforce my Worker function has read-only access to data stored in D1 on an API/network level?
I don't think there is, but wanted to confirm

RRobert Is there a way to enforce my Worker function has read-only access to data stored...

LarryOP•8/1/25, 5:30 PM

You are correct. No way to do it directly. You could proxy it through another worker where you did that or you could use a DO and save a network hop compared to the Worker only proxy

RRobert Is there a way to enforce my Worker function has read-only access to data stored...

LarryOP•8/1/25, 5:33 PM

D1 is implemented in DO so that approach would be just as efficient but it’s more work for you and you lose read replication

Llambrospetrou There have been several bugfixes shipped over the past few months since the init...

Zack•8/1/25, 9:59 PM

Oh yes, I should probably give you some feedback, since I turned on replication again a few weeks back we have now torn down our individual databases for each region and exclusively use the replicas now

Zack•8/1/25, 10:02 PM

Now that it bypasses the replica for the first request if the replica isn't active, we've had no issues except seemingly the occasional issue with APAC where the network latency spikes

ZZack Oh yes, I should probably give you some feedback, since I turned on replication ...

lambrospetrou•8/2/25, 6:27 AM

Thanks for getting back to us

If you notice anything, please reach out.

Llambrospetrou There have been several bugfixes shipped over the past few months since the init...

gruntlord6•8/2/25, 8:04 PM

So even without the API we can benefit from read replicas now?

onewug•8/2/25, 10:11 PM

hi! is this example really working in vitest watch mode for you? it fails after a second run with error code SQLITE_CANTOPEN https://github.com/cloudflare/workers-sdk/tree/main/fixtures/vitest-pool-workers-examples/d1

GitHub

workers-sdk/fixtures/vitest-pool-workers-examples/d1 at main · clo...

Home to Wrangler, the CLI for Cloudflare Workers® - cloudflare/workers-sdk

workers-sdk/fixtures/vitest-pool-workers-examples/d1 at main · clo...

onewug•8/2/25, 10:24 PM

created an issue: https://github.com/cloudflare/workers-sdk/issues/10195

GitHub

vitest-pool-workers-examples/d1 fails in vitest watch mode · Issue...

What versions & operating system are you using? System: OS: macOS 15.5 CPU: (8) arm64 Apple M2 Memory: 188.55 MB / 24.00 GB Shell: 4.0.2 - /opt/homebrew/bin/fish Binaries: Node: 22.17.1 - ~/.lo...

gruntlord6•8/3/25, 5:14 AM

Yea I feel like they are saying there's still a benefit which would be big even if you don't use the api

gruntlord6•8/3/25, 5:18 AM

Yea they just worded it in a way I felt like should ask

gruntlord6•8/3/25, 5:18 AM

Not breaking things is of course good lol

Ggruntlord6 So even without the API we can benefit from read replicas now?

lambrospetrou•8/3/25, 7:11 AM

If by "API" you mean the Sessions API, then you still have to use it to take advantage of replicas, otherwise there is no way to guarantee your queries are consistent. Without using the Sessions API, only the primary executes your queries even if they flow through replicas. If by API you mean the REST API, read replication is not yet exposed in the REST API.

Pepper•8/3/25, 9:37 PM

small suggestion: there should be a pop up warning if you try to leave d1 studio with unsaved edits