Cloudflare Developers•9mo ago

Can't you replace the DB instead of dropping the table? 1. Create new DB + insert data 2. Replace w

Can't you replace the DB instead of dropping the table?

Create new DB + insert data
Replace worker binding
Delete old DB

Zack•4/7/25, 2:45 PM

Yea, I wouldn't base anything necessarily off of my tables, we store some decent size blobs of data within each row

amanzad•4/7/25, 3:48 PM

Hey! Quick question about D1—if I run a query to check whether a specific id exists in a table, would that count as just one read, or would it count as one read per row in the table?

Aamanzad Hey! Quick question about D1—if I run a query to check whether a specific id exi...

Zack•4/7/25, 3:51 PM

If you have an index that SQLite can use for that query, it'd be a single read

Zack•4/7/25, 3:51 PM

If not, it'll scan the entire table

Kkerohije Can't you replace the DB instead of dropping the table? 1. Create new DB + inse...

gruntlord6•4/7/25, 6:18 PM

thats what I would do, might be harder in practice

Ggruntlord6 thats what I would do, might be harder in practice

James•4/7/25, 6:29 PM

Any regular process which involves dropping and recreating millions of rows of a table should be rethought architecture-wise, regardless if this was D1 or Postgres. If I was stuck on such a process, i'd probably run a blue/green deployment on this table to keep both pre-post hot to avoid migration downtime.

Llambrospetrou How are you printing that? An array in JS is an object with numbers as keys, and...

Corsix👾•4/7/25, 6:38 PM

Corsix👾•4/7/25, 6:39 PM

startDataUUIDs is a array of uuids

JJames Any regular process which involves dropping and recreating millions of rows of a...

gruntlord6•4/7/25, 9:05 PM

yea often times its easier to re-think the issue rather then solve obscure edge cases

Kkerohije Can't you replace the DB instead of dropping the table? 1. Create new DB + inse...

Zack•4/7/25, 11:16 PM

That's an option, but one that will suck once D1 global replication exists

Zack•4/7/25, 11:16 PM

Given that will presumably replicate over time, and increase the PoP's it exists in, replacing the entire database will destroy that replication and you'll start from step 1

JJames Any regular process which involves dropping and recreating millions of rows of a...

Zack•4/7/25, 11:17 PM

Something like this would be totally fine in postgres, it's an extremely simple operation that won't impact the database whatsoever

ZZack Given that will presumably replicate over time, and increase the PoP's it exists...

Isaac McFadyen•4/7/25, 11:31 PM

Once it lands, D1 replication is extremely quick so this should still be a reasonable method of replacing the DB

Zack•4/7/25, 11:32 PM

I've not seen any info on that tbh

Zack•4/7/25, 11:34 PM

I imagine the process of replication is fast, but I expect the process of learning where to replicate too is not necessarily as fast

Zack•4/7/25, 11:35 PM

And switching out your DB and dumping all load onto a single node while it replicates out is not going to be great

Isaac McFadyen•4/7/25, 11:38 PM

I can't really say too much at the moment other than the fact that switching out a DB will still be a reasonable thing to do once replication lands

Zack•4/7/25, 11:41 PM

We'll see whenever that lands I guess, regardless, dropping a table should not be something that takes out a production database

gruntlord6•4/7/25, 11:41 PM

if its anything like other platforms the writes go to all replicas without any work on your end

gruntlord6•4/7/25, 11:41 PM

and it's basically instant

Zack•4/7/25, 11:41 PM

If you replace the entire database, there are no replicas

gruntlord6•4/7/25, 11:41 PM

but like they said can't speak for something we don't have

ZZack We'll see whenever that lands I guess, regardless, dropping a table should not b...

Isaac McFadyen•4/7/25, 11:43 PM

Sure but if you want zero downtime, you duplicate the DB, drop the table, and switch the "connection" (binding)

IIsaac McFadyen Sure but if you want zero downtime, you duplicate the DB, drop the table, and sw...

Zack•4/7/25, 11:44 PM

Yes, that is something we've considered, it's just more work

Zack•4/7/25, 11:44 PM

We're already having to work around multiple other issues that we didn't expect lol

く

くもりにくもったクラウドサービスlocaler•4/9/25, 2:00 PM

Hi. I encountered a consistent issue when editing a row in a D1 table via the dashboard. Attempting to edit the following string causes a 404 Bad Request:

data.txt51B

く

くもりにくもったクラウドサービスlocaler•4/9/25, 2:00 PM

I'm not exactly sure which character is triggering the error — it might be due to a special character or even an invisible whitespace — but the issue is 100% reproducible when trying to update a row containing this exact value.

ZZack We're already having to work around multiple other issues that we didn't expect ...

Tait•4/9/25, 5:24 PM

I'd love to hear more. What other issues did you unexpectedly encounter?

TTait I'd love to hear more. What other issues did you unexpectedly encounter?

Zack•4/9/25, 5:46 PM

Bulk import API causing the database to be unusable (which is documented to be fair), the D1 API having severe rate limits (worked around by using our own worker as a proxy for SQL commands), some issue internally with D1 causing huge latency after we replaced a bulk of the data within the database that the team fixed

Zack•4/9/25, 5:47 PM

We are using D1 in a slightly abnormal way, as a lookup table basically, so you probably won't experience the same problems if you're using it as a more traditional database

Tait•4/9/25, 8:57 PM

Thank you!

くくもりにくもったクラウドサービスlocaler I'm not exactly sure which character is triggering the error — it might be due t...

gruntlord6•4/9/25, 10:24 PM

does that end up in the url of the edit page?

Ggruntlord6 does that end up in the url of the edit page?

く

くもりにくもったクラウドサービスlocaler•4/9/25, 10:36 PM

Yes, the issue happens when click "Update" on a row that contains that string the value ends up in the URL, and that's when the 400 error oCcurs.
So it seems like the string being placed in the URL is causing the problem.

gruntlord6•4/9/25, 10:56 PM

its likely a character that can't be in a url, like the one used as a space character, the browser is interpreting it as a space etc and that causes the 404

gruntlord6•4/9/25, 10:57 PM

can't say which specific one is the issue but I'd say thats the issue IMO