What you are proposing is a sort of two-phase commit which would work but I don't think it's necessa

What you are proposing is a sort of two-phase commit which would work but I don't think it's necessary. I can think of two alternatives. sqlite (and I assume D1) has a "function"

last_insert_rowid()

last_insert_rowid()

https://www.sqlite.org/c3ref/last_insert_rowid.html. That would allow you to insert one child with the parent. If you really want to insert more than one, then do the child inserts into a temporary table and delete as you move over to the real child table. I'm sure you could find an example of this on Stack Overflow. The second alternative would be to use a GUID that you generate client-side for your primary key. That doesn't take advantage of SQLite's really cool auto-increment rowid capability but that's really a nice-to-have.

Isaac McFadyen•3/31/24, 2:46 PM

In general the SQLite project actually recommends staying away from autoincrement on rowids because it imposes extra CPU/memory constraints, and recommends either leaving it as default (which will auto increment but also reuse the IDs of deleted rows) or using a UUID.

https://www.sqlite.org/autoinc.html

The AUTOINCREMENT keyword imposes extra CPU, memory, disk space, and disk I/O overhead and should be avoided if not strictly needed. It is usually not needed.

Isaac McFadyen•3/31/24, 2:47 PM

AUTOINCREMENT literally just prevents reuse of row IDs from deleted rows, so if you're fine reusing row IDs then just don't specify it.

Tony Leung•3/31/24, 3:47 PM

hi guys, will there be a feature which allows foreign key check to be turned off while migrating database? I have an issue here: https://github.com/cloudflare/workers-sdk/issues/5438

GitHub

🚀 Feature Request: Allow turning off foreign key check while migrat...

Some table altering tasks in SQLite cannot be achieved without replacing it with a new table (by dropping - recreating the table) (like adding foreign keys, changing primary keys, updating column t...

IIsaac McFadyen AUTOINCREMENT literally just prevents reuse of row IDs from deleted rows, so if ...

Tony Leung•3/31/24, 3:52 PM

In my experience, having AUTOINCREMENT in database will make a huge performance impact on database cluster. Everytime I insert a new record, the database has to lock up the table and generate new ID for that record. I have to let it check across cluster or turn off cross-cluster check, which might cause an conflict. So unless that table don't have much activity and records are inserted in batches or you are not clustering/replicating the database, having a AUTOINCREMENT column in database is not a good idea. Use UUID or having a key generation service is better. I personally use ULID to allow time-based sorting on my database without indexing the

created_at

created_at

column since most of tables are required to be sorted with

created_at

created_at

descending.

LLarry What you are proposing is a sort of two-phase commit which would work but I don'...

John_rand•3/31/24, 4:56 PM

Thank you for your response.

Unfortunately, last_insert_rowid is not thread safe and should be used only in the same transaction. (Its local to the connection actually, but if the connection drops, you're left with tangling rows).
Having a temporary table is equivalent to having the same table with a flag (is_orphan) I think.

As for the guid, you're in the mercy of collisions, maybe equivalent to having a monotonically incremented separate table (sequence).

Unfortunately, it is indeed a 2-phase-commit problem, no solution feels right. Which is why posted the question on how people are handling it.

It is very unfortunate that D1 forces these problems on the consumer and that there is no clear guidance. Is it that much of a corner case wanting to insert an "order" with "order-lines" in a transactional manner?

JJohn_rand Thank you for your response. Unfortunately, last_insert_rowid is not thread saf...

Jason Hostetter•3/31/24, 5:07 PM

Aren’t batch inserts on D1 done in the same transaction? So you could use last_row_id safely? Generating your own uid seems like a good approach, are collisions a realistic problem for your application?

Original message was deleted

John_rand•3/31/24, 5:15 PM

Both correct. Needs very carefully crafted code on the client though. Not sure how I could do that with drizzle (call the function once, store it in a variable, have subsequent batches use that var)

On the collisions, it would affect my case yes, or at least it would help me sleep better at nights. ULID could be much easier option to adopt maybe - and drizzle friendly.

RexNormous•3/31/24, 10:18 PM

D1 updates incoming! https://github.com/cloudflare/cloudflare-docs/commit/0c71ecec03b2ac5e8103aa1e38f67a156ca26fe7

GitHub

[D1] fix some typos (#13367) · cloudflare/cloudflare-docs@0c71ece

d1: fix some typos
workers: Update betas.md
d1: Update pricing.md
d1: Update limits.md
d1: Update time-travel.md
d1: Update debug-d1.md
d1: Update backups.md

RRexNormous D1 updates incoming! https://github.com/cloudflare/cloudflare-docs/commit/0c71ec...

James•3/31/24, 10:26 PM

what do you mean? That PR is just fixing some typos! :Kappa:

James•3/31/24, 10:26 PM

Some super cool stuff tho, yeah

RexNormous•3/31/24, 10:26 PM

10gb max size db, 250gb total per paid account
and foreign key support too https://github.com/cloudflare/cloudflare-docs/pull/13708/files

Rafael França•3/31/24, 10:49 PM

Has anyone already tried to use a multitenant using multiple databases? How to solve the binding of each tenant?

KKai just noticed beta badge is gone, did d1 go GA?

isaiah•3/31/24, 11:32 PM

https://tenor.com/view/obi-wan-kenobi-these-are-not-the-droids-star-wars-jedi-mind-trick-gif-23067514

Tenor

isaiah•3/31/24, 11:32 PM

No, it has always been that way

JJames what do you mean? That PR is just fixing some typos! :Kappa:

isaiah•3/31/24, 11:35 PM

d1-beta channel to d1 channel when?

RRexNormous D1 updates incoming! https://github.com/cloudflare/cloudflare-docs/commit/0c71ec...

Tony Leung•4/1/24, 3:26 AM

I'm waiting for them to let me run

PRAGMA foreign_key=off

PRAGMA foreign_key=off

since the last time I update my table it was a nightmare

RRexNormous D1 updates incoming! https://github.com/cloudflare/cloudflare-docs/commit/0c71ec...

Kai•4/1/24, 7:29 AM

looks at current date hmmm

KKai *looks at current date* hmmm

Max (@rozenmd)•4/1/24, 8:41 AM

my favourite was this PR https://github.com/cloudflare/workers-sdk/pull/744

GitHub

Get out in front of trademark issues with `wrangler jeans` and `wra...

It's no secret that the wrangler 2 release is going to be big. Bigger than big. In fact, I don't think it's too much of a stretch to say it will change the way that software is develope...

RRexNormous 10gb max size db, 250gb total per paid account and foreign key support too https...

Matt Silverlock•4/1/24, 10:59 AM

D1 has always supported foreign keys: they are enabled by default. This is just documenting them so folks don’t have to dive into the (much denser) SQLite docs.

Matt Silverlock•4/1/24, 1:02 PM

@D1 Database - D1 is now GA! Read more about what it means for D1 to be production ready and what we have planned for D1: https://blog.cloudflare.com/making-full-stack-easier-d1-ga-hyperdrive-queues

The Cloudflare Blog

Making state easy with D1 GA, Hyperdrive, Queues and Workers Analyt...

We kick off the week with announcements that help developers build stateful applications on top of Cloudflare, including making D1, our SQL database and Hyperdrive, our database accelerating service, generally available

MMatt Silverlock @D1 Database - D1 is now GA! Read more about what it means for D1 to be producti...

Matt Silverlock•4/1/24, 1:04 PM

We also have a deep dive on the upcoming read replication API (we call it the Sessions API) and how it will work under the hood: https://blog.cloudflare.com/building-d1-a-global-database

The Cloudflare Blog

Building D1: a Global Database

D1, Cloudflare’s SQL database, is now generally available. With new support for 10GB databases, data export, and enhanced query debugging, we empower developers to build production-ready applications with D1 to meet all their relational SQL needs

Jengas•4/1/24, 1:05 PM

Hi everyone! I wonder if 10GB storage per database is extendable?

SageSphinx63920•4/1/24, 1:05 PM

JJengas Hi everyone! I wonder if 10GB storage per database is extendable?

bun•4/1/24, 1:08 PM

They postponed it for future

Mozzy•4/1/24, 1:12 PM

Congrats on GA!

JJengas Hi everyone! I wonder if 10GB storage per database is extendable?

Max (@rozenmd)•4/1/24, 1:16 PM

(my canned reply to this is):

D1 is built to work to the strengths of Cloudflare's network, so we can't offer extremely large databases while also loading them quickly. It's significantly easier for us to offer millions of DBs with a size limit instead.

D1 will require a change in how you think about your project's data if you're looking to build a big platform. Instead of storing all of your data in a single DB, you would store each customer's data in their own DB (and we're building tooling to make it easier to dynamically access multiple DBs in a single Worker without redeploying)

MMatt Silverlock Not yet. We’ve shared a lot here already - it’s something we’re working on and w...

James•4/1/24, 1:22 PM

Any news on dynamic bindings now that we’ve hit GA?

Is this the ‘new APIs’ mentioned in the GA post?

MMax (@rozenmd)(my canned reply to this is): D1 is built to work to the strengths of Cloudflar...

Adi•4/1/24, 1:22 PM

Just a question about future tooling you're building. Are there any plans to build an automatic sharding solution? Or something where we can add a pool of DBs and automatically write / query data from that pool?
I know this kind of opposite to the way you're encouraging us to think about these DBs, but perhaps sharding could be useful still in some cases?

Adi•4/1/24, 1:23 PM

Also, huge congrats on GA! Some of the best news i've heard all month

JJames Any news on dynamic bindings now that we’ve hit GA? 👀 Is this the ‘new APIs’ m...

Vy•4/1/24, 1:26 PM

New API design is for the upcoming read replication.

We're discussing dynamic bindings internally and will have more to share soon

rvg•4/1/24, 1:28 PM

Congrats on the GA! Question on smart placement and d1, is this section of the docs still up to date?

Workers with a D1 binding will always be placed in a data center near the location of the D1 database they are bound to. Subrequests to other back-end services are ignored by Smart Placement in this case.

https://developers.cloudflare.com/workers/configuration/smart-placement/#d1

The blog post briefly mentions smart placement, but from what I gather till the session and read replication APIs are in it is still the case that workers with d1 bindings only live in one location?

Rrvg Congrats on the GA! Question on smart placement and d1, is this section of the d...

Vy•4/1/24, 1:35 PM

D1 today still only has 1 copy in a single location. Once read replication is enabled, replicas can exist in multiple locations.

Smart Placement operates on Workers placement not D1. It runs your Worker closer to a bound D1 database

VVy D1 today still only has 1 copy in a single location. Once read replication is en...

rvg•4/1/24, 1:40 PM

Great! I'd love to add d1 to an app but I was a little hesitant reading the comment that it would push all the deployments to one geo region

As I'd only use d1 in some parts/endpoints, I'd love for the others to be servable from any edge location. Could you maybe comment on if that's possible with service bindings today? There's some conflicting info on the docs last I checked: d1-database

Original message was deleted

Vy•4/1/24, 1:43 PM

Oceania region (Australia) is coming as early as next week

Mas Inis•4/1/24, 1:44 PM

Holy... many great announcements today!

Mas Inis•4/1/24, 1:46 PM

However since we have some deployment regulation restriction...will D1 (and workers) support restricted deployment to specific location only?

Mas Inis•4/1/24, 1:48 PM

I see, time to email Cloudflare support for the enterprice plan

Mas Inis•4/1/24, 1:48 PM

~~enterprice~~ enterprise

Mozzy•4/1/24, 1:48 PM

Are the old beta DBs now actually out of beta?

MMozzy Are the old beta DBs now actually out of beta?

Isaac McFadyen•4/1/24, 1:49 PM

Yes, beta -> production now.

Isaac McFadyen•4/1/24, 1:49 PM

If you update wrangler you should see those change.

MMas Inis However since we have some deployment regulation restriction...will D1 (and work...

Vy•4/1/24, 1:50 PM

We are evaluating future D1 jurisdiction support, https://developers.cloudflare.com/durable-objects/reference/data-location/#restrict-durable-objects-to-a-jurisdiction, so share your needs!

Cloudflare Docs

Data location · Cloudflare Durable Objects docs

You can restrict a Durable Object to a jurisdiction, or provide a location hint.

IIsaac McFadyen Yes, beta -> production now.

Mozzy•4/1/24, 1:52 PM

Nice. Updating wrangler still shows beta though

Mas Inis•4/1/24, 1:53 PM

frankly... location jurisdiction is the biggest thing that obstruct us adopting CF dev platforms, shackled with AWS or GCP :NotLikeThis:

especially if you have financial sector clients.

Chief•4/1/24, 1:54 PM

Is the row size limit still 1MB?

MMozzy Nice. Updating wrangler still shows beta though

Isaac McFadyen•4/1/24, 1:55 PM

It's possible it's not released yet, I just saw the upcoming Pull Request at some point.

Vy•4/1/24, 1:58 PM

some in-flight changes to remove remaining "beta" mentions, doesn't affect our production stance

MMax (@rozenmd)(my canned reply to this is): D1 is built to work to the strengths of Cloudflar...

Coalission•4/1/24, 2:31 PM

In a multi tenant setup like this, will D1 have parent child schemas, or will schemas have to be updated one by one?

tadeumaia•4/1/24, 3:06 PM

D1 going GA means it's coming soon to south-america? This is whats making me stay at turso for now.

Fernando Dilland•4/1/24, 4:24 PM

With the new features of D1, do you think it is possible to host a WordPress?

What you are proposing is a sort of two-phase commit which would work but I don't think it's necessa

Similar Threads