Hello has anyone tested the d1 driver for datagrip ? I can't manage to make partial selects with i

Hello has anyone tested the d1 driver for datagrip ?

I can't manage to make partial selects with it

The second request returns the whole user, while the first one fails at parsing

(I'm using this driver https://github.com/isaac-mcfadyen/d1-jdbc-driver/tree/main)

Miki•3/30/24, 12:55 PM

SSoftbear Hello has anyone tested the d1 driver for datagrip ? I can't manage to make pa...

SoftbearOP•3/30/24, 2:50 PM

Looks like a known issue, too bad it's not at all maintained
https://github.com/isaac-mcfadyen/d1-jdbc-driver/issues/3

GitHub

Select anything but * seems to fail · Issue #3 · isaac-mcfadyen/d1-...

where mytable has a primary key of mykey the follow queries error: select distinct field from mytable select field from mytable They succeed in the CF D1 console. I tested 3 different tools, just t...

albyzyx•3/30/24, 2:56 PM

Hello, New to cloudflare ecosystem. I am looking for the right stack for my project. Is D1 stable enough to use on production?

D4nk•3/30/24, 3:46 PM

Anyone know whyt my D1 binding now only has a "Fetcher" property? I can use prepare anymore

D4nk•3/30/24, 4:05 PM

I have been at this for a day now, why is my D1 suddenly a Fetcher I have everything setup correct

shahzaibkhalid•3/30/24, 4:13 PM

Hello, does someone know if it's safe to commit database_iddatabase_id and preview_database_idpreview_database_id in

wrangler.toml

wrangler.toml

file?

Sshahzaibkhalid Hello, does someone know if it's safe to commit `database_id` and `preview_datab...

Cyb3r-Jak3•3/30/24, 4:14 PM

Yes.

Cyb3r-Jak3•3/30/24, 4:14 PM

Everything in the wrangler.toml file is safe to commit. Unless you have sensitive environment variables which you should use secrets for.

CCyb3r-Jak3 Everything in the wrangler.toml file is safe to commit. Unless you have sensitiv...

shahzaibkhalid•3/30/24, 4:15 PM

Thank you!!

Original message was deleted

Isaac McFadyen•3/30/24, 7:18 PM

Yeah I'm not really maintaining that library any more unfortunately. At the time I made it I had a lot of other things going on and couldn't really find the time and then never got back around to it.

Isaac McFadyen•3/30/24, 7:18 PM

I should probably either find the time or archive it

Isaac McFadyen•3/30/24, 7:18 PM

Thanks for the reminder.

RexNormous•3/31/24, 3:08 AM

Does anyone know if making requests to a D1 DB count toward the 50/1000 fetches per worker invocation limit?

JJohn_rand Hi folks, I was wondering how people deal with D1 not supporting transactions on...

Larry•3/31/24, 12:22 PM

What you are proposing is a sort of two-phase commit which would work but I don't think it's necessary. I can think of two alternatives. sqlite (and I assume D1) has a "function" last_insert_rowid()last_insert_rowid() https://www.sqlite.org/c3ref/last_insert_rowid.html. That would allow you to insert one child with the parent. If you really want to insert more than one, then do the child inserts into a temporary table and delete as you move over to the real child table. I'm sure you could find an example of this on Stack Overflow. The second alternative would be to use a GUID that you generate client-side for your primary key. That doesn't take advantage of SQLite's really cool auto-increment rowid capability but that's really a nice-to-have.

Isaac McFadyen•3/31/24, 2:46 PM

In general the SQLite project actually recommends staying away from autoincrement on rowids because it imposes extra CPU/memory constraints, and recommends either leaving it as default (which will auto increment but also reuse the IDs of deleted rows) or using a UUID.

https://www.sqlite.org/autoinc.html

The AUTOINCREMENT keyword imposes extra CPU, memory, disk space, and disk I/O overhead and should be avoided if not strictly needed. It is usually not needed.

Isaac McFadyen•3/31/24, 2:47 PM

AUTOINCREMENT literally just prevents reuse of row IDs from deleted rows, so if you're fine reusing row IDs then just don't specify it.

Tony Leung•3/31/24, 3:47 PM

hi guys, will there be a feature which allows foreign key check to be turned off while migrating database? I have an issue here: https://github.com/cloudflare/workers-sdk/issues/5438

GitHub

🚀 Feature Request: Allow turning off foreign key check while migrat...

Some table altering tasks in SQLite cannot be achieved without replacing it with a new table (by dropping - recreating the table) (like adding foreign keys, changing primary keys, updating column t...

IIsaac McFadyen AUTOINCREMENT literally just prevents reuse of row IDs from deleted rows, so if ...

Tony Leung•3/31/24, 3:52 PM

In my experience, having AUTOINCREMENT in database will make a huge performance impact on database cluster. Everytime I insert a new record, the database has to lock up the table and generate new ID for that record. I have to let it check across cluster or turn off cross-cluster check, which might cause an conflict. So unless that table don't have much activity and records are inserted in batches or you are not clustering/replicating the database, having a AUTOINCREMENT column in database is not a good idea. Use UUID or having a key generation service is better. I personally use ULID to allow time-based sorting on my database without indexing the

created_at

created_at

column since most of tables are required to be sorted with

created_at

created_at

descending.

LLarry What you are proposing is a sort of two-phase commit which would work but I don'...

John_rand•3/31/24, 4:56 PM

Thank you for your response.

Unfortunately, last_insert_rowid is not thread safe and should be used only in the same transaction. (Its local to the connection actually, but if the connection drops, you're left with tangling rows).
Having a temporary table is equivalent to having the same table with a flag (is_orphan) I think.

As for the guid, you're in the mercy of collisions, maybe equivalent to having a monotonically incremented separate table (sequence).

Unfortunately, it is indeed a 2-phase-commit problem, no solution feels right. Which is why posted the question on how people are handling it.

It is very unfortunate that D1 forces these problems on the consumer and that there is no clear guidance. Is it that much of a corner case wanting to insert an "order" with "order-lines" in a transactional manner?

JJohn_rand Thank you for your response. Unfortunately, last_insert_rowid is not thread saf...

Jason Hostetter•3/31/24, 5:07 PM

Aren’t batch inserts on D1 done in the same transaction? So you could use last_row_id safely? Generating your own uid seems like a good approach, are collisions a realistic problem for your application?

Original message was deleted

John_rand•3/31/24, 5:15 PM

Both correct. Needs very carefully crafted code on the client though. Not sure how I could do that with drizzle (call the function once, store it in a variable, have subsequent batches use that var)

On the collisions, it would affect my case yes, or at least it would help me sleep better at nights. ULID could be much easier option to adopt maybe - and drizzle friendly.

RexNormous•3/31/24, 10:18 PM

D1 updates incoming! https://github.com/cloudflare/cloudflare-docs/commit/0c71ecec03b2ac5e8103aa1e38f67a156ca26fe7

GitHub

[D1] fix some typos (#13367) · cloudflare/cloudflare-docs@0c71ece

d1: fix some typos
workers: Update betas.md
d1: Update pricing.md
d1: Update limits.md
d1: Update time-travel.md
d1: Update debug-d1.md
d1: Update backups.md

RRexNormous D1 updates incoming! https://github.com/cloudflare/cloudflare-docs/commit/0c71ec...

James•3/31/24, 10:26 PM

what do you mean? That PR is just fixing some typos! :Kappa:

James•3/31/24, 10:26 PM

Some super cool stuff tho, yeah

RexNormous•3/31/24, 10:26 PM

10gb max size db, 250gb total per paid account
and foreign key support too https://github.com/cloudflare/cloudflare-docs/pull/13708/files

Rafael França•3/31/24, 10:49 PM

Has anyone already tried to use a multitenant using multiple databases? How to solve the binding of each tenant?

KKai just noticed beta badge is gone, did d1 go GA?

isaiah•3/31/24, 11:32 PM

https://tenor.com/view/obi-wan-kenobi-these-are-not-the-droids-star-wars-jedi-mind-trick-gif-23067514

Tenor

isaiah•3/31/24, 11:32 PM

No, it has always been that way

JJames what do you mean? That PR is just fixing some typos! :Kappa:

isaiah•3/31/24, 11:35 PM

d1-beta channel to d1 channel when?

RRexNormous D1 updates incoming! https://github.com/cloudflare/cloudflare-docs/commit/0c71ec...

Tony Leung•4/1/24, 3:26 AM

I'm waiting for them to let me run PRAGMA foreign_key=offPRAGMA foreign_key=off since the last time I update my table it was a nightmare

RRexNormous D1 updates incoming! https://github.com/cloudflare/cloudflare-docs/commit/0c71ec...

Kai•4/1/24, 7:29 AM

looks at current date hmmm

KKai *looks at current date* hmmm

Max (@rozenmd)•4/1/24, 8:41 AM

my favourite was this PR https://github.com/cloudflare/workers-sdk/pull/744

GitHub

Get out in front of trademark issues with `wrangler jeans` and `wra...

It's no secret that the wrangler 2 release is going to be big. Bigger than big. In fact, I don't think it's too much of a stretch to say it will change the way that software is develope...

RRexNormous 10gb max size db, 250gb total per paid account and foreign key support too https...

Matt Silverlock•4/1/24, 10:59 AM

D1 has always supported foreign keys: they are enabled by default. This is just documenting them so folks don’t have to dive into the (much denser) SQLite docs.

Matt Silverlock•4/1/24, 1:02 PM

@D1 Database - D1 is now GA! Read more about what it means for D1 to be production ready and what we have planned for D1: https://blog.cloudflare.com/making-full-stack-easier-d1-ga-hyperdrive-queues

The Cloudflare Blog

Making state easy with D1 GA, Hyperdrive, Queues and Workers Analyt...

We kick off the week with announcements that help developers build stateful applications on top of Cloudflare, including making D1, our SQL database and Hyperdrive, our database accelerating service, generally available

MMatt Silverlock @D1 Database - D1 is now GA! Read more about what it means for D1 to be producti...

Matt Silverlock•4/1/24, 1:04 PM

We also have a deep dive on the upcoming read replication API (we call it the Sessions API) and how it will work under the hood: https://blog.cloudflare.com/building-d1-a-global-database

The Cloudflare Blog

Building D1: a Global Database

D1, Cloudflare’s SQL database, is now generally available. With new support for 10GB databases, data export, and enhanced query debugging, we empower developers to build production-ready applications with D1 to meet all their relational SQL needs

Jengas•4/1/24, 1:05 PM

Hi everyone! I wonder if 10GB storage per database is extendable?

SageSphinx63920•4/1/24, 1:05 PM

JJengas Hi everyone! I wonder if 10GB storage per database is extendable?

bun•4/1/24, 1:08 PM

They postponed it for future

Mozzy•4/1/24, 1:12 PM

Congrats on GA!

JJengas Hi everyone! I wonder if 10GB storage per database is extendable?

Max (@rozenmd)•4/1/24, 1:16 PM

(my canned reply to this is):

D1 is built to work to the strengths of Cloudflare's network, so we can't offer extremely large databases while also loading them quickly. It's significantly easier for us to offer millions of DBs with a size limit instead.

D1 will require a change in how you think about your project's data if you're looking to build a big platform. Instead of storing all of your data in a single DB, you would store each customer's data in their own DB (and we're building tooling to make it easier to dynamically access multiple DBs in a single Worker without redeploying)

MMatt Silverlock Not yet. We’ve shared a lot here already - it’s something we’re working on and w...

James•4/1/24, 1:22 PM

Any news on dynamic bindings now that we’ve hit GA?

Is this the ‘new APIs’ mentioned in the GA post?

MMax (@rozenmd)(my canned reply to this is): D1 is built to work to the strengths of Cloudflar...

Adi•4/1/24, 1:22 PM

Just a question about future tooling you're building. Are there any plans to build an automatic sharding solution? Or something where we can add a pool of DBs and automatically write / query data from that pool?
I know this kind of opposite to the way you're encouraging us to think about these DBs, but perhaps sharding could be useful still in some cases?

Adi•4/1/24, 1:23 PM

Also, huge congrats on GA! Some of the best news i've heard all month

JJames Any news on dynamic bindings now that we’ve hit GA? 👀 Is this the ‘new APIs’ m...

Vy•4/1/24, 1:26 PM

New API design is for the upcoming read replication.

We're discussing dynamic bindings internally and will have more to share soon

rvg•4/1/24, 1:28 PM

Congrats on the GA! Question on smart placement and d1, is this section of the docs still up to date?

Workers with a D1 binding will always be placed in a data center near the location of the D1 database they are bound to. Subrequests to other back-end services are ignored by Smart Placement in this case.

https://developers.cloudflare.com/workers/configuration/smart-placement/#d1

The blog post briefly mentions smart placement, but from what I gather till the session and read replication APIs are in it is still the case that workers with d1 bindings only live in one location?

Rrvg Congrats on the GA! Question on smart placement and d1, is this section of the d...

Vy•4/1/24, 1:35 PM

D1 today still only has 1 copy in a single location. Once read replication is enabled, replicas can exist in multiple locations.

Smart Placement operates on Workers placement not D1. It runs your Worker closer to a bound D1 database

VVy D1 today still only has 1 copy in a single location. Once read replication is en...

rvg•4/1/24, 1:40 PM

Great! I'd love to add d1 to an app but I was a little hesitant reading the comment that it would push all the deployments to one geo region

As I'd only use d1 in some parts/endpoints, I'd love for the others to be servable from any edge location. Could you maybe comment on if that's possible with service bindings today? There's some conflicting info on the docs last I checked: d1-database

Original message was deleted

Vy•4/1/24, 1:43 PM

Oceania region (Australia) is coming as early as next week

Mas Inis•4/1/24, 1:44 PM

Holy... many great announcements today!

Mas Inis•4/1/24, 1:46 PM

However since we have some deployment regulation restriction...will D1 (and workers) support restricted deployment to specific location only?

Hello has anyone tested the d1 driver for datagrip ? I can't manage to make partial selects with i

Similar Threads