Cloudflare Developers•2y ago

So, how will Cloudflare D1 tackle the write lag issue common in distributed DBs?

TTaylor So, how will Cloudflare D1 tackle the write lag issue common in distributed DBs?

Chaika•4/1/24, 4:57 PM

Did you check out the blog post released today?
https://blog.cloudflare.com/building-d1-a-global-database

The Cloudflare Blog

Building D1: a Global Database

D1, Cloudflare’s SQL database, is now generally available. With new support for 10GB databases, data export, and enhanced query debugging, we empower developers to build production-ready applications with D1 to meet all their relational SQL needs

TaylorOP•4/1/24, 5:04 PM

It seems sessions solves the data consistency portion of the issue but how does that improve write latency? Is the write latency just negligible that the delay won't really be too noticeable by the user? '

TTaylor It seems sessions solves the data consistency portion of the issue but how does ...

Vy•4/1/24, 5:07 PM

write latency will be influenced by how long the primary D1 takes to serve a write request. Read replicas asynchronoulsy replicate from the primary, which doesn't impact write latency

VVy New API design is for the upcoming read replication. We're discussing dynamic b...

Rob M.•4/1/24, 6:04 PM

Will it be possible to create a D1 instance or check D1's storage capacity from the worker? One db per customer sounds great but is useless in many cases without on-the-fly management.

RRob M.Will it be possible to create a D1 instance or check D1's storage capacity from ...

rob•4/1/24, 6:06 PM

yeah was curious also about that, with having to deploy the worker for bindng the new customer d1 instances

RRob M.Will it be possible to create a D1 instance or check D1's storage capacity from ...

zwily•4/1/24, 6:25 PM

see d1-database

Zzwily see https://discord.com/channels/595317990191398933/992060581832032316/122434670...

rob•4/1/24, 6:29 PM

(and we're building tooling to make it easier to dynamically access multiple DBs in a single Worker without redeploying) yea that part, that important

RRob M.Will it be possible to create a D1 instance or check D1's storage capacity from ...

Vy•4/1/24, 6:29 PM

something we're designing and hope to share more soon

RRob M.Will it be possible to create a D1 instance or check D1's storage capacity from ...

Max (@rozenmd)•4/1/24, 6:43 PM

without on-the-fly management.

specifically what are you looking for?

Original message was deleted

Vy•4/1/24, 6:58 PM

that's our target

João Castro•4/1/24, 7:45 PM

Really excited for D1, but at the same time it sounds weird that its out of beta when many of what sounds like core features are missing. Read replicas, transactions, dynamic bindings/management, regions like South America...

JJoão Castro Really excited for D1, but at the same time it sounds weird that its out of beta...

Vy•4/1/24, 7:52 PM

for some folks, its tricky to use a "beta" database for production workloads, even if they already fit well with D1 capabilities. D1 GA was centered on hitting reliability/DevEx with the product as is. D1 has many more features planned, which will enable even more workloads.

haochenx•4/1/24, 8:25 PM

Congrats on GA. When will real transaction support be available? Am I correct in assuming transaction will be available to local D1 on Durable Objects once that’s shipped?

Compared to eg postgres, since SQLite does not support stored procedures one has to have transaction for certain queries (the current batched transaction does not support arbitrary data dependency between queries in the batch, unless you write sub queries, but that is a lot of time troubling or too difficult in SQL)

Or maybe rolling out CF’s custom support for stored function might be another viable route? (Something like an Durable Object, but for D1 DB and each method can be used as SQLite function in a query)

MMax (@rozenmd)> without on-the-fly management. specifically what are you looking for? 👀

Rob M.•4/1/24, 10:13 PM

Like the ability to create a new D1 instance from the worker upon new user registration. Then this instance should be available to all other necessary workers without redeployment.

Another use case is when a worker writes to D1 instance until it's full, then creates another one and writes to it.

Rob M.•4/1/24, 10:48 PM

Btw, how to upgrade current D1s to the 10GB storage?

RRob M.Btw, how to upgrade current D1s to the 10GB storage?

Matt Silverlock•4/1/24, 11:21 PM

No need to make any changes. If you have a Paid plan it just works.

matt•4/1/24, 11:28 PM

it's dev christmas

matt•4/1/24, 11:29 PM

excited for read replication, will be able to move my upstash redis workloads over and get proper primary composite keys

Rubi•4/2/24, 12:16 AM

It's ok to have all the data on a single DB right?

JJohn_rand Both correct. Needs very carefully crafted code on the client though. Not sure h...

Larry•4/2/24, 12:17 AM

I've recently started using ULIDs as my favorite GUID. Worries about collisions are all in your mind. The probability is so low with any commonly used GUID, that they aren't worth considering.

Original message was deleted

Vy•4/2/24, 12:26 AM

Prisma ORM release/announcement coming tomorrow

isaiah•4/2/24, 2:41 AM

I was waiting for Prisma to support D1 since last year.

isaiah•4/2/24, 2:41 AM

https://tenor.com/view/shocked-happy-surprised-chris-pratt-parks-and-rec-gif-5362501

Tenor

Revan•4/2/24, 4:41 AM

guys where is the d1-beta channel gone ???

RRevan guys where is the d1-beta channel gone ???

Tony Leung•4/2/24, 5:01 AM

d1 is out of beta, now it's #d1-database channel

Michał Wróblewski•4/2/24, 9:35 AM

50,000 instances are hard limit or can be increased by support? I'm thinking of one db/user to 'physically' separate user data but in my case I'll probably never need more than 300k databases

MMichał Wróblewski 50,000 instances are hard limit or can be increased by support? I'm thinking of ...

forslunds•4/2/24, 10:52 AM

Soft limit/unlimited-but-then-you-wont-believe-us is the talk in discord

Fforslunds Soft limit/unlimited-but-then-you-wont-believe-us is the talk in discord 👍🏼

Max (@rozenmd)•4/2/24, 11:31 AM

It's from our open beta blog:

Support for more than 100,000+ databases for Workers for Platforms users — there’s no limit, but if we said “unlimited” you might not believe us — on top of the 50,000 databases per account that D1 already enables.

MMax (@rozenmd)It's from our open beta blog: > Support for more than 100,000+ databases for Wor...

forslunds•4/2/24, 12:42 PM

lets just hope "workers for platforms" works on pages too once i get 50k paying users

MMichał Wróblewski 50,000 instances are hard limit or can be increased by support? I'm thinking of ...

Tony Leung•4/2/24, 12:46 PM

You can sharding your database by username or date of birth or registered month, btw 300k+ db is a crazy large amount of db to manage

Fforslunds lets just hope "workers for platforms" works on pages too once i get 50k paying ...

Tony Leung•4/2/24, 12:48 PM

It would be nice if they add logpush support for pages function, i think that's more important than

Fforslunds lets just hope "workers for platforms" works on pages too once i get 50k paying ...

rob•4/2/24, 12:49 PM

Yea was wondering if they are going to do pages for platforms or something similar for hosting your users frontend pages

forslunds•4/2/24, 12:50 PM

Pages and workers are going to be unified, how much and when we dont know, but its coming

Rrob Yea was wondering if they are going to do pages for platforms or something simil...

Tony Leung•4/2/24, 12:52 PM

I'm out of the loop, what is worker/pages for platform?

rob•4/2/24, 12:53 PM

https://developers.cloudflare.com/cloudflare-for-platforms/workers-for-platforms/get-started/configuration/

Cloudflare Docs

Configure Workers for Platforms · Cloudflare for Platforms docs

This guide will instruct you on setting up Workers for Platforms. You will configure a dispatch namespace, a dynamic dispatch Worker and a user Worker …

rob•4/2/24, 12:53 PM

Pages for platforms doesn't exist afaik

Max (@rozenmd)•4/2/24, 12:55 PM

Workers for Platforms + Pages + SSL for SaaS covers a lot of potential "Pages for Platforms" use cases tbh

Max (@rozenmd)•4/2/24, 12:56 PM

like if you offered some sort of hosted blog service, you'd have a single Pages app connected to a DB and use SSL for SaaS to configure the custom domains, maybe Workers for Platforms for custom logic on top

rob•4/2/24, 12:57 PM

ya I use SSL for sass now on my domain so customers can point their custom domains

LilHuyyyy•4/2/24, 1:07 PM

Hello guys i just wondering whichh orm lib you guys are using with d1?

rob•4/2/24, 1:16 PM

drizzle

LilHuyyyy•4/2/24, 1:27 PM

I use drizzle too but i like some things like typeorm :v

rob•4/2/24, 1:32 PM

tbh I've been usin "nuxt hub" which has all the cloudflare stuff baked in and it was super easy to setup https://hub.nuxt.com/docs/getting-started

Bernardo•4/2/24, 2:04 PM

Morning guys, is it possible to create a multi tenant database with D1? Do you have some article or demo on github with this approach? Thanks!

Titan•4/2/24, 2:12 PM

is there anyway to perform spatial queries in D1? Considering a switch from Postgres for a new project but worried I won't be able to perform a common query: Find a row that is closest to the user's positionFind a row that is closest to the user's position

Tony Leung•4/2/24, 2:36 PM

Oh i get it, it's like aws outpost

Matt Silverlock•4/2/24, 2:43 PM

With the initial support for Python + Workers (https://blog.cloudflare.com/python-workers), we've put together a brief example on how to query D1 from Python: https://developers.cloudflare.com/d1/examples/query-d1-from-python-workers/

Cloudflare Docs

Query D1 from Python Workers · Cloudflare D1 docs

The Cloudflare Workers platform supports multiple languages, including TypeScript, JavaScript, Rust and Python.

So, how will Cloudflare D1 tackle the write lag issue common in distributed DBs?

Similar Threads

Similar Threads

Similar Threads