Cloudflare Developers•2y ago

Hi everyone! I wonder if 10GB storage per database is extendable?

SageSphinx63920•4/1/24, 1:05 PM

JJengas Hi everyone! I wonder if 10GB storage per database is extendable?

bun•4/1/24, 1:08 PM

They postponed it for future

Mozzy•4/1/24, 1:12 PM

Congrats on GA!

JJengas Hi everyone! I wonder if 10GB storage per database is extendable?

Max (@rozenmd)•4/1/24, 1:16 PM

(my canned reply to this is):

D1 is built to work to the strengths of Cloudflare's network, so we can't offer extremely large databases while also loading them quickly. It's significantly easier for us to offer millions of DBs with a size limit instead.

D1 will require a change in how you think about your project's data if you're looking to build a big platform. Instead of storing all of your data in a single DB, you would store each customer's data in their own DB (and we're building tooling to make it easier to dynamically access multiple DBs in a single Worker without redeploying)

MMatt Silverlock Not yet. We’ve shared a lot here already - it’s something we’re working on and w...

James•4/1/24, 1:22 PM

Any news on dynamic bindings now that we’ve hit GA?

Is this the ‘new APIs’ mentioned in the GA post?

MMax (@rozenmd)(my canned reply to this is): D1 is built to work to the strengths of Cloudflar...

Adi•4/1/24, 1:22 PM

Just a question about future tooling you're building. Are there any plans to build an automatic sharding solution? Or something where we can add a pool of DBs and automatically write / query data from that pool?
I know this kind of opposite to the way you're encouraging us to think about these DBs, but perhaps sharding could be useful still in some cases?

Adi•4/1/24, 1:23 PM

Also, huge congrats on GA! Some of the best news i've heard all month

JJames Any news on dynamic bindings now that we’ve hit GA? 👀 Is this the ‘new APIs’ m...

Vy•4/1/24, 1:26 PM

New API design is for the upcoming read replication.

We're discussing dynamic bindings internally and will have more to share soon

rvg•4/1/24, 1:28 PM

Congrats on the GA! Question on smart placement and d1, is this section of the docs still up to date?

Workers with a D1 binding will always be placed in a data center near the location of the D1 database they are bound to. Subrequests to other back-end services are ignored by Smart Placement in this case.

https://developers.cloudflare.com/workers/configuration/smart-placement/#d1

The blog post briefly mentions smart placement, but from what I gather till the session and read replication APIs are in it is still the case that workers with d1 bindings only live in one location?

Rrvg Congrats on the GA! Question on smart placement and d1, is this section of the d...

Vy•4/1/24, 1:35 PM

D1 today still only has 1 copy in a single location. Once read replication is enabled, replicas can exist in multiple locations.

Smart Placement operates on Workers placement not D1. It runs your Worker closer to a bound D1 database

VVy D1 today still only has 1 copy in a single location. Once read replication is en...

rvg•4/1/24, 1:40 PM

Great! I'd love to add d1 to an app but I was a little hesitant reading the comment that it would push all the deployments to one geo region

As I'd only use d1 in some parts/endpoints, I'd love for the others to be servable from any edge location. Could you maybe comment on if that's possible with service bindings today? There's some conflicting info on the docs last I checked: d1-database

Original message was deleted

Vy•4/1/24, 1:43 PM

Oceania region (Australia) is coming as early as next week

Mas Inis•4/1/24, 1:44 PM

Holy... many great announcements today!

Mas Inis•4/1/24, 1:46 PM

However since we have some deployment regulation restriction...will D1 (and workers) support restricted deployment to specific location only?

Mas Inis•4/1/24, 1:48 PM

I see, time to email Cloudflare support for the enterprice plan

Mas Inis•4/1/24, 1:48 PM

~~enterprice~~ enterprise

Mozzy•4/1/24, 1:48 PM

Are the old beta DBs now actually out of beta?

MMozzy Are the old beta DBs now actually out of beta?

Isaac McFadyen•4/1/24, 1:49 PM

Yes, beta -> production now.

Isaac McFadyen•4/1/24, 1:49 PM

If you update wrangler you should see those change.

MMas Inis However since we have some deployment regulation restriction...will D1 (and work...

Vy•4/1/24, 1:50 PM

We are evaluating future D1 jurisdiction support, https://developers.cloudflare.com/durable-objects/reference/data-location/#restrict-durable-objects-to-a-jurisdiction, so share your needs!

Cloudflare Docs

Data location · Cloudflare Durable Objects docs

You can restrict a Durable Object to a jurisdiction, or provide a location hint.

IIsaac McFadyen Yes, beta -> production now.

Mozzy•4/1/24, 1:52 PM

Nice. Updating wrangler still shows beta though

Mas Inis•4/1/24, 1:53 PM

frankly... location jurisdiction is the biggest thing that obstruct us adopting CF dev platforms, shackled with AWS or GCP :NotLikeThis:

especially if you have financial sector clients.

Chief•4/1/24, 1:54 PM

Is the row size limit still 1MB?

MMozzy Nice. Updating wrangler still shows beta though

Isaac McFadyen•4/1/24, 1:55 PM

It's possible it's not released yet, I just saw the upcoming Pull Request at some point.

Vy•4/1/24, 1:58 PM

some in-flight changes to remove remaining "beta" mentions, doesn't affect our production stance

MMax (@rozenmd)(my canned reply to this is): D1 is built to work to the strengths of Cloudflar...

Coalission•4/1/24, 2:31 PM

In a multi tenant setup like this, will D1 have parent child schemas, or will schemas have to be updated one by one?

tadeumaia•4/1/24, 3:06 PM

D1 going GA means it's coming soon to south-america? This is whats making me stay at turso for now.

Fernando Dilland•4/1/24, 4:24 PM

With the new features of D1, do you think it is possible to host a WordPress?

FFernando Dilland With the new features of D1, do you think it is possible to host a WordPress? 🤔

Hello, I’m Allie!•4/1/24, 4:25 PM

No, since Workers/Pages doesn't support PHP

Hello, I’m Allie!•4/1/24, 4:25 PM

There was someone who built a PHP demo, but I highly doubt that it is stable enough to run the entirety of WP

Hello, I’m Allie!•4/1/24, 4:26 PM

Plus you would probably want something officially supported

Isaac McFadyen•4/1/24, 4:28 PM

And even if it was viable to run Wordpress, each plugin hooks in custom PHP code so adding any plugins would likely break things unless you specifically adapted it for each plugin

Taylor•4/1/24, 4:52 PM

So, how will Cloudflare D1 tackle the write lag issue common in distributed DBs?

TTaylor So, how will Cloudflare D1 tackle the write lag issue common in distributed DBs?

Chaika•4/1/24, 4:57 PM

Did you check out the blog post released today?
https://blog.cloudflare.com/building-d1-a-global-database

The Cloudflare Blog

Building D1: a Global Database

D1, Cloudflare’s SQL database, is now generally available. With new support for 10GB databases, data export, and enhanced query debugging, we empower developers to build production-ready applications with D1 to meet all their relational SQL needs

Taylor•4/1/24, 5:04 PM

It seems sessions solves the data consistency portion of the issue but how does that improve write latency? Is the write latency just negligible that the delay won't really be too noticeable by the user? '

TTaylor It seems sessions solves the data consistency portion of the issue but how does ...

Vy•4/1/24, 5:07 PM

write latency will be influenced by how long the primary D1 takes to serve a write request. Read replicas asynchronoulsy replicate from the primary, which doesn't impact write latency

VVy New API design is for the upcoming read replication. We're discussing dynamic b...

Rob M.•4/1/24, 6:04 PM

Will it be possible to create a D1 instance or check D1's storage capacity from the worker? One db per customer sounds great but is useless in many cases without on-the-fly management.

RRob M.Will it be possible to create a D1 instance or check D1's storage capacity from ...

rob•4/1/24, 6:06 PM

yeah was curious also about that, with having to deploy the worker for bindng the new customer d1 instances

RRob M.Will it be possible to create a D1 instance or check D1's storage capacity from ...

zwily•4/1/24, 6:25 PM

see d1-database

Zzwily see https://discord.com/channels/595317990191398933/992060581832032316/122434670...

rob•4/1/24, 6:29 PM

(and we're building tooling to make it easier to dynamically access multiple DBs in a single Worker without redeploying) yea that part, that important

RRob M.Will it be possible to create a D1 instance or check D1's storage capacity from ...

Vy•4/1/24, 6:29 PM

something we're designing and hope to share more soon

RRob M.Will it be possible to create a D1 instance or check D1's storage capacity from ...

Max (@rozenmd)•4/1/24, 6:43 PM

without on-the-fly management.

specifically what are you looking for?

Original message was deleted

Vy•4/1/24, 6:58 PM

that's our target

João Castro•4/1/24, 7:45 PM

Really excited for D1, but at the same time it sounds weird that its out of beta when many of what sounds like core features are missing. Read replicas, transactions, dynamic bindings/management, regions like South America...

JJoão Castro Really excited for D1, but at the same time it sounds weird that its out of beta...

Vy•4/1/24, 7:52 PM

for some folks, its tricky to use a "beta" database for production workloads, even if they already fit well with D1 capabilities. D1 GA was centered on hitting reliability/DevEx with the product as is. D1 has many more features planned, which will enable even more workloads.

haochenx•4/1/24, 8:25 PM

Congrats on GA. When will real transaction support be available? Am I correct in assuming transaction will be available to local D1 on Durable Objects once that’s shipped?

Compared to eg postgres, since SQLite does not support stored procedures one has to have transaction for certain queries (the current batched transaction does not support arbitrary data dependency between queries in the batch, unless you write sub queries, but that is a lot of time troubling or too difficult in SQL)

Or maybe rolling out CF’s custom support for stored function might be another viable route? (Something like an Durable Object, but for D1 DB and each method can be used as SQLite function in a query)

MMax (@rozenmd)> without on-the-fly management. specifically what are you looking for? 👀

Rob M.•4/1/24, 10:13 PM

Like the ability to create a new D1 instance from the worker upon new user registration. Then this instance should be available to all other necessary workers without redeployment.

Another use case is when a worker writes to D1 instance until it's full, then creates another one and writes to it.

Hi everyone! I wonder if 10GB storage per database is extendable?

Similar Threads