Hi, sorry if this has been asked before, but how are you guys gonna work with this limit, if your se

Hi, sorry if this has been asked before, but how are you guys gonna work with this limit, if your service explodes in popularity?
Maximum database size 10 GB (Workers Paid)
https://developers.cloudflare.com/d1/platform/limits/

Cloudflare Docs

Limits · Cloudflare D1 docs

If you would like to explore other storage solutions for your application, Cloudflare also offers Workers KV, Durable Objects, and R2.

GGunnar Hi, sorry if this has been asked before, but how are you guys gonna work with th...

bun•4/11/24, 10:38 AM

10gb is plenty enough for a lot of users

bun•4/11/24, 10:38 AM

They recommend using multiple databases in any way

bun•4/11/24, 10:39 AM

Like theres this possibility of having a db per customer even

GGunnar Hi, sorry if this has been asked before, but how are you guys gonna work with th...

Max (@rozenmd)•4/11/24, 10:44 AM

(my canned reply to this is):

D1 is built to work to the strengths of Cloudflare's network, so we can't offer extremely large databases while also loading them quickly. It's significantly easier for us to offer millions of DBs with a size limit instead.

D1 will require a change in how you think about your project's data if you're looking to build a big platform. Instead of storing all of your data in a single DB, you would store each customer's data in their own DB (and we're building tooling to make it easier to dynamically access multiple DBs in a single Worker without redeploying)

MMax (@rozenmd)(my canned reply to this is): D1 is built to work to the strengths of Cloudflar...

Hello, I’m Allie!•4/11/24, 10:48 AM

One thing I've been wondering, if the main issue to having large databases is startup speed, what effect does a large database have on execution speed? As in, if you could somehow assume that the DB is always hot, would the upper limit for storage be much higher?

MMax (@rozenmd)(my canned reply to this is): D1 is built to work to the strengths of Cloudflar...

forslunds•4/11/24, 10:51 AM

in a single worker.. not a single pages function?

Fforslunds in a single worker.. not a single pages function? 🥲

Hello, I’m Allie!•4/11/24, 10:52 AM

All of the Functions in a Pages Project are bundled into a Worker, so it should apply to Functions too

HHello, I’m Allie!All of the Functions in a Pages Project are bundled into a Worker, so it should ...

forslunds•4/11/24, 10:55 AM

if i got a cent for every time i heard that..

HHello, I’m Allie!All of the Functions in a Pages Project are bundled into a Worker, so it should ...

Max (@rozenmd)•4/11/24, 10:59 AM

this, but I always worked around Pages functions not having all the features by using a separate Worker for my API

Workers and Pages becoming the same "thing" will make this better

GGunnar Hi, sorry if this has been asked before, but how are you guys gonna work with th...

Beny•4/11/24, 11:03 AM

If your service gets to that size then you shouldn't be using sqlite (which is what D1 is) anyway.
I have one app with 8000+ users and over 11 million records in the db and it is under 2gb, so I dont think it's an issue unless you are storing some big data in there

MMax (@rozenmd)(my canned reply to this is): D1 is built to work to the strengths of Cloudflar...

GunnarOP•4/11/24, 11:04 AM

Is there a max db count limit?

GGunnar Is there a max db count limit?

Beny•4/11/24, 11:05 AM

It's 50,000 but they've mentioned they've just set that so people don't go crazy. It can be increased to anything.
https://developers.cloudflare.com/d1/platform/limits/

Cloudflare Docs

Limits · Cloudflare D1 docs

If you would like to explore other storage solutions for your application, Cloudflare also offers Workers KV, Durable Objects, and R2.

BBeny It's 50,000 but they've mentioned they've just set that so people don't go crazy...

GunnarOP•4/11/24, 11:07 AM

Oh jesus, sorry, I'm blind, it's the top line

GGunnar Is there a max db count limit?

Max (@rozenmd)•4/11/24, 11:08 AM

when we release dynamic bindings, it'll look a bit more like DO's limits page

Beny•4/11/24, 11:09 AM

Next we need an unlimited KV's limit

That one is pretty small

BBeny Next we need an unlimited KV's limit 😄 That one is pretty small

GunnarOP•4/11/24, 11:11 AM

There is a "Limit Increase Request Form" for KV
https://developers.cloudflare.com/kv/platform/limits/

Cloudflare Docs

Limits · Cloudflare Workers KV

Request adjustments to limits that conflict with your project goals by contacting Cloudflare. To request an increase to a limit, complete the Limit …

BBeny Next we need an unlimited KV's limit 😄 That one is pretty small

Hello, I’m Allie!•4/11/24, 11:12 AM

You can also just create new “namespaces” by giving each a prefix

HHello, I’m Allie!You can also just create new “namespaces” by giving each a prefix

Beny•4/11/24, 11:14 AM

Yeah this is a thing, but that workaround wouldn't be good in combination with Workers for Platforms, otherwise the user workers will be able to see other's data

GGunnar There is a "Limit Increase Request Form" for KV https://developers.cloudflare.co...

Beny•4/11/24, 11:18 AM

I've never tried the limit increase before but I've seen some people here mention their requests went unanswered for weeks so it puts me off a bit (and it is strange hosting it on a Google form, doesn't feel official)

Oom Can d1 sizes be increased more then 10gb?

Max (@rozenmd)•4/11/24, 11:42 AM

no, see conversation above: d1-database

Oom I get it. It's awesome. But how much can a 10gb store. If it's json row?

Max (@rozenmd)•4/11/24, 11:44 AM

millions of rows: d1-database

Max (@rozenmd)•4/11/24, 11:45 AM

figure out how big your rows are, divide 10GB by that size, that's how many rows you can store

Oom And if i separate database so much. Like if have blog and it's exceeding 10gb. T...

Max (@rozenmd)•4/11/24, 11:47 AM

the classic challenge is figuring out the right way to split the data, not always easy

Oom Is there way can I invoke a worker or http request if the data in the db changes...

Max (@rozenmd)•4/11/24, 11:47 AM

not without polling the db, not yet anyway

Oom And if i separate database so much. Like if have blog and it's exceeding 10gb. T...

Max (@rozenmd)•4/11/24, 11:54 AM

say you have a blog with 10kb of text (~1k words) per article, that's a million articles for 10GB

ZZiga Zajc I have json string stored in D1. What function should be used to return object b...

Max (@rozenmd)•4/11/24, 12:34 PM

this doc might be useful https://developers.cloudflare.com/d1/build-with-d1/query-json/

BBeny Yeah this is a thing, but that workaround wouldn't be good in combination with W...

Hello, I’m Allie!•4/11/24, 12:44 PM

Could you pass in an RPC target that makes reads only to their “namespace”?

ZZiga Zajc ```sql SELECT uuid, json_array(titles) as titles, year FROM anime; ```

Max (@rozenmd)•4/11/24, 12:46 PM

wouldn't you want to do something like:
json_extract(titles, $.romanji) as romanji, json_extract(titles, $.english) as english, ...json_extract(titles, $.romanji) as romanji, json_extract(titles, $.english) as english, ... ?

Max (@rozenmd)•4/11/24, 12:47 PM

not sure if sqlite supports the idea of nesting objects in a response

Hi, sorry if this has been asked before, but how are you guys gonna work with this limit, if your se

Similar Threads

Similar Threads

Similar Threads