> Are all the limits except the DB size limit available to be potentially raised? You'd need to be

Are all the limits except the DB size limit available to be potentially raised?

You'd need to be specific about the individual limit in question, but generally they aren't raised on a per-user basis and instead are global limits.

is there some way to dynamically set up bindings

Not currently.

Ravi kiran•11/28/23, 5:36 AM

guys can anyone tell me how i can use two d1 databases in a single worker?
i tried this in the toml file
name = "weathered-queen-4070"
main = "src/index.ts"
compatibility_date = "2023-10-30"

[[d1_databases]]
binding = "DB" # available in your Worker on env.DB
database_name = "dogpark"
database_id = "xxxxxxxxxxxx"

binding = "DB2" # i.e. available in your Worker on env.DB
database_name = "adminportalDB"
database_id = "xxxxxxxxxxx"

but then i get the error that Can't redefine existing key

/Users/apple/Documents/weathered-queen-4070/wrangler.toml:10:14:
10 │ binding = "DB2"

kianOP•11/28/23, 5:37 AM

name = "weathered-queen-4070"
main = "src/index.ts"
compatibility_date = "2023-10-30"

[[d1_databases]]
binding = "DB" # available in your Worker on env.DB
database_name = "dogpark"
database_id = "xxxxxxxxxxxx"

[[d1_databases]]
binding = "DB2" # i.e. available in your Worker on env.DB
database_name = "adminportalDB"
database_id = "xxxxxxxxxxx"

name = "weathered-queen-4070"
main = "src/index.ts"
compatibility_date = "2023-10-30"

[[d1_databases]]
binding = "DB" # available in your Worker on env.DB
database_name = "dogpark"
database_id = "xxxxxxxxxxxx"

[[d1_databases]]
binding = "DB2" # i.e. available in your Worker on env.DB
database_name = "adminportalDB"
database_id = "xxxxxxxxxxx"

Ravi kiran•11/28/23, 5:38 AM

oh thank you.

rodrigo•11/28/23, 8:55 AM

I understand when read replicas are implemented there would be one master where all writes take place, right?
Is there any plans to have multiple write masters, based on table partition (for instance)?
Imagine a User table, where users write their own data. Would it be possible to partition this table based on location?
I mean, to create a partition of this table for each location (america, europe, asia) that would become master for writes, and this way a user would write as master to the partition in its location (and then read replica would replicate it to other locations)

VI•11/28/23, 9:48 AM

Since d1 uses SQLite internally, does it run into the same limitations as SQLite, namely handling large number of independent writes/sec. From what I read SQLite is not capable of handling more than 1000 write reqs/sec.
https://www.sqlite.org/whentouse.html

Thanks for any help!

Rrodrigo I understand when read replicas are implemented there would be one master where ...

Max (@rozenmd)•11/28/23, 8:12 PM

So far the plan is just for a single writer

VVI Since d1 uses SQLite internally, does it run into the same limitations as SQLite...

Vy•11/28/23, 8:32 PM

Currently for open beta, ~1K QPS depending on result set size. Our eventual goal is to get 3-10K QPS for reads, 1-2K QPS for writes. Let us know what requirements your workload would need?

Kkian > Are all the limits except the DB size limit available to be potentially raised...

Village (Josef)•11/29/23, 3:06 AM

Sounds good, thank you! And I meant storage per account and number of DBs specifically

janat08•11/29/23, 8:51 AM

Can you promise at this point that the D1 size limitations will begone

janat08•11/29/23, 8:52 AM

You recommend against building on D1 for production, but surely not because you still have forseeable hard limits on db size?

Jjanat08 You recommend against building on D1 for production, but surely not because you ...

Hello, I’m Allie!•11/29/23, 8:54 AM

I don't think it is ever going to be unlimited, but it is definitely going to be higher than it is now

janat08•11/29/23, 9:01 AM

Do you get to shard it or something once at the ceiling. You mean I won't be able to have a petabyte db on cf, but my question is if at some point I'd have to close registration at my app because of the storage limit?

Jjanat08 Do you get to shard it or something once at the ceiling. You mean I won't be abl...

Hello, I’m Allie!•11/29/23, 9:04 AM

I think it is also a case of understanding what platforms are built for. While it would be nice theoretically if D1 could scale infinitely, in practice, if you do get to Petabyte scale, you should probably look at a DB specfically built for that. Until then though, D1 is a good choice

janat08•11/29/23, 9:22 AM

Is there going to be some library for sharding, where you can have multiple d1 dbs, and get to treat them one, so as to bypass the size limitation at the expense of extra reads

Max (@rozenmd)•11/29/23, 9:28 AM

we have folks building products that have 1 db per customer, where each db has the same schema, just stores different data
though migrations + dynamically binding to new dbs are a pain point

MMax (@rozenmd)we have folks building products that have 1 db per customer, where each db has t...

Hello, I’m Allie!•11/29/23, 9:33 AM

Where's my SQLite DOs?

janat08•11/29/23, 9:34 AM

I'm not interested in 1db per customer thing, as I'd like them to be able to interact with each other

janat08•11/29/23, 9:34 AM

is there some kind of workaround, or limitations improvement to make d1 fit for general usage?

Jjanat08 is there some kind of workaround, or limitations improvement to make d1 fit for ...

Max (@rozenmd)•11/29/23, 9:40 AM

It doesn't sound like D1 fits your use case if you're worried about petabyte scale. Hyperdrive is probably a better fit: https://developers.cloudflare.com/hyperdrive/

janat08•11/29/23, 10:38 AM

I have no reason to worry about petabyte scyle, but surely anyone is hitting the 2gb limit

kianOP•11/29/23, 11:04 AM

This is a beta-only limitation. The D1 team will increase the maximum storage per-database, storage per-account and number of databases for Paid plans during the course of D1’s public beta.

kianOP•11/29/23, 11:05 AM

They're beta limitations - there is no forecasted hard limit, but there will naturally be one even in GA.

janat08•11/29/23, 12:13 PM

612345 wtyu wq\vcb

mushang98•11/29/23, 12:30 PM

I'm sorry for asking. I'm a newbie around cloudflare workers and pages. I've been trying to use next.js with D1 with no luck. Is there any template or example code and configs for this ?

kianOP•11/29/23, 12:31 PM

https://github.com/Rei-x/next-on-cloudflare-example & https://github.com/james-elicx/cloudy are both NextJS and use D1

Mmushang98 I'm sorry for asking. I'm a newbie around cloudflare workers and pages. I've bee...

Matt Silverlock•11/29/23, 12:31 PM

And https://developers.cloudflare.com/pages/framework-guides/deploy-a-nextjs-site/ shows how to use KV bindings (which is the same pattern D1 uses)

Deploy a Next.js site · Cloudflare Pages docs

Next.js is an open-source React framework for creating websites and applications. In this guide, you will create a new Next.js application and deploy …

Kkian <https://github.com/Rei-x/next-on-cloudflare-example> & <https://github.com/jame...

mushang98•11/29/23, 12:34 PM

Thank you, I'll look into it

MMatt Silverlock And https://developers.cloudflare.com/pages/framework-guides/deploy-a-nextjs-sit...

mushang98•11/29/23, 12:34 PM

I've tried this, but got error with the "env" variable. Any suggestion ?

kianOP•11/29/23, 12:36 PM

I'd recommend asking in #next-on-pages if you continue to have issues with

env

env

or bindings in general - once you have

env

env

working then D1 will just work as you'd expect normally.

Kkian I'd recommend asking in #next-on-pages if you continue to have issues with `env`...

mushang98•11/29/23, 12:38 PM

Okay, but I'll try something with the example first. thanks for the help

talfiz•11/29/23, 4:11 PM

hi guys I have a table with teacher_image VARBINARY(1048576) NOT NULL, every time I insert a binary into this table the whole database collapse I cant interact with the database anymore someone know why?

Ttalfiz hi guys I have a table with teacher_image VARBINARY(1048576) NOT NULL, every tim...

Matt Silverlock•11/29/23, 4:14 PM

What errors are you seeing?

talfiz•11/29/23, 4:17 PM

after successfuly inserting into the table (I can see the values in the UI) if I try to edit / insert from the ui or with code i get the error FOREIGN KEY constraint failed (Code: 7500) , and If I try to delete the whole table from the UI Its telling me to try again later (its the same for two days)

talfiz•11/29/23, 4:19 PM

I believe its a bug

Max (@rozenmd)•11/29/23, 4:21 PM

the dashboard doesn't support editing binary

talfiz•11/29/23, 4:21 PM

I dont try to edit the binary I try to edit a completly diffrent table

talfiz•11/29/23, 4:22 PM

Its the same message for all the tables

talfiz•11/29/23, 4:22 PM

Even tables with no binary

Mmushang98 I've tried this, but got error with the "env" variable. Any suggestion ?

Rob M.•11/29/23, 4:41 PM

Can you send a snippet with your "env" usage?

Rob M.•11/29/23, 4:44 PM

By the way, you should use 'context.env.*' instead of 'env.*' in Page workers. Maybe that's the issue.

talfiz•11/29/23, 5:56 PM

Hi guys I fixed the bug what I tried to do is to delete a user when a teacher is still refrenceing him to do that I should add ON DELETE CASCADE to the table thanks for however tried to help

MMax (@rozenmd)So far the plan is just for a single writer

rodrigo•11/30/23, 9:57 AM

You talk below about cases where people create 1db per user.. But to be more efficient, if I want to create 1 db per region (enam, wnam...) and redirect each user to the closest/fastest db, is there a way to programatically do that? with the

CF-IPCountry

CF-IPCountry

header ? but should I build a table with states on the east/west cost ... and redirect based on it? or is there a way to get a ping to the D1 center location?

rodrigo•11/30/23, 9:59 AM

And regarding Asia-Pacific region .. that is a region too big (Australia, China , India ...). Is there (or will there) a way to allocate D1 dbs to smaller areas within the region?

Rrodrigo You talk below about cases where people create 1db per user.. But to be more eff...

Rob M.•11/30/23, 10:18 AM

I'm not sure about cloudflare, but you can use any IP database for that like this one:
https://www.maxmind.com/en/geoip-databases

MaxMind GeoIP® Databases | MaxMind

IP geolocation and network data from the creator of GeoIP. Download and host
the data locally to eliminate network latency and per-query charges.

Rrodrigo You talk below about cases where people create 1db per user.. But to be more eff...

mrwind•11/30/23, 11:03 AM

request on worker will include request.cf.continent and request.cf.country . maybe you can use it and map to d1 region.

Rrodrigo You talk below about cases where people create 1db per user.. But to be more eff...

Max (@rozenmd)•11/30/23, 12:08 PM

the Workers platform is built in such a way that you should never need to make decisions like that - we know where your requests are coming from and should route them to the nearest replica for you, without your input (except for cases where you've opted into keeping data in certain regions)

Max (@rozenmd)•11/30/23, 12:09 PM

though I understand it's annoying to have slower requests atm while replication hasn't been shipped yet

MMax (@rozenmd)the Workers platform is built in such a way that you should never need to make d...

rodrigo•11/30/23, 2:10 PM

I was talking more about the case where you want to have several D1 dbs so that you have several writers. In this case, there is a way to redirect users to its closer db writer ? (example I want Australian users to use a master D1 db in Australia instead of having ig generating anywhere in the Asia-Pacific region)

Rrodrigo I was talking more about the case where you want to have several D1 dbs so that ...

Max (@rozenmd)•11/30/23, 2:20 PM

then yeah, request.cfrequest.cf has the data you need: https://developers.cloudflare.com/workers/runtime-apis/request/#incomingrequestcfproperties

Max (@rozenmd)•11/30/23, 2:53 PM

...yet

name = "weathered-queen-4070" main = "src/index.ts" compatibility_date = "2023-10-30" [[d1_databases]] binding = "DB" # available in your Worker on env.DB database_name = "dogpark" database_id = "xxxxxxxxxxxx" [[d1_databases]] binding = "DB2" # i.e. available in your Worker on env.DB database_name = "adminportalDB" database_id = "xxxxxxxxxxx"

> Are all the limits except the DB size limit available to be potentially raised? You'd need to be

Similar Threads

Similar Threads

Similar Threads