Do you have a link for info on developer week? I can’t find anything about it.
Do you have a link for info on developer week? I can’t find anything about it.
LYou can patch the Worker config through the REST API when you create a new database instead of having to put them manually in wrangler. It's a pain though since you need a bit of custom code on your side to do that. If you really want to handle thousands of databases, I would say that Durable Objects SQLite are much more suitable at the moment. The 1GB limit it only until it goes GA, soon.
LD1 databases are based on SQLite Durable Objects. If a Durable Object SQLite database could grow to 1TB then cold starts would be prohibitively slow. We could do lazy loading and all that jazz, but then you lose the super fast performance of local SQLite queries. 1-10GB is a good-enough size to enable many use-cases and OK cold starts.
TI'm thinking about living with my single 10G DB and hoping something changes with the cloudflare offering in the next 6 months. Either dynamic binding to databases in wrangler / workers or a larger single DB limit. If Durable Objects + SQLite were the future of scaling SQLite in the cloudflare stack I could start experimenting with that now but without an knowing that's the case I'd probably sit on my hands.
LIndividual DBs will not grow larger anytime soon, at least not any plans currently.
TThanks for the perspective on that! I can live with a 10GB database for the near term. I'll just need to figure out a plan to shard data should my application be successful and grow (or move to a new provider). I'm sure I'm not the only one. The more developer content that gets produced about scaling D1 or Durable Objects + SQLite the more retention you guys will drive. If I missed something LMK.
LDurable Objects can scale to great extend, as long as you design for sharding from the get go. It takes a bit of a mindshift since all traditional tutorials and online content is focused on big monolithic databases. Depending on the application doing sharding could be easier or harder, so it really depends how far you want to go.
The blog posts in this section https://developers.cloudflare.com/durable-objects/what-are-durable-objects/#durable-objects-in-cloudflare describe how we use DOs in several products, so that you get some idea of how to think with them.
At the end of the day, if you don't want or cannot think in terms of sharding, Hyperdrive is an awesome approach too, so just use that with your traditional database provider. Different tradeoffs in each approach.
The blog posts in this section https://developers.cloudflare.com/durable-objects/what-are-durable-objects/#durable-objects-in-cloudflare describe how we use DOs in several products, so that you get some idea of how to think with them.
At the end of the day, if you don't want or cannot think in terms of sharding, Hyperdrive is an awesome approach too, so just use that with your traditional database provider. Different tradeoffs in each approach.
TI was planning on just sharding by customer account. For my application, I'd be shocked if customers get more than 1G of data. That's just an easy starting place. ALL DB instances look the same.
For hyperdrive, my stack right now is hono + drizzeORM + cloudflare workers with D1 and R2. I'd have to look at how to work. hyperdrive into it and how much of a rewrite that owuld be.
For hyperdrive, my stack right now is hono + drizzeORM + cloudflare workers with D1 and R2. I'd have to look at how to work. hyperdrive into it and how much of a rewrite that owuld be.
OWill D1 Dynamic Bindings essentially be a wrapper around SQLite for DO? Or will it have extra bells and whistles like tooling with the same performance as SQLite for DOs, or same as current performance. (performance as in latencies)
Dthanks. this is useful info
Dgoing to deploy a single db on d1 for now
if it gets out of hand, i guess turso is the next stop
if it gets out of hand, i guess turso is the next stop
LThe SQL query performance itself is identical between D1 and SQLite DOs. The only difference is that D1 is in a different location from your Worker code location so doing multiple D1 roundtrips can be costly depending on those locations. With DOs you could colocate code inside the DO and then you always have fast access. If you are calling your DO from a different worker for each query then performance is again identical between the two products.
D1 REST API is completely separately flow, and should not be used in hot paths of your application on Workers.
D1 REST API is completely separately flow, and should not be used in hot paths of your application on Workers.
OThank you. Best performance will always come directly from a Durable Object with the SQL backed storage - I'm still pondering on building around it.
The only thing left to wonder is what kind of management tooling D1 dynamic bindings will have? (bulk DB deletion, reading/modifying multiple DBs, listing DBs?)
The only thing left to wonder is what kind of management tooling D1 dynamic bindings will have? (bulk DB deletion, reading/modifying multiple DBs, listing DBs?)
LI wouldn't design my architecture based on future features. Maybe Dynamic Bindings come with those niceties you mention, and maybe they come to DOs as well, or maybe it's going to be very different.
If you want to ship your app now, I suggest thinking with the existing tradeoffs.
If you want to ship your app now, I suggest thinking with the existing tradeoffs.
OI guess i can always migrate. Thanks again!
Ahttps://www.cloudflare.com/developer-week/
it still has last years' articles up, they usually update it to the current year a few days before it starts iirc
it still has last years' articles up, they usually update it to the current year a few days before it starts iirc
Cloudflare Developer Week April 1-5, our week-long series of new product announcements and events dedicated to enhancing the developer experience to fuel productivity!
CIs there a cost different on sqllite in do vs using d1 and any other limiting factors?
CIm working on building a discord bot, with d1 as a backend, but considering speed is important with an interactive bot kind of tempted to switch. Especially as all commands come from minimally changing Data centers
ISQLite in DO is limited to 1GB per instantiated DO at the moment.
IThat limit should increase to 10GB (same as D1) once it's out of beta, though.
JYou pay wall time on DO. And expensive computations in sql will cost you cpu time plus R/W in DO. The upside is you can do fancy stored procedure type app logic on the same thread as your DB with DO.
F@Isaac McFadyen
Jyou've probably already considered this, but on the cost side: after you've burned through the free tier, D1 storage is also 3.75x more expensive than SQLite in DO
Shello, is there something going on ?
LThere isn't anything service wide. What queries are you doing? That seems like a scan to me. You already have weird spikes before for queries taking 500ms.
Shey @lambrospetrou thx for this, I am investigating, it went down already but that's odd it's like 4.5 sec !
XIs it possible to create a read only endpoint for D1? For dashboard stats and such for instance. It looks like via the rest API I could possibly just use a
D1 Read
HYou could just expose a Worker with some pre-determined routes that only ever perform selects?
XI want the queries to be driven by the UI à la BI tools like Periscope and Tableau
MAs per docs D1 has TypeScript support for representing rows like:
What if the database has snake case column names, but I want to keep camel case;
is there a simple (automatic) to convert his?
e.g. like Drizzle supports with
If not wouldn't that a nice feature request?
What if the database has snake case column names, but I want to keep camel case;
is there a simple (automatic) to convert his?
e.g. like Drizzle supports with
const db = drizzle({ connection: process.env.DATABASE_URL, casing: 'snake_case' })If not wouldn't that a nice feature request?
M(I'm trying to avoid writing a lot of mappers)
M
LNo, out of the box we don't do that name conversion for you with the current D1 binding.
MIs it worth a feature request? (like the Drizzle example)
BThis is probably something better done in user land.
NHi folks! We are looking to use D1 for a multi tenant application. We are looking to ways to keep d1 databases isolated at a tenant level. Right now the only limitation I see is that bindings are static, is there any way to dynamically bind to my d1 instances? Or any other recommendation on how to achieve this?
IProbably not, D1 is SQLite and SQLite doesn't perform any inspection of the query itself because it's intended as a raw SQL executor. Adding this behavior would require adding non-standard parsing in front of SQLite.
XIt's what all these BI tools let you do
GIs there a place I can accuratly see how many reads/writes I have done in a billing period? The billing section says I have done 0 which is very much not true, although if you want to leave it that way for calculating billing I'm ok with it 
Sno, they don't. they generate SQL from sanitized user input, they don't use raw input as the query
VHey folks, what is D1's current limit for requests per second for paid accounts? I found this response but not sure if it applies. d1-database
VI'm working on a ticket service with high surge potential and I need to let my customers know.
IThat applies, yup.
IIf you need more performance/capacity you'll need to shard across multiple DBs.
Mis there a recommenday way to get better ux on the d1. new to cf and sqlite. the webgui is very basic. is there any recomendation?
do i just get local replicate and even then what is a good viewer for the db
do i just get local replicate and even then what is a good viewer for the db
LYou can use https://www.outerbase.com/blog/now-supporting-cloudflare-d1/ although I haven't personally tested it, but saw happy users before.
Mnot bad was very easy to setup and much better ux. should be ok for now thx.
Sam i the only one that just sqlite3s the db file inside .wrangler? 
BThe docs say this:
But AFAICT this guidance is purely theoretical as there's no way to actually do the sharding?
Each D1 database can store up to 10 GB of data, and you can create up to thousands of separate D1 databases. This allows you to split a single monolithic database into multiple, smaller databases, thereby isolating application data by user, customer, or tenant
But AFAICT this guidance is purely theoretical as there's no way to actually do the sharding?
BIs there something obvious I'm missing?
