Keep in mind that each Dynamo replica adds to your storage costs + adds additional replication costs

Keep in mind that each Dynamo replica adds to your storage costs + adds additional replication costs. D1 will not for the 'default' global replication.

Matt SilverlockOP•7/7/23, 2:33 PM

(I like a lot of what Dynamo did, but replication stacks up fast: https://aws.amazon.com/dynamodb/pricing/on-demand/)

Matt SilverlockOP•7/7/23, 2:37 PM

https://silverlock-d1-remix-svelte-e.cloudflare-docs-7ou.pages.dev/d1/examples/

what framework should we add next? The Pages docs cover the whole list, but (IMO) having specific examples you can copy+paste to get started for D1 == easier

Unsmart•7/7/23, 2:39 PM

I assume D1 will only be single region writer? Or will it be like Dynamo where you can write to the primary node in each region?

UUnsmart I assume D1 will only be single region writer? Or will it be like Dynamo where y...

Matt SilverlockOP•7/7/23, 2:43 PM

Single writer: we think (and others like PlanetScale & Google's Spanner team also share this view) that multiple writers introduces too many consistency issues for user-facing apps. DynamoDB makes a bunch of trade-offs (and they are welcome to!) that we don't think are the right ones for user-facing apps / cause you to have to think about consistency a lot in your app design.

MMatt Silverlock https://silverlock-d1-remix-svelte-e.cloudflare-docs-7ou.pages.dev/d1/examples/ ...

vassbence•7/7/23, 3:08 PM

Next.js definitely

Vvassbence Next.js definitely

Matt SilverlockOP•7/7/23, 3:09 PM

https://github.com/timothymiller/t4-app

GitHub

GitHub - timothymiller/t4-app: Type-Safe, Full-Stack Starter Kit fo...

Type-Safe, Full-Stack Starter Kit for React Native + Web. - GitHub - timothymiller/t4-app: Type-Safe, Full-Stack Starter Kit for React Native + Web.

vassbence•7/7/23, 3:09 PM

In terms of scale do you think you will have an upper cap or it's planned to support even terrabytes in db size?

Vvassbence In terms of scale do you think you will have an upper cap or it's planned to sup...

Matt SilverlockOP•7/7/23, 3:10 PM

I don't think single databases with terabytes of storage are likely, but I also don't think they make sense elsewhere. A Postgres database or MySQL database with > 100GB will start to see scaling and query performance issues. If you need a single DB with terabytes of storage there's likely an application architecture issue.

Colin•7/7/23, 3:15 PM

Colin

Colin•7/7/23, 3:23 PM

Sorry if this has been asked before.. I am new to a discord and to cloudflare. The documentation says it is not stable and thus@not ready for production. What other options are available for a cloudflare-distributed web-app that requires a basic database?

CColin Sorry if this has been asked before.. I am new to a discord and to cloudflare. ...

Matt SilverlockOP•7/7/23, 3:34 PM

https://developers.cloudflare.com/workers/databases/connecting-to-databases/

MMatt Silverlock https://github.com/timothymiller/t4-app 🙂

James•7/7/23, 3:43 PM

from the looks of things, that repo only interacts with D1 in a worker script, not Next.js, so I would probably avoid using that as an example for Next.js personally

JJames from the looks of things, that repo only interacts with D1 in a worker script, n...

Matt SilverlockOP•7/7/23, 3:48 PM

https://developers.cloudflare.com/pages/framework-guides/deploy-a-nextjs-site/#use-bindings-in-your-nextjs-application is the generic example, that applies to KV/R2/D1

Matt SilverlockOP•7/7/23, 3:48 PM

This isn't to say we won't add a pointed D1 example, but there is a Next.js example

MMatt Silverlock https://developers.cloudflare.com/pages/framework-guides/deploy-a-nextjs-site/#u...

James•7/7/23, 3:52 PM

I know - I was the one who updated that

I'm just saying that the example you linked to the person who said Next.js doesn't actually use it in Next.js

MMatt Silverlock This is _definitely_ not related to Chaika’s issue. I think you have a typo in y...

Chaika•7/7/23, 3:53 PM

Did you ever get a chance to check into the broken d1 db 972558fd-cadf-4bec-8b0b-5a140d7d51fb? I don't mean to bother you, not critical, I'm duplicating all my writes/etc to Planetscale for this purpose, but it's still dead as a doornail, seems to go between the states of being completely dead/unreachable (Gateway timeouts) and instant d1 errors D1_ERROR: Error: Internal error while starting up D1 storage caused object to be resetD1_ERROR: Error: Internal error while starting up D1 storage caused object to be reset, weird to me since it's just all of a sudden it died, I'm doing a lot of batched writes/updates to it, maybe maxed out some invisible limit, it's on the experimental backend

CChaika Did you ever get a chance to check into the broken d1 db 972558fd-cadf-4bec-8b0b...

Matt SilverlockOP•7/7/23, 3:55 PM

We're still looking: you've hit what looks like a very rare edge case. Been debugging it since yesterday — we need to cut a release to get more info from the writer flow. Will let you know once we see it fixed

Chaika•7/7/23, 3:57 PM

Thanks! Just my luck lol, I suppose it's good to hit these errors now rather then later though

MMatt Silverlock I don't think single databases with terabytes of storage are likely, but I also ...

Hello, I’m Allie!•7/7/23, 4:17 PM

I’m wondering, for large applications with data in the TB range, would sharding work? And if so, would there be a way to shard natively with D1, or is that something we need to handle ourselves?

HHello, I’m Allie!I’m wondering, for large applications with data in the TB range, would sharding ...

Matt SilverlockOP•7/7/23, 4:27 PM

This is definitely something we're thinking about. Being able to shard across multiple DBs based on some form of shardKeyshardKey (and how to make that easy for users).

HHello, I’m Allie!I’m wondering, for large applications with data in the TB range, would sharding ...

Matt SilverlockOP•7/7/23, 4:34 PM

At the same time, if you have a single DB in the TB range... there are many, many other considerations. You're probably running more OLAP type workloads, storing large blobs, etc.

ajgeiss0702•7/7/23, 6:57 PM

before we start getting charged, will we be able to see metrics for read/write units used?

ajgeiss0702•7/7/23, 6:58 PM

nice. Any idea when thats happening? or just some time before pricing starts

ajgeiss0702•7/7/23, 6:59 PM

ok, thank you

vassbence•7/7/23, 8:35 PM

What's the best way to cache responses from D1? The Cache API?

It seems kinda sub-par, since AFAIK it only caches in the a local PoP. It would be nice to go at least regional

vassbence•7/7/23, 8:43 PM

KV is the best option in the cloudflare arsenal for it?

Vvassbence KV is the best option in the cloudflare arsenal for it?

Max (@rozenmd)•7/8/23, 5:48 AM

until we start handling the regional read replicas for you, probably yeah

Original message was deleted

Max (@rozenmd)•7/8/23, 5:12 PM

can do

manstfu•7/8/23, 6:28 PM

@rozenmd is this still not working?
https://github.com/cloudflare/workers-sdk/issues/3160

GitHub

🐛 BUG: when trying to left join 2 tables that have same name column...

Which Cloudflare product(s) does this pertain to? D1 What version of Wrangler are you using? 2.19.0 What operating system are you using? macOS Describe the Bug As per issue #555 we can see, that d1...

manstfu•7/8/23, 6:28 PM

we need it

Mmanstfu @rozenmd is this still not working? https://github.com/cloudflare/workers-sdk/is...

Cyb3r-Jak3•7/8/23, 6:40 PM

If the issue is still open then I would assume it hasn't been fixed

manstfu•7/8/23, 6:41 PM

just a kind reminder

CharstarAdmin•7/9/23, 2:51 AM

Any timetable on when the 1GB storage will be available?

Mmanstfu @rozenmd is this still not working? https://github.com/cloudflare/workers-sdk/is...

Adi•7/9/23, 8:07 AM

+1 I'm also keen for this to be fixed. It has flow on effects in drizzle-orm, joins just don't work correctly for any queries run against D1.

AAdi +1 I'm also keen for this to be fixed. It has flow on effects in drizzle-orm, jo...

manstfu•7/9/23, 8:40 AM

I can send you a temporary fix if you are stuck with it

Mmanstfu I can send you a temporary fix if you are stuck with it

Adi•7/9/23, 8:44 AM

Nah that's ok, I've worked around it by not using joins for now. But it's definitely something I have to actively work around, so keen for a fix from the Cloudflare team.

manstfu•7/9/23, 8:45 AM

Mmanstfu just a kind reminder🙏

Max (@rozenmd)•7/9/23, 9:51 AM

Please don't do this.

Semigroup•7/9/23, 11:58 AM

It looks like TS bindings don't align with actual types when calling 'all' from a prepared statement

The 'results' field has a type of T[][], when actually it's just T[], so I'm forced to use ts-expect-error. Using version 4.20230628.0 of workers types, but it was the case at least since 4.20230321.0 (that's when I started using D1). Also, I'm using experimental backend for D1

Am I doing something wrong? If not, how do I report it properly (e.g. github issue) so it can be tracked?

SSemigroup It looks like TS bindings don't align with actual types when calling 'all' from ...

itsmatteomanf•7/9/23, 8:00 PM

It seems like it’s passing in Out[] as the result type, which is not what you are passing… have you tried restarting Typescript or the editor?

AAdi Nah that's ok, I've worked around it by not using joins for now. But it's defini...

Unsmart•7/9/23, 8:16 PM

From reading the issue it seems like they are probably just going to make that throw an error instead of returning data at all and you should be using ASAS to rename duplicated columns https://github.com/cloudflare/workerd/pull/696#issuecomment-1561891627

vassbence•7/9/23, 8:51 PM

Durable Objects run in a ton of regions, as for the D1 read replicas will they match these regions or run in fewer ones? DO give the best performance as of now as they run the closest and was just wondering if this will be the same in the future. Guess it's kinda overkill to run 8 read replicas for a db.

Unsmart•7/9/23, 9:07 PM

D1 is a durable object so it will be the same regions but how many it will be in no clue. My guess is probably 3 maybe 4

Unsmart•7/9/23, 9:09 PM

Durable object’s performance really depend though because if you have 1 object used by many users it won’t be close to all of them as they aren’t replicated each object only exists in one location

vassbence•7/9/23, 9:25 PM

I have one DO per user and this is why I was wondering about D1/DO speed

vassbence•7/9/23, 9:26 PM

as with my current setup DO is really close to the user and so everything is snappy. D1 will be single-writer as stated by the team so the write speeds will be always faster for DO, but I guess reads will be at least as good or better for DOs too.

vassbence•7/9/23, 9:27 PM

It's just that I'd love to write SQL instead of DOs, lot easier for other developers in the team too.

vassbence•7/9/23, 9:28 PM

And so which one does one choose, a few hundered ms faster writes, and a couple ten ms faster reads for the end users or easier developer experience

Keep in mind that each Dynamo replica adds to your storage costs + adds additional replication costs

Similar Threads

Similar Threads

Similar Threads