I think people expect D1 to behave more than Turso** where DB read and writes are made only to a loc

I think people expect D1 to behave more than Turso where DB read and writes are made only to a local db edge instance, which is asynchronously later sent to the primary region. This keeps writes and reads fast for the worker as it doesn't have to wait on the primary region (outside of cold starts sync).

Client applications can connect directly to a replica for read and write operations, but any writes are automatically forwarded to the primary.

Client applications can connect directly to a replica for read and write operations, but any writes are automatically forwarded to the primary.

https://docs.turso.tech/concepts

Concepts | Turso

Concepts for understanding how Turso works.

Iitsmatteomanf D1 is on the edge, it’s generated closest to where it’s generated. Read-replicas...

Paul•10/21/23, 11:36 AM

Ok, so you’re saying the main database, which is the single source of truth, exists in the location closest to where it was created?

JJonathan I think people expect D1 to behave more than Turso** where DB read and writes ar...

Paul•10/21/23, 11:37 AM

Yes this is what I expected! Will this be how D1 works once read replicas are live?

Iitsmatteomanf It will still work, it’s just not distributed. It’s a beta after all…

Paul•10/21/23, 11:38 AM

I get that it’s beta, but I wasn’t expecting the main value prop to be missing at the moment. I mean… I could have kept my Postgres database and just added caching, there’s no difference right?

Iitsmatteomanf It does seem weird it’s not written anywhere replicas don’t exist… 🤔

Paul•10/21/23, 11:39 AM

Yea, total gotcha. I’m bummed

JJonathan I think people expect D1 to behave more than Turso** where DB read and writes ar...

itsmatteomanf•10/21/23, 12:12 PM

As far as I know that's never been what the premise was. Single source of truth, edge, close to the first creation location + read-replicas spread around the world (not sure how this works, as I'm not on the team as you know).

PPaul Ok, so you’re saying the main database, which is the single source of truth, exi...

itsmatteomanf•10/21/23, 12:12 PM

Yes, and it always was planned and explained that way. Write will always go there.

PPaul I get that it’s beta, but I wasn’t expecting the main value prop to be missing a...

itsmatteomanf•10/21/23, 12:13 PM

You can still add caching, that is still true... and will always be reccomended, you pay per row read and written, so the less activity, the less it costs. And the Cache API on Workers is free.

itsmatteomanf•10/21/23, 12:15 PM

As far as read-replicas, I'm 100% sure it was said here multiple times, by many. I always just assumed it's in the docs somewhere, but I might have been wrong there.

itsmatteomanf•10/21/23, 12:15 PM

It's also a beta, which is clearly marked for non-production workloads. There are issues that happen sometimes.

itsmatteomanf•10/21/23, 12:22 PM

Gonna add that in no place, where D1 open betas where announced, did anybody say that read-replicas were a thing:

What’s next for D1?
We’re setting a clear goal: we want to make D1 “generally available” (GA) for production use-cases by early next year (Q1 2024). Although you can already use D1 without a waitlist or approval process, we understand that the GA label is an important one for many when it comes to a database (and as do we).

Between now and GA, we’re working on some really key parts of the D1 vision, with a continued focus on reliability and performance.

One of the biggest remaining pieces of that vision is global read replication, which we wrote about earlier this year. Importantly, replication will be free, won’t multiply your storage consumption, and will still enable session consistency (read-your-writes). Part of D1’s mission is about getting data closer to where users are, and we’re excited to land it.
— https://blog.cloudflare.com/d1-open-beta-is-here/

Paul•10/21/23, 12:24 PM

https://blog.cloudflare.com/introducing-d1/

The Cloudflare Blog

Announcing D1: our first SQL database

Today, we’re excited to announce D1, Cloudflare’s first SQL database, designed for Cloudflare Workers

Paul•10/21/23, 12:24 PM

I think this might be the one that got me

itsmatteomanf•10/21/23, 12:25 PM

Yeah, in the announcements they talk about read-replicas, but in both the open alpha, and open beta it's mentioned replication isn't here yet.

Paul•10/21/23, 12:25 PM

That cross referenced with the “Limits” page

itsmatteomanf•10/21/23, 12:26 PM

The open beta blog post mentions a GA timeframe of Q1 2024, in which that is expected to exist.

Paul•10/21/23, 12:26 PM

So really the only way to find out is dig through discord, or just so happen to see it like I did

Paul•10/21/23, 12:26 PM

lol

PPaul So really the only way to find out is dig through discord, or just so happen to ...

itsmatteomanf•10/21/23, 12:26 PM

Well, no. Also in all open alpha and open beta blog posts it's mentioned it's not yet included...

Paul•10/21/23, 12:27 PM

Oh really… I didn’t notice it was mentioned as not included

itsmatteomanf•10/21/23, 12:27 PM

Yeah, it's in there.

itsmatteomanf•10/21/23, 12:29 PM

In the Open Alpha announcement: "The biggest pieces that have been disabled for this alpha release are replication and JavaScript transaction support."
In the Open Beta announcement: "One of the biggest remaining pieces of that vision is global read replication, which we wrote about earlier this year."

Paul•10/21/23, 12:29 PM

Yep, I see that now. Thanks

Paul•10/21/23, 12:30 PM

Luckily I abstracted away the database behind a repo layer… so I can swap it out

Iitsmatteomanf Yeah, it's in there.

Paul•10/21/23, 12:31 PM

Do you know if reads from a read replica will be charged as well? Or is that kind of “caching out of the box”?

Paul•10/21/23, 12:32 PM

That would be awesome

PPaul Do you know if reads from a read replica will be charged as well? Or is that kin...

itsmatteomanf•10/21/23, 12:32 PM

As far as I expect, as I haven't had more details than you do, they will be charged normally.

itsmatteomanf•10/21/23, 12:32 PM

You won't be charged more for the read replicas themselves. So no storage nor replications costs.

Paul•10/21/23, 12:33 PM

So that’s what they mean… well here’s to hoping reads from it are free as well

That would be insane for adoption

itsmatteomanf•10/21/23, 12:34 PM

I really doubt that... but hey, the Cache API still exists

Paul•10/21/23, 12:34 PM

Any really good typescript libs for the cache api?

itsmatteomanf•10/21/23, 12:35 PM

There is the usual

workers-types

workers-types

, which includes everything in Workers (and Pages Functions)

itsmatteomanf•10/21/23, 12:36 PM

And then https://workers-types.pages.dev by @PurpleBlob is a really nice UI

itsmatteomanf•10/21/23, 12:36 PM

(https://workers-types.pages.dev/#CacheStorage for the cache)

Paul•10/21/23, 12:36 PM

I’ll have to find one that is vendor agnostic

Paul•10/21/23, 12:37 PM

With an adapter for CF

itsmatteomanf•10/21/23, 12:37 PM

Then I've got no clue, unfortunately.

Paul•10/21/23, 12:37 PM

No lock-in here!

itsmatteomanf•10/21/23, 12:39 PM

Do note that the Cloudflare Cache API implements a subset of the normal Cache API from Service Workers.
https://developer.mozilla.org/en-US/docs/Web/API/Cache

itsmatteomanf•10/21/23, 12:40 PM

(Ok, technically the CacheStorage part, you need to open the cache...)

itsmatteomanf•10/21/23, 12:40 PM

https://developer.mozilla.org/en-US/docs/Web/API/CacheStorage

Paul•10/21/23, 12:52 PM

Something like this: https://github.com/Xiphe/cachified
I wonder if it’s possible to make a Cloudflare CacheAPI adapter for it…

GitHub

GitHub - Xiphe/cachified: wrap virtually everything that can store ...

wrap virtually everything that can store by key to act as cache with ttl/max-age, stale-while-validate, parallel fetch protection and type-safety support - GitHub - Xiphe/cachified: wrap virtually ...

Paul•10/21/23, 12:53 PM

I’ve read someone made a Cloudflare KV adapter for it, but Caching API is different right? (And cheaper?)

PPaul I’ve read someone made a Cloudflare KV adapter for it, but Caching API is differ...

itsmatteomanf•10/21/23, 2:01 PM

It's free! Not global, though. You can layer multiple levels (as in D1 > KV > Cache API), but not sure it's worth it at some point.

PPaul So that’s what they mean… well here’s to hoping reads from it are free as well �...

Matt Silverlock•10/21/23, 3:13 PM

Reads from replicas are the same - we still have to keep a full copy of the database.

PPaul So that’s what they mean… well here’s to hoping reads from it are free as well �...

Matt Silverlock•10/21/23, 3:15 PM

Not free, but keep in mind $5/mo gets you 25 billion rows read: a lot.

MMatt Silverlock Not free, but keep in mind $5/mo gets you 25 billion rows read: a lot.

Paul•10/21/23, 3:17 PM

sweet, when can we expect read replicas to be enabled?

PPaul sweet, when can we expect read replicas to be enabled?

Matt Silverlock•10/21/23, 3:18 PM

By GA per https://blog.cloudflare.com/d1-open-beta-is-here/

The Cloudflare Blog

D1: open beta is here

D1 is now in open beta, and the theme is “scale”: with higher per-database storage limits and the ability to create more databases, we’re unlocking the ability for developers to build production-scale applications on D1

Up•10/22/23, 12:43 AM

so what exactly is the spec for TIMESTAMPTIMESTAMP columns?

I've got a column defined as created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMPcreated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, and that produces values like 2023-10-16 15:56:512023-10-16 15:56:51, but what actual format is that so I can convert to ISO 8601-1:2019 properly?

UUp so what exactly is the spec for `TIMESTAMP` columns? I've got a column defined ...

Dahoom•10/22/23, 12:53 AM

date() time() YYYY-MM-DD HH-MM-SS

Dahoom•10/22/23, 12:55 AM

Interestingly I just researched this very thing 30 mins ago for my own project. D1 is basically just SQLite so you can see the info in the sqlite documentation https://www.sqlite.org/lang_datefunc.html

I think people expect D1 to behave more than Turso** where DB read and writes are made only to a loc

Similar Threads

Similar Threads

Similar Threads