Thanks guys for this brainstorm.
Thanks guys for this brainstorm. Multiple independent DB's isn't something I can do. I needed one DB, with read and write replicas. Where the write happens also to the nearest DB but would eventually propagate to all others.
11 Replies
This is active-active setup where you have multiple write primaries and introduces big latency spikes to all your writes due to coordination anyway.
I would say it's not very likely to come soon, and most databases don't support this either, unless you go to some fully distributed products like CockroachDB/Spanner but even then you have other tradeoffs.
ah thank you
im new to thinking about this stuff.
so its not very likely to just write to the nearest one with trade-off of if anyone reads from the others they wont have up to date info?
Idk doesn’t Cassandra/Scylla support this?
There are databases supporting writes in multiple nodes, but you end up in an eventually consistent world, or very slow writes.
thanks guys. im trying to use d1 in eventually consistent mode. because my use case is i have churches which is geographically based. so everyone hitting and reading it from that matter is just people in that church. in the extreme rare case, support member helps out and he'll know to wait 1s before expecting things. and in a more extreme rare case, we have random users around the world curious to look at profile pages of churches around the world.
Unknown User•12mo ago
Message Not Public
Sign In & Join Server To View
Thanks very much for asking! That's a great point on the isolation. However there are cases where Churches work with each other. Like shared donation box, or shared event. The most common use case is the ability to follow multiple Churches, and 30% of users do that.
I still think you don't need active-active writers across the world. From what you described above, majority of usage is reads. This can be solved with read replicas as well so all your users read from a replica close to them, even if your writes can take a few ms more due to having to be routed to a single location. See this blog post as example of how that could look like.
What is the ratio of your reads vs writes? I might be wrong, but you seem to be focusing too much on writing across regions, which would complicate your life a lot. You mentioned donations etc and anything that has to do with money doesn't play well with eventual consistency anyway, unless you go through hurdles.
The Cloudflare Blog
Building D1: a Global Database
D1, Cloudflare’s SQL database, is now generally available. With new support for 10GB databases, data export, and enhanced query debugging, we empower developers to build production-ready applications with D1 to meet all their relational SQL needs
Thanks very much for all the thoughts here!
My reads are low as all pages are static with Next.js ISR (incremental static regeneration - https://nextjs.org/docs/pages/building-your-application/data-fetching/incremental-static-regeneration ) - so I only experience problems when people are enrolling into events and its slow or admin dashboard is writing.
Unknown User•11mo ago
Message Not Public
Sign In & Join Server To View
Thanks @algads and everyone for the awesome brainstorming and intro here!
I didn't know writes across regions is sub 500ms thats very cool. That makes it not a problem.