When we implement replication, the idea will be that any particular DO can be replicated to multiple locations around the world. So if a lot of workers are requesting the same DO ID, they'll be routed to appropriate replicas, and the sqlite state is replicated from the primary to all those replicas. Durable Objects with different IDs won't sync with each other.
Making DOs movable is a separate feature we want to work on. I think with the new storage backend it will be a lot easier for us to finally do that, but I think replication is likely to happen first.
The Storage API allows Durable Objects to access transactional and strongly consistent storage. A Durable Object’s attached storage is private to its unique instance and cannot be accessed by other objects.
It depends. Building for DO still means architecting in a new, different way that not everyone is up for. D1's goal is to support the classic three-tier web architecture that people are used to.
Would D1 w/ Smart Placement’s latency be comparable to SQLite on DO’s latency? I’m going to be doing a little test soon but I’m asking just to know what to expect with my results
Smart Placement with D1 will reduce your latency down to a few milliseconds, but SQLite on DO has latency of a few microseconds. Still a big difference, but may or may not matter depending on how you use it.
I want to ask how all DO will work together. For eg: Let's say I want to query across all DO on sqlite Because DO has 1 GB limit, I'm still not getting how to infra my app according to it, if I want to migrate from mongodb.
Generally you want to split your database into logical "shards" of related data, trying to make these shards as fine-grained as reasonably makes sense. Remember that it's totally fine to have millions of DOs -- the system is built for that.
It contains V8-serialized values. We don't want people trying to decode the V8 serialization as it sometimes breaks in backwards-incompatible ways (which we can deal with as long as we control all the deserializers). We are also not completely sure that the parser is resilient against malicious input. Chrome similarly uses V8 serialization but never lets apps see nor modify the serialized bytes.
Thank you for the answer! I was particularly interested in the serialization/deserialization, that clears up why there's no functions for that exposed.
Still plans to put DOs on more regions? Sucks very hard here on South America. And i get the problem of the storage, but in many cases a DO whiout any storage backend will be just fine. For handle Websockets will be very helpfull, today i need to pay the latency to call a DO across the atlantic just make a websocket point. Sometimes i use a DO just to keep my in memory graphql cache, etc. The point is: Every point that runs a worker should be able to run a DO without storage. (more storage locations will be good also);
FYI we've identified an issue with alarms in the beta for SQLite in Durable Objects. While our team implements a fix, we are temporarily disabling alarms for only Durable Objects with SQLite storage backend.
Durable Objects with the default, key-value storage are unaffected and alarms are functioning normally
hope it's ok to solicit feedback on a use-case here.
i'm interested in building a personalized email scheduler, and i want to see if DO would be a good architecture for this & if it would scale to millions of users. the exact use case is that i have a ranking algo where for a given user & set of emails, i'll predict which email to send, and now i want to predict the optimal send time too.
the way i've seen this work at prev companies has been to have a daily cron job that computes the optimal send time for every user & stores them in a database. then a separate cron job would run like hourly to fetch the users eligible to receive an email or notification and then apply the content ranking algorithm. however, that usually means that you'll need at least two cron jobs, a database, and a pubsub topic.
what i'd like to do is to use a durable object per user to represent their send time preferences. in an alarm routine, it would wake up => fetch & rank the user's eligible emails => send an email => set its next alarm. a separate worker would receive webhook events about when an email was sent / opened / clicked / etc, so it would update the durable object per user. if the email is opened, the worker could update the user's preferences & reschedule the alarm.
if this sounds like a bad idea (too overengineered, etc), would love to know before investing too much. i'd be looking at supporting millions of users & emails per week
Is there any way to create durable objects bindings on the fly with api or sdk for each users or maybe I'm missing something. A basic tutorial on scalibility would be appreciated if possible.
See https://developers.cloudflare.com/durable-objects/best-practices/access-durable-objects-from-a-worker/, you would have a binding for a given DO (implemented by a class in your code) and then create DOs on the fly using whatever mechanism works for you (e.g. create a DO using a userId with idFromName). So the DO creation is dynamic, it’s not like other bindings where you create a resource and then you interact with one resource with an ID, with DOs you bind to a class + then generate millions of resources (if you need)
Does anyone have a good way to uniquely identify websocket connections? I have a simple map of id -> websocket that i want to keep in sync even when the ws hibernates or the DO is evicted from memory
Is there any way to get notified when a new DO is created or updated? I guess you can build it yourself with Alarms, but wondering if something built in