Low latency D1 for hourly config updates
If you rarely update that config and you have many requests needing it, then you can have D1 as the source of truth where you write your config updates, and then also put them in Workers KV. Your worker can read from Workers KV when a stale value (up to 60s) is OK for that case, and go to D1 if you need the latest/freshest configuration.
Using the local CDN Cache API from your worker sometimes give you faster lookups than Workers KV as well, but it also depends on your scenario. It's not clear from what you said above.
This design can give you the speed when you can use cached values and have D1 as a fallback for your endpoints that need the latest.
4 Replies
Thanks for the tips!
Would durable objects with D1 on the backend storage be useful for this use case?
Basically the ground truth is currently on external DB.
I can have a worker scheduler pull the external DB off every hour and put it on D1.
The DO could use D1 for the storage and leverage the Cache API somehow.
I essentially want a Configuration Singleton that is at most 1 hour late in its update and all workers wake up with the proper config, and if we update it externally, it would roll out as well. so it's very low latency but costs should also minimal (aka I find KV expensive)
essentially want a Configuration Singleton that is at most 1 hour late in its update and all workers wake up with the proper configI don't understand what you mean. Workers are stateless. They cannot have long-running state. They need to load it from somewhere. BTW, there is no DO with D1 backend. You probably mean DO with SQLite backend. A single DO is also in a single location, similar to D1, so your workers would still need to reach to that DO to get the config.
True
I was refering to in-memory state (https://developers.cloudflare.com/durable-objects/reference/in-memory-state/)
and on this page (https://developers.cloudflare.com/durable-objects/)
they use the term "stateful serverless applications".
So I thought that would allow shared state among workers
The in memory state is within a single Durable Object instance. Not shared memory among Workers calling the Durable Object.