I don't think that would work for my architecture/application. And, again, my issue is with it being stored in expensive ram - even if the architecture was split like you suggest, it would still be orders of magnitude more expensive than ssd.
D1 Manager is a web UI and API for Cloudflare D1, a serverless SQL database. It provides a user-friendly interface for managing databases, tables, and records, as well as an API for performing oper...
I think it depends on how data is organized. Sharding won’t work well for all database. My idea was because of the natural of worker. Sharding won’t be a problem if it suits
Thanks for the clarification! So, if I understand correctly, at some point in the future when D1 is public, we should be able to scale a D1 db relatively infinitely because it would be stored on disk rather than ram? Again, it's easy to load into ram when the limit is 100mb, but if someone wants 40gb or 1 TB (SQLite has theoretical maximum of 281 TB) then the costs as well as logistics of even deploying that much ram would be impossible
p.s. related to all of this, I've been looking into distributed sqlite implementations. There's a lot of options out there - https://rqlite.io/, https://dqlite.io/, https://bedrockdb.com/, https://litestream.io/, https://fly.io/docs/litefs/ and probably more. Basically they tend to offer auto leader-follower selection, auto replication, etc... for being able to easily deploy your own HA SQLite cluster on bare metal, vps servers etc... Seems like a poor man's Vitess. I wonder if you could get any inspiration from some of them?
In case case of being unclear before: I'm wondering if I need to check the "success" property of the result object returned from stmt.run() and .all() , in adittion try wrapping it in a try catch block. So the question is if it's possible that the promise resolves with the result { success: false, ... }
Do you think cloudlfare will provide tooling (or even just examples) of how that process can be automated through workers?
Updating the schema across N instances, tracking if there's any errors, rolling them back, modifying the script etc. Sure it can all be done manually by the devs but it's quite a lot of work compared to the simplicity of everything else in workers