web app makes all kinds of D1 requests on a worker request, get session, get user, update user etc.
web app makes all kinds of D1 requests on a worker request, get session, get user, update user etc. Each SQL stmt is a round trip.
E100 ms seams reasonable. but 500 doesnt. You should consider not splitting up your statements and write real SQL language. I'm not making fun of you FYI a lot of people including me have a issue of splitting up queries when realistically it can be one query.
JAlready use straight SQL, there's no getting around some big requests with lots of disparate SQL for different purposes on a modern app. The downfall of D1 is that it's not colocated with the web server like many other services. Whats close for one user is far for another. Personally a db request should have sub 10ms latency from the web worker for simple queries. Closer to 1-5ms even or less.
Ecompletely agree. The latency is a network latency tho thats the issue
EI had the same argument last time about it in this chat
I
wonder if you could run a SQLite DB in WASM and then stream the client the WAL somehow: https://sqlite.org/wasm/doc/trunk/npm.mdEthere is a sqlite client in WASM
Esql.js i believe
IThe link I sent above is one yeah
INot really a client it's actual full SQLite
IOh yeah sql.js is also one
EEam i missing something about wasm?
Ewhy wasm in the backend?
Esame with the one i sent i believe
IIf you are building a native application in JavaScript (using Electron for instance), or are working in node.js, you will likely prefer to use a native binding of SQLite to JavaScript.They recommend against that actually
ETalking about cloudflare in general :NotLikeThis:
IMainly because WASM is portable and secure. If you allowed anyone to upload binaries (as does AWS Lambda/GCP Cloud Run) then to get enough security you basically need to run them in a full, dedicated VM or at least gVisor (GCP). With WASM even though it's untrusted code, running inside the V8 WASM sandbox is secure enough.
IAnd because it's running in V8 it's far lighter-weight than a VM/gVisor and they can afford to run thousands/tens of thousands of WASM sandboxes.
I
MD1s alpha was SQLite in WASM on DOs, it was uh "fun" to operate
E:kekw:
Ei will have to test this
Eim working o nsomething similar to AWS lambda and CF workers using NixOS microvm.nix and firecracker
BWorker smart placement should help with this though, as it should put your worker closer to D1 rather than closer to user if it sees that you are making multiple round trips to D1. Read replica is also on the roadmap, although there doesn't seem to be any ETA for that.
BI'm also considering if I should build on D1 or DO + SQLite storage, but the latter having to pay extra for DO requests, as well as losing out on D1 features, are putting me off.
JHad the same concern, what clinched it for me is the 10gb limit on d1. Audit trails, request logging etc eats that up quick. Only d1 feature I miss is dumping the SQLite file. Going custom on the rest isn’t hard.
B10 GB limit? Ah, maximum database size.
Emake new dbs i guess
Ebut i swear cloudflare already does those for you
BDO SQLite currently only has 1 GB limit though, and doc seems to indicate that it will be raised to 10 GB when GA, which matches what D1 offers.
J1gb per user is reasonable for a multi tenant app. Can scale that to terabytes if needed. Much harder to scale d1 to 1tb.
SWell i truly think my worker smart placement is bugged, it wont analyze anything anymore, not even stating something in the dashboard. has the minimum traffic for smart placement increased in someway?
Wcould you send me your account id please?
SSure! Account ID is f61e2912c0937987fbd186d3523dc847 and our Dev Worker is called "worker-customer-devinc" (screenshot above is from one of our tenants)
WThanks let me take a look
WSo, it looks like smart placement was re-enabled 30 mins ago and yeah, so far it doesn't have enough requests
Syeah i did it (a few times in fact) to see if it reappears
Sthe other worker hadn't hat it reenabled and doesnt have it either
WI can check that one too, what's it called?
Sworker-customer-***
Slast deployment there was a month ago
WIt looks like that also has pretty minimal traffic. looks like it was placed on the 13th and then not on the 15th - let me see what changed
SThx! The problem is each of our tenants won't have much traffic. we are a cloudbased cash register provider, so we only have a limited amount of users per tenant
WOk I see what changed.. figuring out why
WOk yep, looks like an issue our side - gonna let the team know!
Thanks for reporting, we'll get it fixed!
Thanks for reporting, we'll get it fixed!
S<3 Thank you!!!
Kcan someone help me with this. I'm trying to connect vitest with D1 local generated by wrangler dev, but it's not working, I use Vitest-pool-workers to do it with this configuration:
and to ensure I tried to show in the logs like this
and it's show undefined, it just show some data when I run wrangler dev,
and to ensure I tried to show in the logs like this
and it's show undefined, it just show some data when I run wrangler dev,
AIs it possible to increase the 10 gb Limit d1 has currently?
1
Cloudflare Docs
Cloudflare also offers other storage solutions such as Workers KV, Durable Objects, and R2. Each product has different advantages and limits. Refer to Choose a data or storage product to review which storage option is right for your use case.
AYes, already read that, but was not sure if applicable to the db limit in General too. Thanks!
