So this second KV would have more than 12 million rows? 3 million games with on average 4 categories
So this second KV would have more than 12 million rows? 3 million games with on average 4 categories.
Jbasically, you need to write a schema with your indexes, and import your data into a sqlite database (something like this: https://gist.github.com/johtso/c4de17de513c36f91c948498c8fc7908)
LWhere can I find more about D1's restrictions?
J@louisdck and then you specify the location and shape of the database in
createDbWorker and then you can query it using sql https://github.com/johtso/infinite-novelty-xstate/blob/master/src/machines/db.ts
HYes, but note that other than on reads/writes, KV bills on Stored Data, not rows
HDoes it work if you have to read in the data in chunks?
J@HardAtWork how do you mean?
Jwhat do you mean by read in the data?
HThe CSV file is 550 MB, which is more than a Worker can store in memory, so you would have to read it in chunks
Jyou would never read the whole thing, it's stored in R2 and you read the rows you want to show to the user, and the queries can come straight from the frontend
HSo it uses range requests then?
Jyep
Jand if the database file is bigger than the 512mb cache limit, or you want to trade off higher possibility of cache eviction for quicker "cold" requests, you can break the database into chunks using
create_db.sh
HFor Cache?
HThat’s a paid upgrade
HYeah, maybe at some point, but not right now
JIs there a wrangler command or API to list all keys in a KV?
Jglad I was generally in the right area. Now trying to find my bearer token
KKKJoh nice! the wrangler command works great
BHi all. I apologize for neglecting this channel for a while. We've been busy for the past little while on KV 2.0 and we're getting ready to start early beta testing if you're interested. There's some architectural changes that we're going to be describing later this year in a blog post but the high level summary of the benefits are:
Just to be clear. Everyone is going to get this. This is purely about having an opportunity to provide feedback early - much easier for us to investigate issues when there's only a handful of accounts on this :).
Please register interest by filling out https://forms.gle/d1yrVjxpQNFSYYYf6
- Eventual consistency is a lot better bounded than it is today. We've observed worst case ~days for stale reads but now it should be closer to hours and we're working on cutting that down even further.
- Read after write consistency should be nominally much closer to 1 minute or less globally even when you specify a long cacheTTL.
- There may be some TTFB improvements. I'd ask you please provide feedback to this thread if you do see any changes as I'm curious what people are seeing on their end.
Just to be clear. Everyone is going to get this. This is purely about having an opportunity to provide feedback early - much easier for us to investigate issues when there's only a handful of accounts on this :).
Please register interest by filling out https://forms.gle/d1yrVjxpQNFSYYYf6
Google Docs
KV 2.0 will be rolling out over the next month or so. Submit interest to get into an early beta before we turn this on globally.
HI assume since it is first byte it would be cold reads?
BCan we move this discussion into the sign up thread? I'll answer this there
HIs there a thread that I missed somewhere?
BUggh. Maybe I didn't do it right. Let me figure out Discord
DSupport for assets > 25 MiB in KV
OMG YES I NEED THIS
Neat
I need you to fix something else (please specify)Poll by beelzabub#0369
•2/24/23, 8:20 PM
BI replied in more detail, but TLDR it should be some amount of both. "cold" is even more complicated to discuss without describing what changes we made.
Sis there anyway to gzip/br json content before storing to kv and then decode on get?
KWorkers implements https://developer.mozilla.org/en-US/docs/Web/API/CompressionStream and https://developer.mozilla.org/en-US/docs/Web/API/DecompressionStream but those are things you'd have use with your own wrapper around KV - KV itself doesn't have any (de)compression options
UWrites have to go to central storage so yes thats normal
ZCache hits count has reads, so I would expect you to see 20 reads.
ZI believe so
JI request to Make a Realtime Key-Value storage model, like Firebase real-time database.
BFor KV, requests for the same key are satisfied within milliseconds and it doesn't matter where in the world you access this from. My understanding is that real-time databases have access time measured in hundreds of milliseconds and requests from other parts of the world are slower. Real time databases are great tech, but it's not comparable to the kind of storage KV offers (totally different pricing, totally different usage patterns optimized for, totally different invariants in terms of API).
If you can use KV, I think you'll see more value out of KV (cheaper & faster). It won't be suitable for every problem so YMMV. R2 is strongly consistent like firebase and will be cheaper than firebase, but it won't offer the database-like features if you need them (e.g. transactions).
It can sometimes be more helpful to describe your use-case @Jadugar_Jaggu rather than asking for a specific product. If you can help me understand the problem you're using a realtime database to solve for, it's possible there are other ways to accomplish that. If you're looking for a drop in replacement for Firebase, I don't think CF has anything like that at this time to talk about.
If you can use KV, I think you'll see more value out of KV (cheaper & faster). It won't be suitable for every problem so YMMV. R2 is strongly consistent like firebase and will be cheaper than firebase, but it won't offer the database-like features if you need them (e.g. transactions).
It can sometimes be more helpful to describe your use-case @Jadugar_Jaggu rather than asking for a specific product. If you can help me understand the problem you're using a realtime database to solve for, it's possible there are other ways to accomplish that. If you're looking for a drop in replacement for Firebase, I don't think CF has anything like that at this time to talk about.
JI am using KV on my service which has a total of 15 million traffic per month, It is working well, and Seriously KV helps me to achieve a 60% Profit margin from my business by lowering the various servers and management costs.
I am a KV user for 9 months approx
Now I was thinking of making a realtime application like you know about Trading and Fantasy Sports apps in which a large number of users perform action simultaneously.
I am a KV user for 9 months approx
Now I was thinking of making a realtime application like you know about Trading and Fantasy Sports apps in which a large number of users perform action simultaneously.
JThats Why I just requested if KV can be helpful in realtime database model it can save lots of expenses
JAnd Seriously I don't want to know how Cloudflare manages to lower the costs while delivering good performance, I just enjoy the profits 
AI think for real time, you need durable objects. See this video https://www.youtube.com/watch?v=Frqy7jqnjZs
YouTubeChangelog
From the "JS Party" podcast. More 
https://jsparty.fm/209
Subscribe for more!
Apple: https://jsparty.fm/apple
Spotify: https://jsparty.fm/spotify
Android: https://jsparty.fm/android
Overcast: https://jsparty.fm/overcast
Email: https://jsparty.fm/email
Twitter: https://twitter.com/jspartyfm
#js #javascript #webdev #webdevelopment #opensourc...
https://jsparty.fm/209Subscribe for more!
Apple: https://jsparty.fm/apple
Spotify: https://jsparty.fm/spotify
Android: https://jsparty.fm/android
Overcast: https://jsparty.fm/overcast
Email: https://jsparty.fm/email
Twitter: https://twitter.com/jspartyfm
#js #javascript #webdev #webdevelopment #opensourc...
BDO won't be able to keep up with that volume, not without serious sharding.
BIt depends on specifically your data patterns and what your application needs actually are. It's hard to say without learning more about what kind of actions users are performing, what consistency and transaction requirements are, etc.
