Would KV lend itself as a server side cache such as redis in a Worker or should rather something els
Would KV lend itself as a server side cache such as redis in a Worker or should rather something else being used?
HThe Cache API is the best cache(because it is free), but KV is a good alternative if you want the cache to be persistent.
B@sly_de can you elaborate what Redis in a worker means to you? Do you mean from a latency prospective? Volume? Something else?
BYou can request to enroll only certain namespaces.
Once we start the march forward, everyone is going to start getting ramped transparently. The way it works is that we’ll opt in your key space by percentage. Some keys will use 1.0 while others go 2.0. It’s a transparent layer in the middle of KV internals. Think weeks not months. So opting in early is just a way where you’ll have my attention if there’s ANY issue. That becomes different at scale I can’t provide as much white glove support (I still try to but often observability at scale makes it harder to diagnose individual issues)
Once we start the march forward, everyone is going to start getting ramped transparently. The way it works is that we’ll opt in your key space by percentage. Some keys will use 1.0 while others go 2.0. It’s a transparent layer in the middle of KV internals. Think weeks not months. So opting in early is just a way where you’ll have my attention if there’s ANY issue. That becomes different at scale I can’t provide as much white glove support (I still try to but often observability at scale makes it harder to diagnose individual issues)
SLatency - Expensive calculations in third party api to be cached on server for a certain amount of time
DBear in mind that it does not replicate between colos, so you might see "more" misses depending on the distribution of visitors - for which you can use KV as a second option (BetterKV does this in https://flareutils.pages.dev)
BBetterKV is fine in terms of reducing your KV billing cost but I’m not a fan of it and it feels like an artifact of our billing than anything else. Is there any other features it has beyond “cache in front of Kv”?
BWhat’s the latency you need?
DOh absolutely, the main purpose is to remove the cost of cached KV reads, or at least this is what we use it for.
A quote from the website even says
A quote from the website even says
Save on your Workers bill by "manually" caching every response, without having to lift a finger!
BYeah. I hope to take a look at our billing in Q2 to see what actually makes sense. BetterKV feels technically superfluous and a recipe for subtle bugs at scale.
DWe were actually discussing this with @mrbbot (sorry) last week, whether it would make sense (from a user perspective) to only charge for cold/uncached reads - basically make KV Better 
, as now everything would be double-cached to cache at KV-level, and at Worker-level. From what I remember, the argument against was that it would result in less predictable billing, but the counter-argument to that is while less predictable, still bounded by the current number of reads.
, as now everything would be double-cached to cache at KV-level, and at Worker-level. From what I remember, the argument against was that it would result in less predictable billing, but the counter-argument to that is while less predictable, still bounded by the current number of reads.DFrom there (oh you will not like me for this), cache(+pull if stale) reads and force refresh reads could be separated out and billed separately, turning KV into a powerful key-value storage with a different type of push-pull consistency model than what you have today (with KV 2.0).
BI don’t know that the problem is unpredictability. And I don’t hate you because that’s very much along the lines of how I’m thinking about it.
BBut I do want to keep the API as-is and do the billing transparently. So I’m not sure yet what the new pricing would look like. And I also have 0 concrete idea of what that looks like. Once 2.0 rolls out fully I’ll have data to formulate an actual opinion
B- if it was just up to me, you know I like you guys. I’d just 0-rate everyone on here :D.
DI don't want to call this a problem, but I couldn't come up with any better phrase - so the problem with billing for cached reads is that using Cache API on top (that is, BetterKV) will still be an improvement over the default behaviour (no added latency, but reduced bill).
U0 rate is nice 
UI will say though I do think it is a bit odd to offer a 100% free cache API but then charge for cached reads on KV
UDurable objects also charge for the in memory cached reads which is odd too 
DComing from this model originally I believe is fine, as a next step it would be very Cloudflare (ahem, 0-egress 
, end billing when stream response returned, etc) to now say ok, cached reads are now free
, end billing when stream response returned, etc) to now say ok, cached reads are now free
DI didn't even think about DOs, that's another good one - that would eliminate another 
lru-cacheThe issue with building your own cache on top of DO storage is that you don’t have a real easy way of measuring memory usage, where iirc the default in-memory cache auto-evicts when memory is low
Dlru-cache can (sort of) deal with that, for ~100 keys that shouldn't be an issue even if all keys are 128KB, but most (at least for me) are under 2KB
BYeah. The team is aware of the DO thing. That being said, most providers will do caching underneath of any storage product and eat the savings so it's not rare and I don't think it would change for DO. I wouldn't recommend putting Cache in front of DO because you have no way of guaranteeing the consistency of the contents in cache / DO. lru-cache is a cute idea and would work now but I suspect it'll not be sustainable as DO matures over the years for more than toy examples (although I could be wrong).
Maybe the main actual weirdness is that the Cache API is free. I'll see about making sure we start charging for it. Be right back. Will flip the flag for that.
Maybe the main actual weirdness is that the Cache API is free. I'll see about making sure we start charging for it. Be right back. Will flip the flag for that.
UI think the only reason people complain about charged cache reads on CF is the fact that the cache api is free tbh
HWhat have you done, Dani?! YOU’VE DOOMED US ALL!!!
UDoesnt totally make sense in peoples minds that cache api is free but cached database reads are not free
HEspecially if there isn’t a whole lot of extra logic that goes into the cache. For example, you could probably charge a little bit for SWR caching on KV, but maybe not as much as a full read
HThe auto-pull while cached thing is a feature on top of what the Cache API offers, which is why I think you could justify at least a slight cost there
DWe've actually agreed with @beelzabub in DMs that I'd get 1% of all Cache API bills 
HI’ll be fine with that arrangement if you pay the Cache bills for everyone, not just yourself
DLook at us complaining about products priced at $0-5/million range compared to saas's which build on these and charge you $40-50/user/month
HI mean, I don’t think I can complain at all… :winks_in_green_cloud:
Dalso you're a champ, you have a free ent account, 
D(I'll pay yours of course)
HDisables NoCost just for funzies
DI know this is getting sortof #off-topic, but I no longer superlike you, mr superchamp
BMonkey paw wish granted. Everyone gets a 1% coupon thanks to @Dani Foldi volunteering to cover 1% of everyone's bill.
BThey count. But note it's per request. Not sure why you'd make repeated KV reads within the context of a single request
Dsudo rm -rfI thought https://protomaps.com/blog/serverless-maps-now-open-source was interesting.
A web application with 50,000 map loads using the Google Maps API will cost $350 per month.
A similar amount of traffic running through a Protomaps CDN install on Cloudflare, with an average of 20 tiles per session, costs fifty cents in Cloudflare Workers request fees.
B@_john_n does that answer your question? I'm kind of curious about the context where you're concerned about the 1k limit
HIf you want to see an (admittedly bad) implementation of that, you can try https://maps.goalastair.com
HIs there an easy way to cache Range Requests? Then I could maybe speed it up a little...
BI thought range requests are generally cacheable, no? You might need to store the actual asset in cache once but after that cache.match will honor ranges? I forget the details though. CDN behavior is extremely complex and I don't pretend to know it outside of the specific slice of how I'm using it
HI don't think so? I tried tailing the Function, and I'm getting
