Cloudflare Developers•3y ago

Does data get removed from KV if not accessed for a long time? i.e. 1 year.

VVarna Does data get removed from KV if not accessed for a long time? i.e. 1 year.

Hello, I’m Allie!•12/5/23, 3:45 PM

No. As long as you are willing to pay, it will stay

Chaika•12/5/23, 3:46 PM

(Unless you set an Expiration TTL on it, of course)

Chaika•12/5/23, 3:46 PM

If you wanted to, it could. Otherwise it's a full data store. After a year it wouldn't be cached anywhere but it would still be at the two central stores

w•12/5/23, 10:53 PM

Would Cloudflare KV be secure enough to store encryption keys? I'm trying to store many sets of data with each one being encrypted so thinking of using AWS KMS to store a master encryption key and storing multiple Data Encryption Keys in Cloudflare KV that are encrypted and decrypted using the AWS KMS key before use each time. Since I need to encrypt and decrypt frequently, KMS is a bit too price prohibitive for use entirely.

Santosh•12/6/23, 3:55 PM

Quick question: In case I update a KV value behind the scenes prior to its expiration (say using a DO), would the clients who are using the KV see it upon the next fetch or is the expectation that clients using the KV see the updated value within max of 60 seconds??

SSantosh Quick question: In case I update a KV value behind the scenes prior to its expir...

Hello, I’m Allie!•12/6/23, 3:56 PM

Clients will see it whenever the edge colo nearest to them pulls from the origin(or, if they are in the same colo as updater, then basically instantly)

Hello, I’m Allie!•12/6/23, 3:57 PM

So, if you have a cacheTtl of a week, then it could take up to a week to show an update

Santosh•12/6/23, 3:58 PM

Would I not be able to influence the read using this?

env.NAMESPACE.get(key, { cacheTtl: 3600 });

env.NAMESPACE.get(key, { cacheTtl: 3600 });

. I understand the write has a ttl as well and the read ttl has a min of 60secs

SSK there is no mistake, kv is in-fact not negligible latency like you might assume

Santosh•12/6/23, 4:53 PM

Interesting that you mentioned Cache API, what use cases would require to use KV vs Cache API(given thats local to the Data center that the caches it locally)? Just curious. Would use cases that require heavier processing for making the "object" available require KV? What kind of latencies have you seen with cache API?

SSantosh Interesting that you mentioned Cache API, what use cases would require to use KV...

Unsmart•12/6/23, 4:56 PM

The Cache API is just a cache and can be evicted at any time. KV basically has a very basic API similar to cache but its durable storage.

So Cache API if you just want a very basic cache that can be evicted at any time. KV if you want a more durable storage API with low latency (KV automatically uses the Cache API in front of it).

Santosh•12/6/23, 4:57 PM

A simple config read from R2, should we consider that more suitable for Cache API vs KV?

Unsmart•12/6/23, 4:58 PM

Yeah I would use Cache API and not KV for that fastnod

No need to have 2 durable storage layers.

Unsmart•12/6/23, 5:00 PM

KV may be a better fit for configs than R2 given KV will have lower latency but its also added cost so depends if you care that much about getting that slight edge in latency by having 2 origins vs 1 before the cache.

Unsmart•12/6/23, 5:02 PM

I will say though with KV only having 1 extra origin I do almost never see a reason to use KV instead of R2 + Cache API

If it had like 3 or 4 origins it might start to make more sense for truly latency critical things

UUnsmart I will say though with KV only having 1 extra origin I do almost never see a rea...

Santosh•12/6/23, 5:04 PM

Can you pl explain "1 extra origin" concept? Not sure I understand what you mean by 3/4 origins.

UUnsmart KV may be a better fit for configs than R2 given KV will have lower latency but ...

Santosh•12/6/23, 5:06 PM

It appears like KV latency is jumping all over even after Cache (theoretically KV within colo already uses cache ??) from the comments above ?Using plain vanilla Cache API -

Unsmart•12/6/23, 5:06 PM

KV stores data in 2 locations. One in the US, one in the EU.

R2 stores data in 1 location (location is selected at creation time).

So for reads that miss the cache KV can be slightly faster because it might have the data stored closer to the user. With only having 1 extra location the data is stored though the added benefit is pretty minimal IMO. But if KV stored data in like 3-4 regions across the world the latency boost for cache misses could be pretty noticeable.

Unsmart•12/6/23, 5:08 PM

Throwing the Cache API in front of KV will not really change the latency of KV given KV already caches data.

Santosh•12/6/23, 5:09 PM

What about using Cache alone without using KV api? WOuld that change the latency to access the data?

Unsmart•12/6/23, 5:10 PM

It would yes unless whatever source you are getting the data from already uses cloudflare cache in some way.

Unsmart•12/6/23, 5:11 PM

A cache will make things faster but if the data is already cached it cant make things faster than that.

SSantosh Interesting that you mentioned Cache API, what use cases would require to use KV...

SK•12/7/23, 2:23 AM

I don't have high enough RPS for KV to cache my data and the cold reads are pretty slow

SK•12/7/23, 2:23 AM

few people visit my website, and when they do it's slow lol

SK•12/7/23, 2:23 AM

https://upstash.com/blog/edgecaching-benchmark

SK•12/7/23, 2:24 AM

upstash benchmarked KV and they found the same thing. But I've had a much better time simply using the cache api on-top of KV

SK•12/7/23, 2:56 AM

ok I just ran a very quick benchmark

SK•12/7/23, 2:56 AM

https://test.blueradar.net/

SK•12/7/23, 2:57 AM

there are 2 routes, one fetches the data from KV, and the other tries the cache first and fallbacks to kv

SK•12/7/23, 2:57 AM

I ran a simple python script that hit each route 100 times and recorded the latencies to get some summary stats

SK•12/7/23, 2:57 AM

UUnsmart A cache will make things faster but if the data is already cached it cant make t...

SK•12/7/23, 2:57 AM

it turns out that it does actually make it faster

SK•12/7/23, 2:58 AM

sure not by a whole lot but but it does

SK•12/7/23, 2:58 AM

the last time I tested this KV was performing much worse (around 50-100ms) so I'm super happy that it's working heaps better now

Unsmart•12/7/23, 3:02 AM

I mean if you think it helps you can do it sure no one going to stop you

SK•12/7/23, 3:03 AM

but yeah, when I refactor this I'll switch to r2 + cache api instead of layering it on kv, that was just placeholder to get it working

SSK <https://test.blueradar.net/>

Vikash•12/9/23, 3:46 PM

Code of your test?

Are you using kv.get(key, { cacheTtl: 60 })kv.get(key, { cacheTtl: 60 }) or the cache API?

VVikash Code of your test? Are you using `kv.get(key, { cacheTtl: 60 })` or the cache ...

SK•12/10/23, 1:24 AM

https://gist.github.com/Repo-SK/76775be58020c964c17eae45be8b3104

SK•12/10/23, 1:25 AM

/kv uses kv.get with a cacheTtl value of an hour (3600) and /kv-cache-api uses the cache api

SK•12/10/23, 1:25 AM

for the benchmark simply send requests to both endpoints and extract the latencies from the html returned (I used a python script to send 100 requests to each)

kian•12/10/23, 6:18 PM

As in the read takes that long to fulfill, or that long until you see the new value?

zehawk•12/16/23, 3:09 PM

Where does one see how much storage is being used by KV? I can never get an accurate value. I got billed for 1.69GB while the KV page shows 67KB. So what gives?

Mmatt_d @izdi No, this behavior is an inescapable aspect of WorkersKV.

Advany•12/17/23, 1:00 PM

is it already as fast as 5 seconds? I though it would take up to 60 seconds to get consistent with the write

Zzehawk Where does one see how much storage is being used by KV? I can never get an accu...

Matt Silverlock•12/17/23, 1:03 PM

Please open a Support ticket here.

Does data get removed from KV if not accessed for a long time? i.e. 1 year.

Similar Threads

Similar Threads

Similar Threads