Cloudflare Developers•2y ago

Are cached KV values in local DCs evicted earlier than the specified TTL if they are infrequently re

Are cached KV values in local DCs evicted earlier than the specified TTL if they are infrequently read?

Aanirudh Are cached KV values in local DCs evicted earlier than the specified TTL if they...

Matt Silverlock•3/16/24, 2:21 PM

The TTL is not for the cache. It’s for the key-value pair itself (after which it is deleted).

anirudhOP•3/16/24, 2:29 PM

Yes I’m aware, but in the docs its says that values are stored in central servers and then cached in regional tiers and finally in local DCs closest to the workers. I was asking about the eviction behavior of these caches.

Aanirudh Yes I’m aware, but in the docs its says that values are stored in central server...

Hello, I’m Allie!•3/16/24, 2:30 PM

If you are referring to the

cacheTtl

cacheTtl

parameter, then yes. It is best effort, which means that it will, at the most, be cached for the specified amount of time

anirudhOP•3/16/24, 2:36 PM

The eviction behavior of these caches

Aanirudh The eviction behavior of these caches

Hello, I’m Allie!•3/16/24, 2:45 PM

Yes. Values cached in local datacenters may be evicted earlier than the

cacheTtl

cacheTtl

, based on demand/other factors

anirudhOP•3/16/24, 2:46 PM

Got it, thanks for the clarification

AnotherAlex•3/18/24, 11:56 AM

Can you use cacheTtl on a list command? I think I understand how values are cached for individual keys but not sure about the list of keys in a namespace. Thanks.

AAnotherAlex Can you use cacheTtl on a list command? I think I understand how values are cach...

anirudhOP•3/19/24, 4:47 AM

The cache is for each key, the list command will only list all keys

anirudhOP•3/19/24, 4:48 AM

if a key has expired, it won't appear in the list

Marc•3/22/24, 2:12 PM

Anyone have any idea how we could possibly be hitting timeout errors accessing KV over the HTTP api? Is that a symptom of being rate limited or something? Our timeout is set to 15 seconds right now. Seems like a crazy long time for KV to respond. Anyone ever deal with something similar?

MMarc Anyone have any idea how we could possibly be hitting timeout errors accessing K...

Matt Silverlock•3/22/24, 3:49 PM

The HTTP API has a rate limit, but you should see a HTTP 429 when you hit it: https://developers.cloudflare.com/fundamentals/api/reference/limits/#:~:text=The%20global%20rate%20limit%20for,API%20key%2C%20or%20API%20token.

Cloudflare Docs

Limits · Cloudflare Fundamentals docs

The global rate limit for the Cloudflare API is 1200 requests per five minutes per user, and applies cumulatively regardless of whether the request is …

MMatt Silverlock The HTTP API has a rate limit, but you should see a HTTP 429 when you hit it: ht...

Marc•3/22/24, 4:50 PM

Yeah we've hit that a couple times. This is seemingly different since it's just timeouts? Also is there a better way I should interface with kv or is that just the rate limit I have to deal with?

MMarc Yeah we've hit that a couple times. This is seemingly different since it's just ...

Matt Silverlock•3/22/24, 5:20 PM

You won't have this issue via a Worker. There's a plan to improve how we do rate limits via the HTTP API, but a larger project.

squareclamp•3/22/24, 6:41 PM

I'm a bit confused on what should go in cache api vs kv? I've run some simple numbers and the numbers for read seem comparable. The specific use case, I'm looking at is caching database responses which seem to be ~ 100ms from my laptop.

squareclamp•3/22/24, 6:43 PM

I would really appreciate some help on how to better think about latencies between cache api, kv and d1.

Ssquareclamp I'm a bit confused on what should go in cache api vs kv? I've run some simple nu...

Chaika•3/22/24, 6:49 PM

cache api -> free but not guaranteed, no replication, always fast though

kv -> always cost money, but replicates, potentially cold/slow hits if that location hasn't pulled it before

d1 -> right now just a single durable object in a single location, cheaper then kv at access

Chaika•3/22/24, 6:50 PM

kv has two central stores (one in us, one in eu)

Chaika•3/22/24, 6:51 PM

I would probably say if your goal is just to cache d1 responses, use normal cache api instead of KV, as KV costs money and potentially has cold hits from one of the two stores, unless the query is very slow/its critical to be cached for most users

squareclamp•3/22/24, 6:53 PM

Between cache and kv, are the latencies expected to be similar if the data is present in pop? I'm assuming locations for KV are same as cache and D1 locations are lot fewer.

Ssquareclamp Between cache and kv, are the latencies expected to be similar if the data is pr...

Chaika•3/22/24, 7:02 PM

I think KV is just a bit slower for cached hits, but generally the same. I just tested and they're about the same with KV being maybe a few ms slower on average for cached hits.

I'm assuming locations for KV are same as cache and D1 locations are lot fewer.

KV has two central stores and can be cached in every location.
D1 is only in a few places (https://where.durableobjects.live/), where Durable Objects are, and once you create it, it's only in that single location. They're working on read replicas but nothing at the moment

Chaika•3/22/24, 7:06 PM

There's no rule saying you can't use all 3 either. If you have a really expensive query, could have a Worker on cron pushing it into KV, and then use BetterKV (https://flareutils.pages.dev/betterkv/old), which uses local Cache API + KV to bring your bill down.

Mmare arcis Does this pull immediately upon expiry, or upon a worker request when expired?

Isaac McFadyen•3/24/24, 2:56 AM

Worker request when expired.

Tony Leung•3/24/24, 8:52 AM

hi guys, is there any API to list KV Item with values or an API which is similar to BatchGetItem API in Dynamo? Currently I have to use list API to get keys and sequentially get their values which is really inconvenient. My current workaround is putting the item into

metadata

metadata

so it could be fetched during the list operation when the item data is small otherwise I still have to manually get each item out of KV.

x03•3/24/24, 5:41 PM

I want to store user sessions in KV, but am not sure how to approach it.

I need each session entry to have a unique UUID, but also have the user's ID as part of the key so that I can list a user's sessions.

doing

UID:SID

UID:SID

as the key lets me use prefix to list a user's sessions, but I cant query a session by just having the session ID (without the user id)

just doing

SID

SID

as the key lets me query sessions with just a session ID, but I dont know how to list a user's sessions.

x03•3/24/24, 5:45 PM

The only solution I see at the moment is just using

SID

SID

key and

UID

UID

value, and if I need to list a user's sessions I just list every session and filter it in the code itself. This seems a bit weird though cause if I have thousands of sessions this may slow things down

Xx03 I want to store user sessions in KV, but am not sure how to approach it. I need...

Hello, I’m Allie!•3/24/24, 6:15 PM

You could also use a Durable Object for each user?

HHello, I’m Allie!You could also use a Durable Object for each user?

x03•3/24/24, 6:21 PM

Wouldn't that be expensive?

x03•3/24/24, 6:21 PM

I need some efficient way of storing users and sessions. My current approach is D1 database for users and KV for sessions

Xx03 Wouldn't that be expensive?

Hello, I’m Allie!•3/24/24, 6:23 PM

It shouldn't be? DO compute is slightly more expensive than Workers on Standard, but if you are just using it for storage, then compute shouldn't be much of an issue

HHello, I’m Allie!It shouldn't be? DO compute is slightly more expensive than Workers on Standard,...

x03•3/24/24, 6:23 PM

Where would sessions be stored though?

x03•3/24/24, 6:23 PM

As part of the DO?

x03•3/24/24, 6:26 PM

Durable objects seems really complicated, are they really necessary for what I want to achieve?

Xx03 Where would sessions be stored though?

Hello, I’m Allie!•3/24/24, 6:28 PM

In DO Storage

Cloudflare Docs

Transactional Storage · Cloudflare Durable Objects docs

The Transactional Storage API allows you to achieve consistent key-value storage.

Xx03 Durable objects seems really complicated, are they really necessary for what I w...

Hello, I’m Allie!•3/24/24, 6:28 PM

Not necessarily, though going with KV will also mean that it may be hard to modify a session more than once a minute

HHello, I’m Allie!Not necessarily, though going with KV will also mean that it may be hard to modi...

x03•3/24/24, 6:29 PM

I dont really have a need to modify it once a minute. My main reason for using KV is because I need to fetch the session on EVERY request, so the speed of the fetch is really important. Also KV has expiration so its super easy to make sessions expire after a week

x03•3/24/24, 6:30 PM

I've never used durable objects and it seems like a really weird approach to storing state, especially since currently my users and apps and subscriptions are all nicely related in the database

Xx03 I dont really have a need to modify it once a minute. My main reason for using K...

Hello, I’m Allie!•3/24/24, 6:44 PM

Ok, then another option would be to have two types of records in KV:

SID

SID

, and

UID:SID

UID:SID

SID

SID

stores the actual session data, and

UID:SID

UID:SID

doesn't actually store anything, but just allows you to list through a user's Sessions

Hello, I’m Allie!•3/24/24, 6:44 PM

It's slightly more expensive, but in return it should be simpler to work with

HHello, I’m Allie!Ok, then another option would be to have two types of records in KV: `SID`, and ...

x03•3/24/24, 6:49 PM

I guess that will have to do, thanks for the suggestion.

Xx03 I dont really have a need to modify it once a minute. My main reason for using K...

Isaac McFadyen•3/26/24, 2:33 AM

Keep in mind that KV reads are $0.50/million [1] (and if you're doing a KV fetch on every request that can get expensive fast). I'd recommend something like BetterKV [2] if you're doing a KV read on every request.

[1] https://developers.cloudflare.com/workers/platform/pricing/#workers-kv
[2] https://flareutils.pages.dev/betterkv/

mr nooli•3/26/24, 2:28 PM

Question from a noob here, why would you use something like CF KV vs Upstash's Redis implementation? Cost wise alone (unless my maths isn't quite right) it's a hell of a lot cheaper going upstash?

Mmr nooli Question from a noob here, why would you use something like CF KV vs Upstash's R...

Chaika•3/26/24, 2:44 PM

Is it? I'm sure there are some advantages to other KV products, CF's KV is kind of narrow in scope, but it looks like Upstash Redis is $2 per million commands times the amount of replicas you have (KV is $0.50/mil), plus $0.03 per gb used of bandwidth (where CF is 0)

mr nooli•3/26/24, 2:44 PM

Okay! I massively misread that as 0.2 cents