1. It may work in a lot of cases, but there's no guarantee because KV is eventually consistent. 2. T

It may work in a lot of cases, but there's no guarantee because KV is eventually consistent.
The limit for writing to a single key is once per second. The system will error if you try to do it more frequently than that. While it's not a bad practice, if you have a lot of write volume overall (lots of different keys), we might start to rate throttle you (it's unlikely you'll be doing enough volume, but just keep that in mind).

beelzabubOP•7/14/23, 1:21 PM

Not at this time, no. The system auto-purges on writes from upper tiers (https://blog.cloudflare.com/faster-workers-kv-architecture/). We've talked about it, but our current thinking is to work on exposing a tunable refreshTtlrefreshTtl parameter to .get.get that lets you set a lower consistency value. However, there's a bunch of stuff we have to do under the hood to support that.

Bbeelzabub Not at this time, no. The system auto-purges on writes from upper tiers (https:/...

Hello, I’m Allie!•7/14/23, 2:05 PM

Any plans to add a way to reset expiration when getting a value?

HHello, I’m Allie!Any plans to add a way to reset expiration when getting a value?

beelzabubOP•7/14/23, 2:31 PM

What does this mean?

Bbeelzabub What does this mean?

Hello, I’m Allie!•7/14/23, 2:33 PM

Some way of adding

expiration

expiration

expirationTtl

expirationTtl

to a

get()

get()

call, so you could have, as an example, a key that expires 30 minutes after the last read

HHello, I’m Allie!Some way of adding `expiration`/`expirationTtl` to a `get()` call, so you could ...

beelzabubOP•7/14/23, 2:35 PM

Changing the expiration requires a write and having read operations secretly write isn't a good idea for multiple reasons.

Bbeelzabub Changing the expiration requires a write and having read operations secretly wri...

Hello, I’m Allie!•7/14/23, 2:37 PM

No, I mean that it would be explicit, like

get(“someKey”, {expirationTtl: 1800});

get(“someKey”, {expirationTtl: 1800});

. And it seems like overkill to have to rewrite a key every time to reset the expiration timer…

Hello, I’m Allie!•7/14/23, 2:38 PM

So a get would never be implicitly be making any changes

HHello, I’m Allie!No, I mean that it would be explicit, like `get(“someKey”, {expirationTtl: 1800}...

beelzabubOP•7/14/23, 2:48 PM

From the perspective of the system, it boils down to a HTTP GET. That it internally has custom parameters to say "but actually do a write" doesn't matter so much - you're asking for a read operation to secretly mutate.

The challenge with such a proposal is that the entire point of GETs is to be cached & not hit the underlying storage. If you specify the

expirationTtl

expirationTtl

, that defeats the purpose since you need to hit the storage. And what happens if we fail to update the TTL is that an error? If yes, then we'd have to wait for the response from the storage & it's not cacheable. It would have to be no which means there's no way to signal to the user that we failed to update the TTL.

If you need this behavior today, you could do something like:

const value = await get("somKey")
ctx.waitUntil(put("somKey", value))

const value = await get("somKey")
ctx.waitUntil(put("somKey", value))

Of course this is race condition prone and would need conditional uploads (not implemented at this time) or mediation through something like a DO.

Bbeelzabub 1. It may work in a lot of cases, but there's no guarantee because KV is eventua...

zehawk•7/14/23, 3:22 PM

Got it. Is there any service in the Cloudflare offerings that can be used to create a debouncer in front of KV writes.

Zzehawk Got it. Is there any service in the Cloudflare offerings that can be used to cre...

beelzabubOP•7/14/23, 3:24 PM

Durable Objects. A common mechanism is to use DOs as the "ground truth" for storage operations & KV is the read path in front.

Bbeelzabub Durable Objects. A common mechanism is to use DOs as the "ground truth" for stor...

zehawk•7/14/23, 3:30 PM

Yup, I'm already using DOs for suitable use cases - multi-location sync / multi-user sync etc. But there's a lot of complexity etc involved with DO, whereas KV is very clean. So I'm wondering whether there is a way to debounce writes to KV. I though queues might be a way, but I dont suppose that would have such a feature either.

Zzehawk Yup, I'm already using DOs for suitable use cases - multi-location sync / multi-...

beelzabubOP•7/14/23, 3:31 PM

By debounce you mean to make sure the value is read/modify/write updated atomically in a safe way? Or something else

Bbeelzabub By debounce you mean to make sure the value is read/modify/write updated atomica...

zehawk•7/14/23, 3:41 PM

Just a simple way of reducing multiple writes to the same key to the last write once there is a pause. Eg
edit #1, save #1 - this produces a KV write
edit #2, save #2 - this produces a KV write
.
.
.
edit #5, save #5 - this produces a KV write, all within few seconds.
<pause>
A debouncer would just take the 5th KV write, since earlier superseded by #5. No read being done in this set of ops.

Zzehawk Just a simple way of reducing multiple writes to the same key to the last write ...

beelzabubOP•7/14/23, 6:33 PM

Ah. Queues is probably a nicer mechanism than DOs, although I don't believe it has any ordering guarantees which make things hard.

Unsmart•7/14/23, 6:38 PM

Queues also doesnt really have any state so it wouldnt know when the last write to a specific key happened to debounce it properly. DOs would realistically be the only option to debounce writes like that.

Would also limit you to writing 128KB values unless you used multiple DO keys to save the value and then do a list or something

Unsmart•7/14/23, 6:42 PM

If you are ok with the slight chance of losing some data I feel like the easiest option might be to just block the request until at least 1 second from the last write and if you have many requests come in a single second do a fake success return for the previous write and then only the latest value will be waiting for 1 second to then write to KV

zehawk•7/14/23, 7:02 PM

Certainly DO can be used - I'm already using it as a debouncer in some cases by setting and canceling+setting the alarm on each update, and the last set alarm will finally trigger. Thats how I handle a stream of actions generated within the DO, eg notifications. But was looking for a simpler more elegant way to do this with KV.

Zzehawk Certainly DO can be used - I'm already using it as a debouncer in some cases by ...

beelzabubOP•7/14/23, 7:03 PM

Unfortunately not. You need to put a FIFO queue in front. Since queues doesn't yet have a FIFO mode, you'd need to roll it manually via a DO.

beelzabubOP•7/14/23, 10:49 PM

@HardAtWork you maintain the BetterKV project, right?

Bbeelzabub @HardAtWork you maintain the BetterKV project, right?

Hello, I’m Allie!•7/14/23, 10:53 PM

Yeah, I do

HHello, I’m Allie!Yeah, I do

beelzabubOP•7/14/23, 10:53 PM

I think there's a performance issue. Do you want me to file a formal ticket on GitHub or just describe it briefly here?

Bbeelzabub I think there's a performance issue. Do you want me to file a formal ticket on G...

Hello, I’m Allie!•7/14/23, 10:54 PM

Uh oh. You can send it here, then I’ll add it to GH and credit you there

Hello, I’m Allie!•7/14/23, 10:54 PM

Or whichever way you prefer

Hello, I’m Allie!•7/14/23, 10:55 PM

I need to add formalizing contributions to Flareutils to my evergrowing todo list…

beelzabubOP•7/14/23, 11:00 PM

Basically the cacheTTL you use for the fronting cache is the same as the cacheTTL for KV. This means that every time you read from KV it's a cold read whereas KV takes great pains to make sure that you never see cold reads for frequently accessed keys.

The latest version of the PR isn't building, but I'm working on docs providing guidance:
https://vlovich-kv-advanced-guide.cloudflare-docs-7ou.pages.dev/workers/learning/advanced-kv-guide/#avoid-hand-rolling-cache-in-front-of-workers-kv

I think a good change to BetterKV that follows the recommendation while cutting costs is:

Set the cache TTL in the fronting cache to ~40-50 seconds (regardless of the cacheTTL the user is requesting from KV).
Have 1% of cache hits do a KV.get anyway in a wait until (+ update your cache with the updated value & extend the 40-50 second deadline out further).

Advanced Workers KV Topics · Cloudflare Workers docs

This guide provides best practices for optimizing Workers KV latency as well as covering advanced tricks
that our customers sometimes employ for their …

beelzabubOP•7/14/23, 11:01 PM

Also, please add a note saying that the fronting cache is incompatible with future KV improvements to allow reducing the the refreshTTL < 60s.

Original message was deleted

beelzabubOP•7/14/23, 11:02 PM

This change would achieve both goals.

beelzabubOP•7/14/23, 11:03 PM

Note that this change also saves you more money. Because you're sampling at 1%, you will have a small number of requests that end up going through to KV & updating the value you have cached. The current behavior has a stampeding herd every 1 minute where ALL requests suddenly go to KV.

beelzabubOP•7/14/23, 11:05 PM

You can make the sampling threshold and the fronting cache duration tunable but cap the fronting cache duration at 50-55s at most. I would also make sure to call out in the docs that customers should set long cacheTTL as described in that document.

beelzabubOP•7/14/23, 11:07 PM

And ultimately my goal is for there to be no pricing difference of putting the Cache API in front, but we're still a ways away from realizing that vision. We're working on making KV faster than the cache API first.

beelzabubOP•7/14/23, 11:09 PM

My goal is to win the bet & get Sid (TL for pages) to shave his head bald if we hit certain performance milestones

beelzabubOP•7/14/23, 11:09 PM

(We have a grafana panel tracking that)

Bbeelzabub My goal is to win the bet & get Sid (TL for pages) to shave his head bald if we ...

James•7/14/23, 11:10 PM

If this happens, Sid is obligated to come to our next meetup bald

Hello, I’m Allie!•7/14/23, 11:11 PM

Very good point. Skye is correct that performance is not the #1 priority for BetterKV, pricing is, but I do also definitely see your point. I’ll probably add your suggestions as a performance mode that people can enable

HHello, I’m Allie!Very good point. Skye is correct that performance is not the #1 priority for Bet...

beelzabubOP•7/14/23, 11:13 PM

It's also net a cost savings. Would you be open to making it the default & the user needing to turn it off? Even if you don't want to do the probabilistic piece, can you shorten the cacheTTL so that it's under what you give KV? Otherwise, BetterKV is not really different than cache+R2.

Bbeelzabub My goal is to win the bet & get Sid (TL for pages) to shave his head bald if we ...

Walshy•7/14/23, 11:13 PM

I can't wait for this

Bbeelzabub It's also net a cost savings. Would you be open to making it the default & the u...

Hello, I’m Allie!•7/14/23, 11:16 PM

Qq, what’s your username on Discord, so I can credit you for the changes?

HHello, I’m Allie!Qq, what’s your username on Discord, so I can credit you for the changes?

beelzabubOP•7/14/23, 11:16 PM

vlovich

beelzabubOP•7/14/23, 11:17 PM

Oh that's my github username

beelzabubOP•7/14/23, 11:17 PM

beelzabub#0369 on Discord

Hello, I’m Allie!•7/14/23, 11:21 PM

Wait

Hello, I’m Allie!•7/14/23, 11:21 PM

No, I meant GH

Hello, I’m Allie!•7/14/23, 11:21 PM

I guess I’m just brain-tired from the heat…

Hello, I’m Allie!•7/14/23, 11:21 PM

beelzabubOP•7/14/23, 11:23 PM

But yeah, being more likely to refreshing the value as you approach the expriy end date is a significant cost saving measure. Counter-intuitive but true. It's how KV has to operate under the hood for similar reasons. We technically use a bounded exponentially increasing algorithm where after 5s there's an infinitesimaly small probability of a refresh that doubles with every passing second up to 100% at something like 45s or 50s at which point every read has a 100% probability of causing a background refresh. That way only a small subset of requests end up triggering a refresh while the rest just always serve from the cache without any stampeding herd. We have other tricks we're layering in as well, but they're not available externally yet.

Bbeelzabub Basically the cacheTTL you use for the fronting cache is the same as the cacheTT...

zehawk•7/15/23, 8:56 AM

How old is this writeup? I did not even know that putting cache in front of KV was a thing at some point, and from the time I started using (~2 years now), I always assumed KV is parallel to Cache. In other words, since KV is already at the edge, what's the point of cache in front of a cache?

Zzehawk How old is this writeup? I did not even know that putting cache in front of KV w...

Unsmart•7/15/23, 11:36 AM

To reduce billing since KV cache hits are still billed but the cache api is free

Bbeelzabub Basically the cacheTTL you use for the fronting cache is the same as the cacheTT...

DaniFoldi•7/15/23, 11:48 AM

Now that you have exposed the cache status of reads, this becomes a lot easier to implement - previously when I thought about a probabilistic update, using the time the read took as an indicator for hit/miss was rather messy, so thanks for implementing MeowHeartCloudflare

DaniFoldi•7/15/23, 11:49 AM

(one small issue I noticed with the docs PR, the ToC seems to cut off at some point)

1. It may work in a lot of cases, but there's no guarantee because KV is eventually consistent. 2. T

Similar Threads

Similar Threads

Similar Threads