Hey all quick question, I'm getting an email from CF that says KV is reaching 90% of the free tier l

Hey all quick question, I'm getting an email from CF that says KV is reaching 90% of the free tier limit. Looking at the dashboard though it says I have only done ~9k reads, which is only 9% of the free tier limit of 100,000 reads /day.

Am I not understanding the free tier limits?

Oh I think it may be the list request limits that is 1000k

MMrBithles Hey all quick question, I'm getting an email from CF that says KV is reaching 90...

Chaika•7/11/24, 12:44 PM

yea I would check the other limits: https://developers.cloudflare.com/kv/platform/pricing/, you only get 1,000 List reqs/day

Tthomasgauvin Yes, you would set the expirationTtl for each get, they keys will not be returne...

similardisaster•7/11/24, 7:39 PM

are missing keys cached as being null? and would writing to the key bust the cache? as in, if I read a key with cachettl of 30 days and it's non-existant, then immediatley after I write to that key, if I read the key again with cachettl of 30 days would the key come back as null until the 30 days has passed? or would the write be reflected within 60 seconds or so?

Chaika•7/11/24, 7:42 PM

https://developers.cloudflare.com/kv/reference/how-kv-works/

Negative lookups indicating that the key does not exist are also cached, so the same delay exists noticing a value is created as when a value is changed.

At the Cloudflare global network location at which changes are made, these changes are usually immediately visible. However, this is not guaranteed and therefore it is not advised to rely on this behaviour. In other global network locations changes may take up to 60 seconds or more to be visible as their cached versions of the data time-out.

Chaika•7/11/24, 7:46 PM

if I read the key again with cachettl of 30 days would the key come back as null until the 30 days has passed? or would the write be reflected within 60 seconds or so?

As for that, with KV 2.0 (or whatever they were calling the new version they announced and then rolled back) it was always doing background refreshes regardless of cache ttl and you'd see changes within ~60s or so, some docs may still reference that. But with KV 1.0, which we are currently on, it's up to cache ttl

Vikash•7/15/24, 6:23 AM

Is it recommended to re-use the KV storage in multiple workers? Or should be create multiple KV ?

For example I have one KV storage :

KVCACHE

KVCACHE

and re-using in 4-5 workers to cache the basic info using different keys. Is it ok?

or any performance improvement is expected if I create multiple cache storage for each worker as they are running on different region using smart placement binding with D1 APAC, EUROPE etc.

VVikash Is it recommended to re-use the KV storage in multiple workers? Or should be cre...

Hello, I’m Allie!•7/15/24, 7:18 AM

All KV instances share the same regions, so making a new namespace is mostly just a visual representation

HHello, I’m Allie!All KV instances share the same regions, so making a new namespace is mostly jus...

Vikash•7/15/24, 7:27 AM

So, re-using is good.

VVikash So, re-using is good.

Hello, I’m Allie!•7/15/24, 7:37 AM

Yeah, it’s fine

Coffee11•7/16/24, 6:47 AM

▲ [WARNING] Processing wrangler.toml configuration:

- Unexpected fields found in browser field: "kv_namespaces"

kv_namespaces = [
{ binding = "CRAWLER_SCREENSHOTS_KV", id = "test" },
{ binding = "CRAWLER_LINKS_KV", id = "test" }
]

what is the correct way?

akazwz•7/16/24, 8:53 AM

@Coffee11 [[kv_namespaces]]
binding = "CRAWLER_SCREENSHOTS_KV"
id = "test"

[[kv_namespaces]]
binding = "CRAWLER_LINKS_KV"
id = "test"

Disciple of Caoximeihao•7/17/24, 5:03 AM

help

Disciple of Caoximeihao•7/17/24, 7:32 AM

how to used kv in production

Disciple of Caoximeihao•7/17/24, 7:33 AM

in localhost

DDisciple of Caoximeihao how to used kv in production

Hello, I’m Allie!•7/17/24, 7:36 AM

Use

—remote

—remote

DDisciple of Caoximeihao help

akazwz•7/17/24, 7:56 AM

add preview_id will work. [[kv_namespaces]]
binding = "KV"
id = "xxxx"
preview_id = "xxxx"

Julian•7/18/24, 10:14 PM

Hello. Where or how can I generate the KV API Key and Authorization Token? I will use the API so yeah I need those things.

Julian•7/18/24, 10:14 PM

Using CF Dash I can only generate the API Key, not the Auth token

JJulian Hello. Where or how can I generate the KV API Key and Authorization Token? I wil...

Cyb3r-Jak3•7/18/24, 10:18 PM

There are two ways to use the API, either your email and global token or an API token (recommended).

Julian•7/18/24, 10:25 PM

So the API Token would be generated by going to the profile -> API Keys -> KV read and KV edit, right?

JJulian So the API Token would be generated by going to the profile -> API Keys -> KV re...

Cyb3r-Jak3•7/18/24, 10:41 PM

Correct

anhhoang•7/24/24, 8:03 AM

Hello, When I query the value of a key in Kv, the result returned is null even though the value is not null. I have tested the REST API, and when running it locally, the results are correct. However, the issue occurs only when the application is deployed and the API is called on the worker. Please help me !

Aanhhoang Hello, When I query the value of a key in Kv, the result returned is null even t...

Hard@Work•7/24/24, 8:05 AM

The KV API may have cached a null response. Try to give it a minute or two, and see if the issue persists

anhhoang•7/24/24, 8:09 AM

i have data in Kv before call api and it not works

anhhoang•7/24/24, 8:10 AM

I have been encountering this issue with this key for the past 5 hours.

anhhoang•7/24/24, 8:12 AM

It works correctly when I remote from local. The issue only occurs when deployed.

Aanhhoang It works correctly when I remote from local. The issue only occurs when deployed...

thomasgauvin•7/25/24, 3:11 PM

what is your configuration in wrangler.toml? you mention using

wrangler dev --remote

wrangler dev --remote

. That will be using the preview id of your kv_namespace configuration. If you want your deployed worker to use the same KV store, you would need to have the same namespace id for both id and preview_id. Is that the case?

ex wrangler.toml

[[kv_namespaces]]
binding = "assets"
id = "<some-id>"
preview_id = "<some-id-2>"

Stoke•7/26/24, 7:40 AM

Hi friends, a question re visibility of writes. The docs are not fully clear to me in this regard. When I make an update to a value in KV, does the visibility of the change depend on the

cacheTtl

cacheTtl

value? According to the docs, changes are immediately visible in the same location and take up to 60s in other locations. Is that also true for already cached values with high

cacheTtl

cacheTtl

? I would like to use

24h

24h

, but the (rare) changes need to be visible faster than that. A few minutes is acceptable.
I did some rough testing and it seems like

cacheTtl

cacheTtl

does not matter, but I just want to make sure. Thank you!

Yu•7/26/24, 10:39 AM

Hi team, regarding write visibility, I would like to know approximately how long it takes for global data synchronization. Right now, after writing to KV and then reading, I basically have to wait for an additional 20 seconds to obtain the data.

YYu Hi team, regarding write visibility, I would like to know approximately how long...

Hard@Work•7/26/24, 10:41 AM

Around 60 seconds by default, but if you set a

cacheTtl

cacheTtl

, then whatever value you set it to(minimum 60 seconds)

SStoke Hi friends, a question re visibility of writes. The docs are not fully clear to ...

Hard@Work•7/26/24, 10:42 AM

For global updates, there isn't really a good way to cache-bust them. Assuming that you read with the same

cacheTtl

cacheTtl

value every time, it would show stale values for up to 24 hours

Hard@Work•7/26/24, 10:42 AM

You could do background revalidation though, if that fits better?

HHard@Work You could do background revalidation though, if that fits better?

Stoke•7/26/24, 11:42 AM

Thanks for the clarification! What exactly do you mean with doing background revalidation? I was under the assumption that KV is doing that already? From https://developers.cloudflare.com/kv/reference/how-kv-works/: "reads are periodically revalidated in the background"

SStoke Thanks for the clarification! What exactly do you mean with doing background rev...

Hard@Work•7/26/24, 11:43 AM

Uh, that part may require an update? There was a thing called KV v2, which was supposed to do background revalidation, but iirc that was rolled back, and it doesn't happen anymore

Stoke•7/26/24, 11:43 AM

I read that plus the note about the "60s for consistency" as "background revalidation can take up to 60s"

Hard@Work•7/26/24, 11:43 AM

Though let me check

SStoke I read that plus the note about the "60s for consistency" as "background revalid...

thomasgauvin•7/26/24, 6:55 PM

KV does indeed perform revalidation at some point before the cacheTtl is elapsed. We'll continue keeping the docs up to date, but as of now, they are accurate.

SStoke Hi friends, a question re visibility of writes. The docs are not fully clear to ...

thomasgauvin•7/26/24, 7:15 PM

When you write to KV, the changes are immediately made to KV's origin stores. They are also immediately visible in the region where the KV write came through, regardless of cacheTtl. However, we recommend you do not rely on this behavior because in rare instances your request to your Worker may get routed to another location which will not have this change immediately visible. This is why we recommend using Durable Objects for atomic operations.

In other regions, it is possible that a stale value has been read & cached in a location before the write has been made. In this case, the change will only be visible in that location when the cache in that region elapses and is revalidated.

Marcus•7/27/24, 6:19 PM

Is it (still?) true that there's no way to move a KV namespace to another Cloudflare account besides writing a script to bulk-download every key/value pair from one and bulk-upload to another?

Marcus•7/27/24, 7:03 PM

(and that would consume 1 write per key, so on the free tier this would have to be spread over ~keys/1000keys/1000 days?)

Marcus•7/27/24, 7:26 PM

I see a bulk write endpoint, but it only takes JSON and our values are binary so that doesn't help

Walshy•7/27/24, 8:18 PM

you can provide base64

Walshy•7/27/24, 8:19 PM

https://developers.cloudflare.com/api/operations/workers-kv-namespace-write-multiple-key-value-pairs

Cloudflare API Documentation

Interact with Cloudflare's products and services via the Cloudflare API

Walshy•7/27/24, 8:19 PM

base64
boolean
Whether or not the server should base64 decode the value before storing it. Useful for writing values that wouldn't otherwise be valid JSON strings, such as images.

Marcus•7/27/24, 9:07 PM

ooh, interesting. I guess I could download one-by-one, convert to base64 locally, bulk upload. do you know if that endpoint counts as a single write re: the daily limits?

Walshy•7/27/24, 9:11 PM

https://developers.cloudflare.com/kv/platform/pricing/#frequently-asked-questions

Each key-value pair in the (bulk write) PUT request is counted as a single write, identical to how each call to PUT in the Workers API counts as a write. Writing 5,000 keys via the REST API incurs the same write costs as making 5,000 PUT calls in a Worker.

John Spurlock•7/28/24, 10:35 PM

nice! new KV storage and operations datasets available in the cf graphql api: kvOperationsAdaptiveGroupskvOperationsAdaptiveGroups and kvStorageAdaptiveGroupskvStorageAdaptiveGroups

was always a bit strange having to go the rest api for those

MMarcus ooh, interesting. I guess I could download one-by-one, convert to base64 locally...

Marcus•7/28/24, 11:05 PM

resulting (somewhat hacky) script, in case someone else needs to do this: https://github.com/marcustyphoon/cloudflare-kv-migrate

Tthomasgauvin When you write to KV, the changes are immediately made to KV's origin stores. Th...

Stoke•7/29/24, 8:46 AM

I see, thanks for the info. Are durable objects really a viable replacement for KV? To be a bit more concrete on the use case; let's imagine a link shortener service with 100k links (per month) and 20M clicks per month. My real/actual use case is close enough to this example scenario. I would store link id -> link URL as key -> value. The load is very read heavy with spikes of over 200 rps, but there are only very few updates to existing keys, let's say 1 per hour. I would traditionally use Redis (or similar) for such a use case, that's why I'm looking at KV. Using durable objects, this would be a single object for the entire map of links? Or one object per link (i.e. 100k objects per month)? Since duration is billed, would the latter mean that 100k objects per month are "active" indefinitely?

SStoke I see, thanks for the info. Are durable objects really a viable replacement for ...

Hard@Work•7/29/24, 8:50 AM

No, they are only billable when processing requests, even if they are still "active"

Stoke•7/29/24, 9:35 AM

Ok thanks. Every request activates for 30s, if I understand correctly. Makes sense. Then after reading through the docs I can see that the recommendationis to not use single global objects, so I think "object per link entity" would be the way to go here. I still struggle to wrap my head around why/how Durable Objects are the best fit for this use case. They're certainly cool and powerful, but I don't really need all the extra guarantees. All I'd really need is a bit of a narrower windows for eventual consistency in KV. I think not having a TTL and the first lookup being "a bit slow" are both real downsides for me, albeit both would be acceptable. I'm mainly worried about the complexity here... KV is very simple to grasp, whereas DO is quite complex conceptually, even when misused as a key-value store

Hey all quick question, I'm getting an email from CF that says KV is reaching 90% of the free tier l

Similar Threads

Similar Threads

Similar Threads