is ctx.waitUntil() suitable for making kv writes without adding latency (blocking) serving a respons

is ctx.waitUntil() suitable for making kv writes without adding latency (blocking) serving a response? Also, is there an estimate we can use for how much added latency reading 1 key from kv per request will add to an existing worker?

similardisasterOP•7/9/24, 7:33 PM

additionally, when we roll out there will be an initial influx of writes - any concerns with rolling out straight into up to 10k writes/second (mostly to unique keys)?

Hello, I’m Allie!•7/9/24, 7:37 PM

Question #1: Yes
#2: Not really, it depends where in the world you are writing from, since it writes to US/EU.
#3: 10k writes/s should be perfectly manageable assuming, as you pointed out, that it is spread across unique keys as much as possible

HHello, I’m Allie!Question #1: Yes #2: Not really, it depends where in the world you are writing f...

similardisasterOP•7/9/24, 7:39 PM

thank you!

similardisasterOP•7/10/24, 2:12 PM

I'm a bit confused on cacheTtl - where is the best place to set it? In each get? Each key is write once with an expirationTtl of 30 days (not even checking the value just checking the keys existence). Should I just do each get with a 30 day cacheTtl? Or would that result in keys returning a cached value even though they actually have since expired?

thomasgauvin•7/10/24, 3:26 PM

Yes, you would set the expirationTtl for each get, they keys will not be returned past expiration

MrBithles•7/11/24, 12:35 PM

Hey all quick question, I'm getting an email from CF that says KV is reaching 90% of the free tier limit. Looking at the dashboard though it says I have only done ~9k reads, which is only 9% of the free tier limit of 100,000 reads /day.

Am I not understanding the free tier limits?

Oh I think it may be the list request limits that is 1000k

MMrBithles Hey all quick question, I'm getting an email from CF that says KV is reaching 90...

Chaika•7/11/24, 12:44 PM

yea I would check the other limits: https://developers.cloudflare.com/kv/platform/pricing/, you only get 1,000 List reqs/day

Tthomasgauvin Yes, you would set the expirationTtl for each get, they keys will not be returne...

similardisasterOP•7/11/24, 7:39 PM

are missing keys cached as being null? and would writing to the key bust the cache? as in, if I read a key with cachettl of 30 days and it's non-existant, then immediatley after I write to that key, if I read the key again with cachettl of 30 days would the key come back as null until the 30 days has passed? or would the write be reflected within 60 seconds or so?

Chaika•7/11/24, 7:42 PM

https://developers.cloudflare.com/kv/reference/how-kv-works/

Negative lookups indicating that the key does not exist are also cached, so the same delay exists noticing a value is created as when a value is changed.

At the Cloudflare global network location at which changes are made, these changes are usually immediately visible. However, this is not guaranteed and therefore it is not advised to rely on this behaviour. In other global network locations changes may take up to 60 seconds or more to be visible as their cached versions of the data time-out.

Chaika•7/11/24, 7:46 PM

if I read the key again with cachettl of 30 days would the key come back as null until the 30 days has passed? or would the write be reflected within 60 seconds or so?

As for that, with KV 2.0 (or whatever they were calling the new version they announced and then rolled back) it was always doing background refreshes regardless of cache ttl and you'd see changes within ~60s or so, some docs may still reference that. But with KV 1.0, which we are currently on, it's up to cache ttl

Vikash•7/15/24, 6:23 AM

Is it recommended to re-use the KV storage in multiple workers? Or should be create multiple KV ?

For example I have one KV storage :

KVCACHE

KVCACHE

and re-using in 4-5 workers to cache the basic info using different keys. Is it ok?

or any performance improvement is expected if I create multiple cache storage for each worker as they are running on different region using smart placement binding with D1 APAC, EUROPE etc.

VVikash Is it recommended to re-use the KV storage in multiple workers? Or should be cre...

Hello, I’m Allie!•7/15/24, 7:18 AM

All KV instances share the same regions, so making a new namespace is mostly just a visual representation

HHello, I’m Allie!All KV instances share the same regions, so making a new namespace is mostly jus...

Vikash•7/15/24, 7:27 AM

So, re-using is good.

VVikash So, re-using is good.

Hello, I’m Allie!•7/15/24, 7:37 AM

Yeah, it’s fine

Coffee11•7/16/24, 6:47 AM

▲ [WARNING] Processing wrangler.toml configuration:

- Unexpected fields found in browser field: "kv_namespaces"

kv_namespaces = [
{ binding = "CRAWLER_SCREENSHOTS_KV", id = "test" },
{ binding = "CRAWLER_LINKS_KV", id = "test" }
]

what is the correct way?

akazwz•7/16/24, 8:53 AM

@Coffee11 [[kv_namespaces]]
binding = "CRAWLER_SCREENSHOTS_KV"
id = "test"

[[kv_namespaces]]
binding = "CRAWLER_LINKS_KV"
id = "test"

Disciple of Caoximeihao•7/17/24, 5:03 AM

help

Disciple of Caoximeihao•7/17/24, 7:32 AM

how to used kv in production

Disciple of Caoximeihao•7/17/24, 7:33 AM

in localhost

DDisciple of Caoximeihao how to used kv in production

Hello, I’m Allie!•7/17/24, 7:36 AM

Use

—remote

—remote

DDisciple of Caoximeihao help

akazwz•7/17/24, 7:56 AM

add preview_id will work. [[kv_namespaces]]
binding = "KV"
id = "xxxx"
preview_id = "xxxx"

Julian•7/18/24, 10:14 PM

Hello. Where or how can I generate the KV API Key and Authorization Token? I will use the API so yeah I need those things.

Julian•7/18/24, 10:14 PM

Using CF Dash I can only generate the API Key, not the Auth token

JJulian Hello. Where or how can I generate the KV API Key and Authorization Token? I wil...

Cyb3r-Jak3•7/18/24, 10:18 PM

There are two ways to use the API, either your email and global token or an API token (recommended).

Julian•7/18/24, 10:25 PM

So the API Token would be generated by going to the profile -> API Keys -> KV read and KV edit, right?

JJulian So the API Token would be generated by going to the profile -> API Keys -> KV re...

Cyb3r-Jak3•7/18/24, 10:41 PM

Correct

anhhoang•7/24/24, 8:03 AM

Hello, When I query the value of a key in Kv, the result returned is null even though the value is not null. I have tested the REST API, and when running it locally, the results are correct. However, the issue occurs only when the application is deployed and the API is called on the worker. Please help me !

Aanhhoang Hello, When I query the value of a key in Kv, the result returned is null even t...

Hard@Work•7/24/24, 8:05 AM

The KV API may have cached a null response. Try to give it a minute or two, and see if the issue persists

anhhoang•7/24/24, 8:09 AM

i have data in Kv before call api and it not works

anhhoang•7/24/24, 8:10 AM

I have been encountering this issue with this key for the past 5 hours.

anhhoang•7/24/24, 8:12 AM

It works correctly when I remote from local. The issue only occurs when deployed.

Aanhhoang It works correctly when I remote from local. The issue only occurs when deployed...

thomasgauvin•7/25/24, 3:11 PM

what is your configuration in wrangler.toml? you mention using

wrangler dev --remote

wrangler dev --remote

. That will be using the preview id of your kv_namespace configuration. If you want your deployed worker to use the same KV store, you would need to have the same namespace id for both id and preview_id. Is that the case?

ex wrangler.toml

[[kv_namespaces]]
binding = "assets"
id = "<some-id>"
preview_id = "<some-id-2>"

Stoke•7/26/24, 7:40 AM

Hi friends, a question re visibility of writes. The docs are not fully clear to me in this regard. When I make an update to a value in KV, does the visibility of the change depend on the

cacheTtl

cacheTtl

value? According to the docs, changes are immediately visible in the same location and take up to 60s in other locations. Is that also true for already cached values with high

cacheTtl

cacheTtl

? I would like to use

24h

24h

, but the (rare) changes need to be visible faster than that. A few minutes is acceptable.
I did some rough testing and it seems like

cacheTtl

cacheTtl

does not matter, but I just want to make sure. Thank you!

Yu•7/26/24, 10:39 AM

Hi team, regarding write visibility, I would like to know approximately how long it takes for global data synchronization. Right now, after writing to KV and then reading, I basically have to wait for an additional 20 seconds to obtain the data.

YYu Hi team, regarding write visibility, I would like to know approximately how long...

Hard@Work•7/26/24, 10:41 AM

Around 60 seconds by default, but if you set a

cacheTtl

cacheTtl

, then whatever value you set it to(minimum 60 seconds)

SStoke Hi friends, a question re visibility of writes. The docs are not fully clear to ...

Hard@Work•7/26/24, 10:42 AM

For global updates, there isn't really a good way to cache-bust them. Assuming that you read with the same

cacheTtl

cacheTtl

value every time, it would show stale values for up to 24 hours

Hard@Work•7/26/24, 10:42 AM

You could do background revalidation though, if that fits better?

HHard@Work You could do background revalidation though, if that fits better?

Stoke•7/26/24, 11:42 AM

Thanks for the clarification! What exactly do you mean with doing background revalidation? I was under the assumption that KV is doing that already? From https://developers.cloudflare.com/kv/reference/how-kv-works/: "reads are periodically revalidated in the background"

SStoke Thanks for the clarification! What exactly do you mean with doing background rev...

Hard@Work•7/26/24, 11:43 AM

Uh, that part may require an update? There was a thing called KV v2, which was supposed to do background revalidation, but iirc that was rolled back, and it doesn't happen anymore

Stoke•7/26/24, 11:43 AM

I read that plus the note about the "60s for consistency" as "background revalidation can take up to 60s"

Hard@Work•7/26/24, 11:43 AM

Though let me check

SStoke I read that plus the note about the "60s for consistency" as "background revalid...

thomasgauvin•7/26/24, 6:55 PM

KV does indeed perform revalidation at some point before the cacheTtl is elapsed. We'll continue keeping the docs up to date, but as of now, they are accurate.

SStoke Hi friends, a question re visibility of writes. The docs are not fully clear to ...

thomasgauvin•7/26/24, 7:15 PM

When you write to KV, the changes are immediately made to KV's origin stores. They are also immediately visible in the region where the KV write came through, regardless of cacheTtl. However, we recommend you do not rely on this behavior because in rare instances your request to your Worker may get routed to another location which will not have this change immediately visible. This is why we recommend using Durable Objects for atomic operations.

In other regions, it is possible that a stale value has been read & cached in a location before the write has been made. In this case, the change will only be visible in that location when the cache in that region elapses and is revalidated.

Marcus•7/27/24, 6:19 PM

Is it (still?) true that there's no way to move a KV namespace to another Cloudflare account besides writing a script to bulk-download every key/value pair from one and bulk-upload to another?

Marcus•7/27/24, 7:03 PM

(and that would consume 1 write per key, so on the free tier this would have to be spread over ~

keys/1000

keys/1000

days?)

Marcus•7/27/24, 7:26 PM

I see a bulk write endpoint, but it only takes JSON and our values are binary so that doesn't help

Walshy•7/27/24, 8:18 PM

you can provide base64

Walshy•7/27/24, 8:19 PM

https://developers.cloudflare.com/api/operations/workers-kv-namespace-write-multiple-key-value-pairs

Cloudflare API Documentation

Interact with Cloudflare's products and services via the Cloudflare API

Walshy•7/27/24, 8:19 PM

base64
boolean
Whether or not the server should base64 decode the value before storing it. Useful for writing values that wouldn't otherwise be valid JSON strings, such as images.

Marcus•7/27/24, 9:07 PM

ooh, interesting. I guess I could download one-by-one, convert to base64 locally, bulk upload. do you know if that endpoint counts as a single write re: the daily limits?

is ctx.waitUntil() suitable for making kv writes without adding latency (blocking) serving a respons

Similar Threads

Similar Threads

Similar Threads