oh wow it did the trick! Why? so does this mean that on the wrangler toml file the kv namespace sho

oh wow it did the trick!

Why? so does this mean that on the wrangler toml file the kv namespace should be declared above d1 database?

DDenver James Duran oh wow it did the trick! Why? so does this mean that on the wrangler toml file ...

Hard@Work•9/13/24, 1:15 PM

TOML Shenanigans. Basically, when you add a table declaration(

[placement]

[placement]

) or a table array declaration(

[[d1_databases]]

[[d1_databases]]

), everything until the next declaration is assumed to be a part of that table

Hard@Work•9/13/24, 1:15 PM

This is why I no longer use

toml

toml

for personal projects

Denver James DuranOP•9/13/24, 1:21 PM

that makes sense! Hmm how do we prevent that?

also, isn't wrangler.toml required in deploying apps in cloudflare?

DDenver James Duran that makes sense! Hmm how do we prevent that? also, isn't wrangler.toml require...

Hard@Work•9/13/24, 1:23 PM

I use

wrangler.json

wrangler.json

. It is supported for Workers, but not for Pages

NNoil My application accesses files from the KV-store and have been running into the i...

Noil•9/16/24, 12:56 PM

Any possible answer to why files in the KV store can be unfindable? could it be the caching or something else?

NNoil Any possible answer to why files in the KV store can be unfindable? could it be ...

Zack•9/16/24, 1:18 PM

If you make a request for a key that doesn't yet exist, the not found response will be cached for the same TTL as if it were found. Default 60 seconds iirc, but can be overridden

ZZack If you make a request for a key that doesn't yet exist, the not found response w...

Hard@Work•9/16/24, 1:19 PM

Note here that it can be raised above 60 seconds, but not lowered

TigersWay•9/17/24, 8:47 AM

For the first time I got the message "Daily request limit exceeded for Cloudflare Workers"
This worker have an average (and pretty stable) of 250/350 writes every day, and I just checked on the dashboard without seeing anything strange. How should I best proceed?
Thank you!

nejc•9/17/24, 9:03 AM

Hi all, when I am using list method and cursor for pagination (https://developers.cloudflare.com/kv/api/list-keys/#list-method). What happens if I have 1 milion keys and when I get a page I trigger delete of that keys and then use cursor to go to the second page? Would cursor have an old list so it will continue serving correct pages even though that I am deleting keys in the background?

Cloudflare Docs

List keys | Cloudflare Workers KV

To list all the keys in your KV namespace, call the list() method of the KV binding on any KV namespace you have bound to your Worker code:

thomasgauvin•9/17/24, 9:46 AM

Hey folks! We just shipped the new Analytics GraphQL API and Dashboard UI for Workers KV

Now, you can get per-namespace analytics and more details such as cache status and latency information, and it's standardized on the same GraphQL API layer as other Cloudflare products.

Dashboard UI: You can try this one directly from the dashboard, just go to one of your KV namespaces and see the new metrics UI.

GraphQL API: You can now get your KV analytics directly from the standard GraphQL API (sample queries and how to configure your requests here: https://developers.cloudflare.com/kv/observability/metrics-analytics/)

Check it out in action (looks even better in person)!
https://developers.cloudflare.com/kv/platform/changelog/#2024-08-08

nejc•9/17/24, 10:00 AM

@thomasgauvin really awesome view and great information source. One thing that I noticed is that if KV is big it doesn't show KV count

Marcus•9/17/24, 9:51 PM

Heya—I assume probably not, but is there a way to tweak the usage limit email threshold? I'm getting daily "you have used 50% of the free tier!" emails... from a cron job that reliably uses just over half of the free tier allotment :D

TTigersWay For the first time I got the message "Daily request limit exceeded for Cloudflar...

Marcus•9/17/24, 10:56 PM

(er, wait... no, it doesn't, it uses like 1/3. seems like essentially a duplicate of this thread, I'll stay tuned on that)

Matt Silverlock•9/18/24, 12:32 PM

https://x.com/thomasgauvin/status/1836376211771081188?s=46

EXE KL•9/19/24, 1:28 PM

Does KV have a feature for checking existing keys on write? By default it does not but is there a feature that can be turned on to enable such a thing?

EEXE KL Does KV have a feature for checking existing keys on write? By default it does n...

Hard@Work•9/19/24, 1:30 PM

No, and no

HHard@Work No, and no

EXE KL•9/19/24, 1:34 PM

I knew but I had to ask anyway. Can you clarify if I'm being paranoid then. I want to store magic tokens(that expire after an hour) and although I've gone to great lengths to make the token unique, there is always that statistically improbable chance that a duplicate token is created within that ttl window. I'm I crazy for trying to check for duplicates when the chance for that is as probable as the sun going out?

EXE KL•9/19/24, 1:36 PM

It also could just be the way I'm implementing the magic token system that invites uncertainties.

EEXE KL I knew but I had to ask anyway. Can you clarify if I'm being paranoid then. I wa...

Hard@Work•9/19/24, 1:39 PM

You could also just use DOs?

Hard@Work•9/19/24, 1:39 PM

Then have it fail the write if it already exists?

HHard@Work You could also just use DOs?

EXE KL•9/19/24, 1:41 PM

Forgive me, what is a DO?

EEXE KL Forgive me, what is a DO?

Hard@Work•9/19/24, 1:42 PM

https://developers.cloudflare.com/durable-objects/#_top

Cloudflare Docs

Overview | Cloudflare Durable Objects docs

Durable Objects provide a powerful API for coordinating multiple clients or users, each with private, transactional and strongly consistent storage attached.

HHard@Work https://developers.cloudflare.com/durable-objects/#_top

EXE KL•9/19/24, 1:44 PM

Ah okay, so this is available through the transactional storage api?

Hard@Work•9/19/24, 1:44 PM

Or well, each DO has a separate KV Store

HHard@Work Or well, each DO has a separate KV Store

EXE KL•9/19/24, 1:46 PM

Thanks, I'll take a proper read through it.

EEXE KL Thanks, I'll take a proper read through it.

thomasgauvin•9/19/24, 8:13 PM

This seems like an optimization for an extremely unlikely use case. How many keys do you plan on storing and is it anywhere near risking hash collision?

You could also reduce probably by making a read first, and writing it if it doesn't exist. It won't deterministically guarantee that you aren't overwriting, because it may have been written/propagated to KV since the read, but it wouldn't hurt in reducing chances of collision

If you need stronger unique guarantees, DO/D1 may be better options

Tthomasgauvin This seems like an optimization for an extremely unlikely use case. How many key...

EXE KL•9/19/24, 8:35 PM

My plan is to reuse this authentication and session system whenever I can when making Sveltekit projects so I wanted to make something robust enough to handle a large number of users authenticating themselves frequently. I've considered reading before the writing but as you mentioned it only reduces the chances on top of an extra network call. D1 was the only real solution for me before learning about DO.

EXE KL•9/19/24, 8:44 PM

As for the use cases. I probably may never generate enough magic tokens to increase the odds of a duplication within a period of time as the size of the tokens is 64 chars generated by the crypto module. I think I just wanted a solution to a hypothetical problem.:NotLikeThis:

EEXE KL My plan is to reuse this authentication and session system whenever I can when m...

hfcRed•9/20/24, 11:09 AM

Make a separate KV for every project and you should be fine

hfcRed•9/20/24, 11:10 AM

If you really want to avoid collision at all cost, then you can append a unix timestamp to the token

hfcRed•9/20/24, 11:11 AM

If you manage to generate two identical 64 char tokens within a single second you would probably be the luckiest human to have ever existed. Heck, even getting a single collision at all would make you the luckiest person of all time

hfcRed•9/20/24, 11:12 AM

You could also generate a UUID7 which has a granularity of up to 50 nanoseconds

EEXE KL My plan is to reuse this authentication and session system whenever I can when m...

thomasgauvin•9/20/24, 12:27 PM

You should probably store your information in D1, then optimize with KV (ie once a user logs in, check hash against db hashed, but then store tokens in kv and use that for auth validation)

Anton•9/20/24, 4:04 PM

Why key-value data storage is so expensive comparing to other resources?

KV read price - $0.50/million
Durable objects (that sounds a lot more complex, "permanent" and expensive) - $0.15/million
D1 database - $0.001 / million rows reads

Why everything else, where I can store some data is cheaper? Isn't kv - a cheap, temporary, cache storage?

AAnton Why key-value data storage is so expensive comparing to other resources? KV rea...

Matt Silverlock•9/20/24, 4:13 PM

KV isn't temporary - KV storage is persistent: https://developers.cloudflare.com/kv/

Cloudflare Docs

Cloudflare Workers KV | Cloudflare Workers KV

Workers KV is a data storage that allows you to store and retrieve data globally. With Workers KV, you can build dynamic and performant APIs and websites that support high read volumes with low latency.

AAnton Why key-value data storage is so expensive comparing to other resources? KV rea...

thomasgauvin•9/20/24, 4:24 PM

And to add on to Matt's response, each key-value pair in KV can handle a much higher throughput than individual rows in D1

Anton•9/20/24, 4:31 PM

Hmm, ok. Is KV the best tool to store and read some cache data? Like I have a complex worker function. During its work, I have to hit several 3rd party API. And I don't want to call them on every worker execution. I want to temporary cache data. On regular server I would use redis or a simple in-memory cache. What about worker? What should I use here? KV?

AAnton Hmm, ok. Is KV the best tool to store and read some cache data? Like I have a co...

thomasgauvin•9/20/24, 4:39 PM

Yes, KV is the tool for this. there's cache api https://developers.cloudflare.com/workers/runtime-apis/cache/ but that is definitely an optimization and might get evicted from cache

AAnton Hmm, ok. Is KV the best tool to store and read some cache data? Like I have a co...

JacobMGEvans•9/21/24, 10:26 PM

Definitely something you can do, I use KV to store Tailwind off the CDN (for funsies), I try weird uses of Cloudflare products in this project all the time just to see if I can and if there is other novel things you can do https://github.com/JacobMGEvans/JacobMGEvans/blob/main/src/index.ts

JacobMGEvans•9/21/24, 10:28 PM

Additionally you can use the Worker Cron to run occasionally and purge anything thats older than a certain age you give them, since you mentioned temporary

Ashish Rawat•9/23/24, 5:50 PM

I think KV is good for storing permission level data on top of D1. Like checking for permissions of a user on each api call before accessing D1 or other db.

TigersWay•9/25/24, 7:41 AM

Still receiving the message "Daily request limit exceeded for Cloudflare Workers" for a "1000 put operations" which is in fact of 226 right now!
More than a week has passed, shouldn't we have more information?

Bọc Link VN•9/25/24, 10:35 AM

Hello every one ! Have anyone know how to rename the KV Namespace?

Bọc Link VN•9/25/24, 10:35 AM

I remember that it could be renamed in the past but now it seems not to work anymore

TTigersWay Still receiving the message "Daily request limit exceeded for Cloudflare Workers...

thomasgauvin•9/25/24, 1:53 PM

Thanks for bringing it up again. This is our top priority fix at the moment. A query that is being made to alert on free quotas is misreporting quotas, so we’re fixing that. I will keep this message thread updated. Note that the query resets every day so you may not encounter this all the time

JJacobMGEvans Additionally you can use the Worker Cron to run occasionally and purge anything ...

thipperz•9/25/24, 11:59 PM

That's interesting, how do you check the current age of a KV key? I would like to set a high cacheTtl for a key while also being able to show a 'refresh' button if the current value is cached but also 3 days old for example

Tthomasgauvin Thanks for bringing it up again. This is our top priority fix at the moment. A q...

TigersWay•9/26/24, 2:13 AM

I somehow understand it needs time to fix it, but shouldn't WE have a test period for this kind of things?

Tthipperz That's interesting, how do you check the current age of a KV key? I would like t...

JacobMGEvans•9/26/24, 2:21 AM

I'd have to look at how I've done it before but my knee jerk (from my phone) reaction without looking is probably timestamp in the Metadata, maybe something like this

async function setKVValueWithTimestamp(key, value, cacheTtl) {
  const timestamp = Date.now();
  const data = JSON.stringify({ value, timestamp });
  await KV_NAMESPACE.put(key, data, { expirationTtl: cacheTtl });
}

async function setKVValueWithTimestamp(key, value, cacheTtl) {
  const timestamp = Date.now();
  const data = JSON.stringify({ value, timestamp });
  await KV_NAMESPACE.put(key, data, { expirationTtl: cacheTtl });
}

JJacobMGEvans I'd have to look at how I've done it before but my knee jerk (from my phone) rea...

thipperz•9/26/24, 3:28 AM

Ahh, sure! Thank you! That will do it

Matt Silverlock•9/26/24, 3:11 PM

In case you missed it — here's the details on the latest Workers KV performance improvements that just shipped! https://blog.cloudflare.com/faster-workers-kv/ @Workers KV

The Cloudflare Blog

We made Workers KV up to 3x faster — here’s the data

Today, we’re announcing up to 3x faster KV hot reads, with all KV operations faster by up to 20ms. And we want to pull back the curtain and show you how we did it.

async function setKVValueWithTimestamp(key, value, cacheTtl) { const timestamp = Date.now(); const data = JSON.stringify({ value, timestamp }); await KV_NAMESPACE.put(key, data, { expirationTtl: cacheTtl }); }

oh wow it did the trick! Why? so does this mean that on the wrangler toml file the kv namespace sho

Similar Threads

Similar Threads

Similar Threads