My application accesses files from the KV-store and have been running into the issue that specifc fi

M

Mozzy•9/11/24, 5:45 PM

What's the highest cacheTtl allowed for

.get()

.get()

?

M

Mozzy•9/11/24, 6:01 PM

Also, editing KV entries in the dashboard is a horrendous experience. Editing a single KV key seems to refresh the entire view for about 5 seconds, preventing me from editing others during that time.

Looking at the requests, it's refetching every single visible entry once I change one of them

Ssilentdevnull There away to delete all records from a kv database or do I just have to do the...

M

Mozzy•9/12/24, 7:53 PM

Can always delete the database I suppose

MMozzy Also, editing KV entries in the dashboard is a horrendous experience. Editing a ...

T

thomasgauvin•9/12/24, 11:25 PM

Thanks for feedback, noting this down as something to fix

M

Mozzy•9/12/24, 11:26 PM

Awesome

Ssilentdevnull There away to delete all records from a kv database or do I just have to do the...

T

thomasgauvin•9/12/24, 11:26 PM

Like Mozzy said, delete the database or delete them one by one. What’s the context?

D

Denver James Duran•9/13/24, 12:37 PM

Hello team, just a suggestion, maybe if it would be better if we remove the first part cause it somehow causes an error with toml reader... or perhaps its just my vscode extension..

Additional properties are not allowed ('kv_namespaces' was unexpected)Even Better TOML

Additional properties are not allowed ('kv_namespaces' was unexpected)Even Better TOML

Additional properties are not allowed ('kv_namespaces' was unexpected)Even Better TOML

Additional properties are not allowed ('kv_namespaces' was unexpected)Even Better TOML

# Here.. remove this part vvv
kv_namespaces = [
  { binding = "<BINDING_NAME1>", id = "<NAMESPACE_ID1>" },
  { binding = "<BINDING_NAME2>", id = "<NAMESPACE_ID2>" }
]  

# Here.. remove this part ^^^
# or

[[kv_namespaces]]
binding = "<BINDING_NAME1>"
id = "<NAMESPACE_ID1>"

[[kv_namespaces]]
binding = "<BINDING_NAME2>"
id = "<NAMESPACE_ID2>"

# Here.. remove this part vvv
kv_namespaces = [
  { binding = "<BINDING_NAME1>", id = "<NAMESPACE_ID1>" },
  { binding = "<BINDING_NAME2>", id = "<NAMESPACE_ID2>" }
]  

# Here.. remove this part ^^^
# or

[[kv_namespaces]]
binding = "<BINDING_NAME1>"
id = "<NAMESPACE_ID1>"

[[kv_namespaces]]
binding = "<BINDING_NAME2>"
id = "<NAMESPACE_ID2>"

# Here.. remove this part vvv
kv_namespaces = [
  { binding = "<BINDING_NAME1>", id = "<NAMESPACE_ID1>" },
  { binding = "<BINDING_NAME2>", id = "<NAMESPACE_ID2>" }
]  

# Here.. remove this part ^^^
# or

[[kv_namespaces]]
binding = "<BINDING_NAME1>"
id = "<NAMESPACE_ID1>"

[[kv_namespaces]]
binding = "<BINDING_NAME2>"
id = "<NAMESPACE_ID2>"

# Here.. remove this part vvv
kv_namespaces = [
  { binding = "<BINDING_NAME1>", id = "<NAMESPACE_ID1>" },
  { binding = "<BINDING_NAME2>", id = "<NAMESPACE_ID2>" }
]  

# Here.. remove this part ^^^
# or

[[kv_namespaces]]
binding = "<BINDING_NAME1>"
id = "<NAMESPACE_ID1>"

[[kv_namespaces]]
binding = "<BINDING_NAME2>"
id = "<NAMESPACE_ID2>"

also.. on the default comments. can this be link be added as well so that not only you have resource for pages.. but also for worker and wrangler.toml configuration as well... https://developers.cloudflare.com/workers/wrangler/configuration/#kv-namespaces

Cloudflare Docs

Configuration | Cloudflare Workers docs

Use a

wrangler.toml

wrangler.toml

configuration file to customize the development and deployment setup for your Worker project and other Developer Platform products.

DDenver James Duran Hello team, just a suggestion, maybe if it would be better if we remove the firs...

H

Hard@Work•9/13/24, 12:41 PM

I only see an error if you have both?

HHard@Work I only see an error if you have both?

D

Denver James Duran•9/13/24, 12:58 PM

I get an error when using the kv_namesapces=[...]

could it be because of the extension I am using?

Name: Even Better TOML
Id: tamasfe.even-better-toml
Description: Fully-featured TOML support
Version: 0.19.2
Publisher: tamasfe
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=tamasfe.even-better-toml

Even Better TOML - Visual Studio Marketplace

Extension for Visual Studio Code - Fully-featured TOML support

DDenver James Duran I get an error when using the kv_namesapces=[...] could it be because of the ex...

H

Hard@Work•9/13/24, 1:01 PM

I'm using the same, no error with

kv_namespaces = [
  { binding = "<BINDING_NAME1>", id = "<NAMESPACE_ID1>" },
  { binding = "<BINDING_NAME2>", id = "<NAMESPACE_ID2>" }
]

kv_namespaces = [
  { binding = "<BINDING_NAME1>", id = "<NAMESPACE_ID1>" },
  { binding = "<BINDING_NAME2>", id = "<NAMESPACE_ID2>" }
]

kv_namespaces = [
  { binding = "<BINDING_NAME1>", id = "<NAMESPACE_ID1>" },
  { binding = "<BINDING_NAME2>", id = "<NAMESPACE_ID2>" }
]

kv_namespaces = [
  { binding = "<BINDING_NAME1>", id = "<NAMESPACE_ID1>" },
  { binding = "<BINDING_NAME2>", id = "<NAMESPACE_ID2>" }
]

D

Denver James Duran•9/13/24, 1:03 PM

hmm.. I wonder what's wrong with mine...

DDenver James Duran hmm.. I wonder what's wrong with mine...

H

Hard@Work•9/13/24, 1:05 PM

Can you post the whole

wrangler.toml

wrangler.toml

?

D

Denver James Duran•9/13/24, 1:09 PM

I dont have nitro... I'll be sending it in parts

D

Denver James Duran•9/13/24, 1:10 PM

#:schema node_modules/wrangler/config-schema.json
name = "demo"
compatibility_date = "2024-09-09"
pages_build_output_dir = ".svelte-kit/cloudflare"

# Automatically place your workloads in an optimal location to minimize latency.
# If you are running back-end logic in a Pages Function, running it closer to your back-end infrastructure
# rather than the end user may result in better performance.
# Docs: https://developers.cloudflare.com/pages/functions/smart-placement/#smart-placement
[placement]
mode = "smart"

# Variable bindings. These are arbitrary, plaintext strings (similar to environment variables)
# Docs:
# - https://developers.cloudflare.com/pages/functions/bindings/#environment-variables
# Note: Use secrets to store sensitive data.
# - https://developers.cloudflare.com/pages/functions/bindings/#secrets
# [vars]
# MY_VARIABLE = "production_value"

# Bind the Workers AI model catalog. Run machine learning models, powered by serverless GPUs, on Cloudflare’s global network
# Docs: https://developers.cloudflare.com/pages/functions/bindings/#workers-ai
# [ai]
# binding = "AI"

# Bind a D1 database. D1 is Cloudflare’s native serverless SQL database.
# Docs: https://developers.cloudflare.com/pages/functions/bindings/#d1-databases
[[d1_databases]]
binding = "DB"
database_name = "demo"
database_id = "xxxxxxx-46f8-41c4-bad3-xxxxxxxxx"
preview_database_id="xxxxxxx-46f8-41c4-bad3-xxxxxxxxx"
migrations_dir="src/lib/server/databases/d1/migrations"

#:schema node_modules/wrangler/config-schema.json
name = "demo"
compatibility_date = "2024-09-09"
pages_build_output_dir = ".svelte-kit/cloudflare"

# Automatically place your workloads in an optimal location to minimize latency.
# If you are running back-end logic in a Pages Function, running it closer to your back-end infrastructure
# rather than the end user may result in better performance.
# Docs: https://developers.cloudflare.com/pages/functions/smart-placement/#smart-placement
[placement]
mode = "smart"

# Variable bindings. These are arbitrary, plaintext strings (similar to environment variables)
# Docs:
# - https://developers.cloudflare.com/pages/functions/bindings/#environment-variables
# Note: Use secrets to store sensitive data.
# - https://developers.cloudflare.com/pages/functions/bindings/#secrets
# [vars]
# MY_VARIABLE = "production_value"

# Bind the Workers AI model catalog. Run machine learning models, powered by serverless GPUs, on Cloudflare’s global network
# Docs: https://developers.cloudflare.com/pages/functions/bindings/#workers-ai
# [ai]
# binding = "AI"

# Bind a D1 database. D1 is Cloudflare’s native serverless SQL database.
# Docs: https://developers.cloudflare.com/pages/functions/bindings/#d1-databases
[[d1_databases]]
binding = "DB"
database_name = "demo"
database_id = "xxxxxxx-46f8-41c4-bad3-xxxxxxxxx"
preview_database_id="xxxxxxx-46f8-41c4-bad3-xxxxxxxxx"
migrations_dir="src/lib/server/databases/d1/migrations"

#:schema node_modules/wrangler/config-schema.json
name = "demo"
compatibility_date = "2024-09-09"
pages_build_output_dir = ".svelte-kit/cloudflare"

# Automatically place your workloads in an optimal location to minimize latency.
# If you are running back-end logic in a Pages Function, running it closer to your back-end infrastructure
# rather than the end user may result in better performance.
# Docs: https://developers.cloudflare.com/pages/functions/smart-placement/#smart-placement
[placement]
mode = "smart"

# Variable bindings. These are arbitrary, plaintext strings (similar to environment variables)
# Docs:
# - https://developers.cloudflare.com/pages/functions/bindings/#environment-variables
# Note: Use secrets to store sensitive data.
# - https://developers.cloudflare.com/pages/functions/bindings/#secrets
# [vars]
# MY_VARIABLE = "production_value"

# Bind the Workers AI model catalog. Run machine learning models, powered by serverless GPUs, on Cloudflare’s global network
# Docs: https://developers.cloudflare.com/pages/functions/bindings/#workers-ai
# [ai]
# binding = "AI"

# Bind a D1 database. D1 is Cloudflare’s native serverless SQL database.
# Docs: https://developers.cloudflare.com/pages/functions/bindings/#d1-databases
[[d1_databases]]
binding = "DB"
database_name = "demo"
database_id = "xxxxxxx-46f8-41c4-bad3-xxxxxxxxx"
preview_database_id="xxxxxxx-46f8-41c4-bad3-xxxxxxxxx"
migrations_dir="src/lib/server/databases/d1/migrations"

#:schema node_modules/wrangler/config-schema.json
name = "demo"
compatibility_date = "2024-09-09"
pages_build_output_dir = ".svelte-kit/cloudflare"

# Automatically place your workloads in an optimal location to minimize latency.
# If you are running back-end logic in a Pages Function, running it closer to your back-end infrastructure
# rather than the end user may result in better performance.
# Docs: https://developers.cloudflare.com/pages/functions/smart-placement/#smart-placement
[placement]
mode = "smart"

# Variable bindings. These are arbitrary, plaintext strings (similar to environment variables)
# Docs:
# - https://developers.cloudflare.com/pages/functions/bindings/#environment-variables
# Note: Use secrets to store sensitive data.
# - https://developers.cloudflare.com/pages/functions/bindings/#secrets
# [vars]
# MY_VARIABLE = "production_value"

# Bind the Workers AI model catalog. Run machine learning models, powered by serverless GPUs, on Cloudflare’s global network
# Docs: https://developers.cloudflare.com/pages/functions/bindings/#workers-ai
# [ai]
# binding = "AI"

# Bind a D1 database. D1 is Cloudflare’s native serverless SQL database.
# Docs: https://developers.cloudflare.com/pages/functions/bindings/#d1-databases
[[d1_databases]]
binding = "DB"
database_name = "demo"
database_id = "xxxxxxx-46f8-41c4-bad3-xxxxxxxxx"
preview_database_id="xxxxxxx-46f8-41c4-bad3-xxxxxxxxx"
migrations_dir="src/lib/server/databases/d1/migrations"

D

Denver James Duran•9/13/24, 1:10 PM

# Bind a Durable Object. Durable objects are a scale-to-zero compute primitive based on the actor model.
# Durable Objects can live for as long as needed. Use these when you need a long-running "server", such as in realtime apps.
# Docs: https://developers.cloudflare.com/workers/runtime-apis/durable-objects
# [[durable_objects.bindings]]
# name = "MY_DURABLE_OBJECT"
# class_name = "MyDurableObject"
# script_name = 'my-durable-object'

# Bind a KV Namespace. Use KV as persistent storage for small key-value pairs.
# Docs: https://developers.cloudflare.com/pages/functions/bindings/#kv-namespaces
kv_namespaces = [
  { binding = "<BINDING_NAME1>", id = "<NAMESPACE_ID1>" },
  { binding = "<BINDING_NAME2>", id = "<NAMESPACE_ID2>" }
]  


# Bind a Queue producer. Use this binding to schedule an arbitrary task that may be processed later by a Queue consumer.
# Docs: https://developers.cloudflare.com/pages/functions/bindings/#queue-producers
# [[queues.producers]]
# binding = "MY_QUEUE"
# queue = "my-queue"

# Bind a Durable Object. Durable objects are a scale-to-zero compute primitive based on the actor model.
# Durable Objects can live for as long as needed. Use these when you need a long-running "server", such as in realtime apps.
# Docs: https://developers.cloudflare.com/workers/runtime-apis/durable-objects
# [[durable_objects.bindings]]
# name = "MY_DURABLE_OBJECT"
# class_name = "MyDurableObject"
# script_name = 'my-durable-object'

# Bind a KV Namespace. Use KV as persistent storage for small key-value pairs.
# Docs: https://developers.cloudflare.com/pages/functions/bindings/#kv-namespaces
kv_namespaces = [
  { binding = "<BINDING_NAME1>", id = "<NAMESPACE_ID1>" },
  { binding = "<BINDING_NAME2>", id = "<NAMESPACE_ID2>" }
]  


# Bind a Queue producer. Use this binding to schedule an arbitrary task that may be processed later by a Queue consumer.
# Docs: https://developers.cloudflare.com/pages/functions/bindings/#queue-producers
# [[queues.producers]]
# binding = "MY_QUEUE"
# queue = "my-queue"

# Bind a Durable Object. Durable objects are a scale-to-zero compute primitive based on the actor model.
# Durable Objects can live for as long as needed. Use these when you need a long-running "server", such as in realtime apps.
# Docs: https://developers.cloudflare.com/workers/runtime-apis/durable-objects
# [[durable_objects.bindings]]
# name = "MY_DURABLE_OBJECT"
# class_name = "MyDurableObject"
# script_name = 'my-durable-object'

# Bind a KV Namespace. Use KV as persistent storage for small key-value pairs.
# Docs: https://developers.cloudflare.com/pages/functions/bindings/#kv-namespaces
kv_namespaces = [
  { binding = "<BINDING_NAME1>", id = "<NAMESPACE_ID1>" },
  { binding = "<BINDING_NAME2>", id = "<NAMESPACE_ID2>" }
]  


# Bind a Queue producer. Use this binding to schedule an arbitrary task that may be processed later by a Queue consumer.
# Docs: https://developers.cloudflare.com/pages/functions/bindings/#queue-producers
# [[queues.producers]]
# binding = "MY_QUEUE"
# queue = "my-queue"

# Bind a Durable Object. Durable objects are a scale-to-zero compute primitive based on the actor model.
# Durable Objects can live for as long as needed. Use these when you need a long-running "server", such as in realtime apps.
# Docs: https://developers.cloudflare.com/workers/runtime-apis/durable-objects
# [[durable_objects.bindings]]
# name = "MY_DURABLE_OBJECT"
# class_name = "MyDurableObject"
# script_name = 'my-durable-object'

# Bind a KV Namespace. Use KV as persistent storage for small key-value pairs.
# Docs: https://developers.cloudflare.com/pages/functions/bindings/#kv-namespaces
kv_namespaces = [
  { binding = "<BINDING_NAME1>", id = "<NAMESPACE_ID1>" },
  { binding = "<BINDING_NAME2>", id = "<NAMESPACE_ID2>" }
]  


# Bind a Queue producer. Use this binding to schedule an arbitrary task that may be processed later by a Queue consumer.
# Docs: https://developers.cloudflare.com/pages/functions/bindings/#queue-producers
# [[queues.producers]]
# binding = "MY_QUEUE"
# queue = "my-queue"

DDenver James Duran ```toml # Bind a Durable Object. Durable objects are a scale-to-zero compute pri...

H

Hard@Work•9/13/24, 1:11 PM

I see the issue

D

Denver James Duran•9/13/24, 1:11 PM

the rest are just literal comments.. so I guess it doesn't matter?

H

Hard@Work•9/13/24, 1:11 PM

kv_namespace

kv_namespace

kv_namespace

kv_namespace is under

d1_databases

d1_databases

d1_databases

d1_databases

H

Hard@Work•9/13/24, 1:12 PM

Place it underneath

pages_build_output_dir

pages_build_output_dir

D

Denver James Duran•9/13/24, 1:13 PM

oh wow it did the trick!

Why? so does this mean that on the wrangler toml file the kv namespace should be declared above d1 database?

DDenver James Duran oh wow it did the trick! Why? so does this mean that on the wrangler toml file ...

H

Hard@Work•9/13/24, 1:15 PM

TOML Shenanigans. Basically, when you add a table declaration(

[placement]

[placement]

[placement]

[placement]) or a table array declaration(

[[d1_databases]]

[[d1_databases]]

[[d1_databases]]

[[d1_databases]]), everything until the next declaration is assumed to be a part of that table

H

Hard@Work•9/13/24, 1:15 PM

This is why I no longer use

toml

toml

for personal projects

D

Denver James Duran•9/13/24, 1:21 PM

that makes sense! Hmm how do we prevent that?

also, isn't wrangler.toml required in deploying apps in cloudflare?

DDenver James Duran that makes sense! Hmm how do we prevent that? also, isn't wrangler.toml require...

H

Hard@Work•9/13/24, 1:23 PM

I use

wrangler.json

wrangler.json

. It is supported for Workers, but not for Pages

NNoil My application accesses files from the KV-store and have been running into the i...

N

NoilOP•9/16/24, 12:56 PM

Any possible answer to why files in the KV store can be unfindable? could it be the caching or something else?

NNoil Any possible answer to why files in the KV store can be unfindable? could it be ...

Z

Zack•9/16/24, 1:18 PM

If you make a request for a key that doesn't yet exist, the not found response will be cached for the same TTL as if it were found. Default 60 seconds iirc, but can be overridden

ZZack If you make a request for a key that doesn't yet exist, the not found response w...

H

Hard@Work•9/16/24, 1:19 PM

Note here that it can be raised above 60 seconds, but not lowered

T

TigersWay•9/17/24, 8:47 AM

For the first time I got the message "Daily request limit exceeded for Cloudflare Workers"
This worker have an average (and pretty stable) of 250/350 writes every day, and I just checked on the dashboard without seeing anything strange. How should I best proceed?
Thank you!

N

nejc•9/17/24, 9:03 AM

Hi all, when I am using list method and cursor for pagination (https://developers.cloudflare.com/kv/api/list-keys/#list-method). What happens if I have 1 milion keys and when I get a page I trigger delete of that keys and then use cursor to go to the second page? Would cursor have an old list so it will continue serving correct pages even though that I am deleting keys in the background?

Cloudflare Docs

List keys | Cloudflare Workers KV

To list all the keys in your KV namespace, call the list() method of the KV binding on any KV namespace you have bound to your Worker code:

T

thomasgauvin•9/17/24, 9:46 AM

Hey folks! We just shipped the new Analytics GraphQL API and Dashboard UI for Workers KV

Now, you can get per-namespace analytics and more details such as cache status and latency information, and it's standardized on the same GraphQL API layer as other Cloudflare products.

Dashboard UI: You can try this one directly from the dashboard, just go to one of your KV namespaces and see the new metrics UI.

GraphQL API: You can now get your KV analytics directly from the standard GraphQL API (sample queries and how to configure your requests here: https://developers.cloudflare.com/kv/observability/metrics-analytics/)

Check it out in action (looks even better in person)!
https://developers.cloudflare.com/kv/platform/changelog/#2024-08-08

N

nejc•9/17/24, 10:00 AM

@thomasgauvin really awesome view and great information source. One thing that I noticed is that if KV is big it doesn't show KV count

M

Marcus•9/17/24, 9:51 PM

Heya—I assume probably not, but is there a way to tweak the usage limit email threshold? I'm getting daily "you have used 50% of the free tier!" emails... from a cron job that reliably uses just over half of the free tier allotment :D

TTigersWay For the first time I got the message "Daily request limit exceeded for Cloudflar...

M

Marcus•9/17/24, 10:56 PM

(er, wait... no, it doesn't, it uses like 1/3. seems like essentially a duplicate of this thread, I'll stay tuned on that)

M

Matt Silverlock•9/18/24, 12:32 PM

https://x.com/thomasgauvin/status/1836376211771081188?s=46

E

EXE KL•9/19/24, 1:28 PM

Does KV have a feature for checking existing keys on write? By default it does not but is there a feature that can be turned on to enable such a thing?

EEXE KL Does KV have a feature for checking existing keys on write? By default it does n...

H

Hard@Work•9/19/24, 1:30 PM

No, and no

HHard@Work No, and no

E

EXE KL•9/19/24, 1:34 PM

I knew but I had to ask anyway. Can you clarify if I'm being paranoid then. I want to store magic tokens(that expire after an hour) and although I've gone to great lengths to make the token unique, there is always that statistically improbable chance that a duplicate token is created within that ttl window. I'm I crazy for trying to check for duplicates when the chance for that is as probable as the sun going out?

E

EXE KL•9/19/24, 1:36 PM

It also could just be the way I'm implementing the magic token system that invites uncertainties.

EEXE KL I knew but I had to ask anyway. Can you clarify if I'm being paranoid then. I wa...

H

Hard@Work•9/19/24, 1:39 PM

You could also just use DOs?

H

Hard@Work•9/19/24, 1:39 PM

Then have it fail the write if it already exists?

HHard@Work You could also just use DOs?

E

EXE KL•9/19/24, 1:41 PM

Forgive me, what is a DO?

EEXE KL Forgive me, what is a DO?

H

Hard@Work•9/19/24, 1:42 PM

https://developers.cloudflare.com/durable-objects/#_top

Cloudflare Docs

Overview | Cloudflare Durable Objects docs

Durable Objects provide a powerful API for coordinating multiple clients or users, each with private, transactional and strongly consistent storage attached.

HHard@Work https://developers.cloudflare.com/durable-objects/#_top

E

EXE KL•9/19/24, 1:44 PM

Ah okay, so this is available through the transactional storage api?

H

Hard@Work•9/19/24, 1:44 PM

Or well, each DO has a separate KV Store

HHard@Work Or well, each DO has a separate KV Store

E

EXE KL•9/19/24, 1:46 PM

Thanks, I'll take a proper read through it.

EEXE KL Thanks, I'll take a proper read through it.

T

thomasgauvin•9/19/24, 8:13 PM

This seems like an optimization for an extremely unlikely use case. How many keys do you plan on storing and is it anywhere near risking hash collision?

You could also reduce probably by making a read first, and writing it if it doesn't exist. It won't deterministically guarantee that you aren't overwriting, because it may have been written/propagated to KV since the read, but it wouldn't hurt in reducing chances of collision

If you need stronger unique guarantees, DO/D1 may be better options

Tthomasgauvin This seems like an optimization for an extremely unlikely use case. How many key...

E

EXE KL•9/19/24, 8:35 PM

My plan is to reuse this authentication and session system whenever I can when making Sveltekit projects so I wanted to make something robust enough to handle a large number of users authenticating themselves frequently. I've considered reading before the writing but as you mentioned it only reduces the chances on top of an extra network call. D1 was the only real solution for me before learning about DO.

E

EXE KL•9/19/24, 8:44 PM

As for the use cases. I probably may never generate enough magic tokens to increase the odds of a duplication within a period of time as the size of the tokens is 64 chars generated by the crypto module. I think I just wanted a solution to a hypothetical problem.:NotLikeThis:

EEXE KL My plan is to reuse this authentication and session system whenever I can when m...

H

hfcRed•9/20/24, 11:09 AM

Make a separate KV for every project and you should be fine

My application accesses files from the KV-store and have been running into the issue that specifc fi

Similar Threads

Similar Threads

Similar Threads