Cloudflare Developers•2y ago

If this is in pages dev then it’s local and doesn’t write to your actual namespace

Kkian If this is in pages dev then it’s local and doesn’t write to your actual namespa...

Chinoman10•6/23/24, 5:07 PM

I figured it could be that, so I already pushed (deployed) it and it still doesn't work.

Kkian If this is in pages dev then it’s local and doesn’t write to your actual namespa...

Chinoman10•6/23/24, 5:09 PM

Seeing this on the dashboard...

Does that mean I have to specify two namespaces for the two different environments?

Chinoman10•6/23/24, 5:10 PM

Here's a more complete version of my current toml:

name="pages-name"
pages_build_output_dir = "dist"
compatibility_date = "2024-06-12"

[[kv_namespaces]]
    binding = "OAUTH_KV"
    id = "..."

[dev]
    port = 3000

[env.production]
    [env.production.vars]
        PUBLIC_ENVIRONMENT = "production"
        SOME_VAR = "BLA BLA"

[env.preview]
    [env.preview.vars]
        PUBLIC_ENVIRONMENT = "preview"
        SOME_VAR = "BLA BLA"

name="pages-name"
pages_build_output_dir = "dist"
compatibility_date = "2024-06-12"

[[kv_namespaces]]
    binding = "OAUTH_KV"
    id = "..."

[dev]
    port = 3000

[env.production]
    [env.production.vars]
        PUBLIC_ENVIRONMENT = "production"
        SOME_VAR = "BLA BLA"

[env.preview]
    [env.preview.vars]
        PUBLIC_ENVIRONMENT = "preview"
        SOME_VAR = "BLA BLA"

So perhaps I have to declare the KV under the [env.production] and [env.preview] as well?

ehrenmann•6/23/24, 6:03 PM

Hi, i have 2 workers that are spread on multiple domains. I save data in d1 and read kv values on each request and im afraid the limits will not be enough for me (cpu time, reads and writes and size pf the D1). Are there limits if i purchase workers for platforms?

Walshy•6/23/24, 6:13 PM

Workers Paid is what you'd want

CPU time on Paid is 30 seconds, reads and writes are charged if you go over the included usage.
D1's size however cannot be increased, the recommendation there is sharding into multiple DBs

Kkian If this is in pages dev then it’s local and doesn’t write to your actual namespa...

Chinoman10•6/23/24, 10:04 PM

Hey Kian, any idea what could be wrong with what I shared earlier?

CChinoman10 Hey Kian, any idea what could be wrong with what I shared earlier?

Chinoman10•6/24/24, 9:14 AM

Found it. It was my suspicion... the namespace needs to be declared under each env...

thomasgauvin•6/24/24, 7:34 PM

This isn't a feature that KV provides currently, can you provide more information on the use case? We're collecting more info in the area, interested in hearing about the level of queryability you're looking to get vs D1

Chinoman10•6/24/24, 8:19 PM

I've been getting highly inconsistent experiences with KV unfortunately...
I know that KV is "eventually consistent", but even after several hours of an entry being added (put), I still get null when trying to obtain those keys... Sometimes even hours later.

I can understand a few minutes delay for newly created values, and I can understand it takes much longer for changes/deletes (unless expiresexpires is set, in which case I'd assume it doesn't take long to disappear after), but not being able to 'GET' the value on a Pages Function, several days after it's already been putput (and is shown on the dashboard), feels a lot like a bug and not a "that's how it is".

CChinoman10 I've been getting highly inconsistent experiences with KV unfortunately... I kno...

thomasgauvin•6/24/24, 8:57 PM

That does seem highly unusual (ie not something that is intended behavior). Do you specify a cacheTtl on these read operations? Are you accessing via the binding or using the API? Are you verifying from the deployed Worker? How are you verifying that your values are not getting updated (via a print in Worker logs, etc.)?

Tthomasgauvin That does seem highly unusual (ie not something that is intended behavior). Do y...

Chinoman10•6/24/24, 10:24 PM

Thanks for replying Thomas

I don't specify a cache, no.

I'm accessing via the Binding (declared via

wrangler.toml

wrangler.toml

) on a Pages project (with a Function). I can see that when I run wrangler pages locally it binds correctly (if it didn't, it wouldn't be able to fetch any value anyway, but it can, albeit only one).

If you'd like I can tell you the endpoint where the function is deployed to and you can test on your own browser (both the put and the read); right now it doesn't have any auth (yet). Just send me a DM and I'll reply

(or open a private thread here if you have that permission)

PPerry Sahnow KV works really well for this since you set it once, then leave the key basicall...

Matt Silverlock•6/24/24, 11:10 PM

Your options:
- D1 as an index
- Use KV with list()list() and a consistent prefix - e.g. ${userId}-${urlId}${userId}-${urlId} so you can list all URLs by user - this may be tricky and lengthen your URLs.
- Durable Objects as an index - e.g. write to the DO itself for that userId/account as your index, and write (from within the DO) to KV to give you the fast read path for the URLs themselves.

PPerry Sahnow I thought about doing the second one, but then my issue is that given an input c...

Matt Silverlock•6/24/24, 11:50 PM

e.g. https://developers.cloudflare.com/durable-objects/examples/use-kv-from-durable-objects/

Cloudflare Docs

Durable Objects - Use KV within Durable Objects · Cloudflare Durabl...

The following Worker script shows you how to configure a Durable Object to read from and/or write to a Workers KV namespace. This is useful when using …

kindaraffy•6/25/24, 5:02 PM

Is there a way to purge specific keys (or forcing a purge by resetting the cacheTtl)?

I have the perfect use case for KV: read heavy, long cacheTtls and very infrequent writes. However, I do need kv updates to be reflected relatively quickly. Should I delete the key and re-insert? What's the recommendation for this when you have very long ttls but need to refresh quickly when it is (rarely) needed?

Kkindaraffy Is there a way to purge specific keys (or forcing a purge by resetting the cache...

thomasgauvin•6/25/24, 6:05 PM

That's not possible @kindaraffy, no. Deletions are also eventual (may still be cached elsewhere in the network). The recommended for a refresh is to have 60s ttl to have your KV pairs expire, and then return the updated value after expiry. What's your use case/what are you building? Might help us determine a better alternative

EEisenbergEffect Reading up on the limites of KV store, I have a few questions I wonder if someon...

thomasgauvin•6/25/24, 6:13 PM

1. they will fail silently, ie one write will apply and the others will be dropped. 2. you can make 1000 KV operations (lists, reads, puts, deletes) within a single Worker invocation/request.

Tthomasgauvin That's not possible @kindaraffy, no. Deletions are also eventual (may still be c...

kindaraffy•6/25/24, 6:34 PM

I wanted to break off a part of our permission’s system to be stored almost entirely within a cf KV store.

These permissions are essentially just read-heavy strings and most of them gets written to once during a setup phase.

We expect to have high ttls on these strings. Only issue is that when a user changes permissions, we expect these change to be reflected relatively quickly.

Nameless Monster•6/25/24, 7:06 PM

I was following the Cloudflare Workers KV quick start guide and ran into an unavoidable error:

✘ [ERROR] Error processing request: TypeError: Cannot read properties of undefined (reading 'put')

I have no idea what this error means, and there aren't any docs or resources available to help.

NNameless Monster I was following the Cloudflare Workers KV quick start guide and ran into an unav...

Hello, I’m Allie!•6/25/24, 7:07 PM

That sounds like you didn't bind a KV namespace

HHello, I’m Allie!That sounds like you didn't bind a KV namespace

Nameless Monster•6/25/24, 7:09 PM

I did

NNameless Monster I did

Hello, I’m Allie!•6/25/24, 7:09 PM

What does your wranglerwrangler config look like?

PPerry Sahnow right, and at that point I might as well just keep a DO as my index.

Chinoman10•6/25/24, 9:21 PM

Speaking from experience, a really nice use-case for DO's is having one per user. Has several benefits, one of them being that they'll always be physically located close to that specific user, making your webapp seem super fast, unless the person is constantly travelling between different countries, but even still...

NNameless Monster I was following the Cloudflare Workers KV quick start guide and ran into an unav...

Chinoman10•6/25/24, 9:22 PM

Yea, I had the same issue.

On your wrangler toml you have to define the kv_namespace for all the environments and also under "no environment". Only then did it work "well" for me (binding wise) for dev, preview and prod.

Chinoman10•6/25/24, 9:23 PM

I agree that the docs should be made clearer though IMHO

mohamed.adham•6/28/24, 7:51 PM

Hello everyone!

I need to integrate my nextJs app with cloudflare KV, I made the app with next-on-pages from cloudflare CLI

cosbgn•6/29/24, 3:29 PM

I'm building something like google sheets. Each user could have multiple sheets and I could have thousands of users. I'm thinking how to store individual rows. I would use d1, but I'm scared to hit the 10GB limit and multiple databases seems like a lot of work.

I first though to store it as kv, key being the spreadsheet_id and value being the csv. This seems fairly easy, but my issue is that with every update I need to replace the entire csv, having "eventually" consistent data will most likely lead to loosing data when multiple updates occur.

cosbgn•6/29/24, 3:29 PM

My final solution is:

KV, with key being spreadsheet_id/row_id and value being the csv only for that row

cosbgn•6/29/24, 3:30 PM

this seems fine, but it means I'll have all data of all spreadsheets in the same kv

cosbgn•6/29/24, 3:30 PM

is this feasable? will it become super slow?

cosbgn•6/29/24, 3:31 PM

I've read about prefix, are these like indexes and will they speed up my queries? so if I prefix by spreadsheet_id it will look only there and avoid looking on millions of rows?

cosbgn•6/29/24, 3:31 PM

any suggestions on how to do this? How would you do it?

Ccosbgn I'm building something like google sheets. Each user could have multiple sheets ...

Matt Silverlock•6/29/24, 4:21 PM

You almost definitely want strong consistency (serializable) - Durable Objects would be the fit here.

Each DO is a sheet. Use y.js to manage state/changes/apply deltas. A framework like PartyKit can help simplify this: https://www.partykit.io/

KV is almost definitely not a fit as any changes you make won’t be immediately reflected to other users.

cosbgn•6/29/24, 4:26 PM

Ok I'll have a look at do. Thanks!

Chiquinho2001•6/30/24, 11:02 AM

Hi everyone,

I'm currently facing issues with using KV Storage in my project. Specifically, I'm having trouble getting an image to load when I test my worker locally. I've followed the documentation, but I still can't seem to get it to work. Has anyone encountered a similar problem or have any suggestions on how to resolve this?

Thanks in advance for your help!

EEisenbergEffect When getting a value from a KV store, is there a network hop involved in that op...

Hello, I’m Allie!•6/30/24, 5:08 PM

There may be a network hop, depending on whether or not the value is cached in the local datacenter

If this is in pages dev then it’s local and doesn’t write to your actual namespace

Similar Threads

Similar Threads

Similar Threads