I did get it to work a bit but I still don't understand. I deployed, my wrangler.toml file has worke

I did get it to work a bit but I still don't understand. I deployed, my wrangler.toml file has workers_dev=true yet the key value doesn't come from the namespace specified by preview_id

Eeg_scl I did get it to work a bit but I still don't understand. I deployed, my wrangler...

eg_sclOP•7/4/23, 2:17 AM

SOLVED: I just needed to add --remote to the end of 'wrangler dev --remote' now I can read KV from localhost.

marina.lipeck•7/4/23, 1:56 PM

Hey, this announcement - https://blog.cloudflare.com/faster-workers-kv-architecture/#:~:text=If%20these%20sound%20like%20the%20killer%20features%20you%20need%2C%20please%20reach%20out%20to%20us%20if%20you%E2%80%99re%20interested%20in%20trying%20them%20out about new exiting features says that one can reach out and try out them earlier then ga - how can I do it?

The Cloudflare Blog

Workers KV is faster than ever with a new architecture

With the new architecture powering Workers KV our service will become faster and more scalable than ever. We have significantly reduced cold read probability, and enabled KV to serve over a trillion requests a month

Cyb3r-Jak3•7/5/23, 12:12 PM

What do you mean by updating KV? If you have an existing worker that has an existing KV binding then it should continue to function. With the maintenance, there might be issues with creating a new worker, kv namespace, or binding the namespace to a worker

Ddam6pl Managing KV values in existing namespace in existing Worker binding 🙂

Cyb3r-Jak3•7/5/23, 12:19 PM

My understanding is those operations should not be affected

0x33•7/7/23, 12:31 AM

I'm trying to write something to a KV and it stays unaffected. :/

await env.HEADERS.put(uuid, packedHeaders, { expirationTtl: 60 * 60 * 24 * 7 });

await env.HEADERS.put(uuid, packedHeaders, { expirationTtl: 60 * 60 * 24 * 7 });

I tried on my localhost and in deployed production. Wrangler (3.1.2) does not issue any error/warning. What can be the cause?

0x33•7/7/23, 12:34 AM

Am I using outdated tomltoml?

[[kv_namespaces]]
binding = "HEADERS"
id = "xxx"
preview_id = "xxx"

[[kv_namespaces]]
binding = "HEADERS"
id = "xxx"
preview_id = "xxx"

0x33•7/7/23, 12:37 AM

Just tried

kv_namespaces = [
  { binding = "<MY_NAMESPACE>", id = "<KV_ID>" }
]

kv_namespaces = [
  { binding = "<MY_NAMESPACE>", id = "<KV_ID>" }
]

Doesn't work..

0x33•7/7/23, 1:00 AM

I inserted some keys and values via the dashboard and await env.HEADERS.get(..)await env.HEADERS.get(..) returns

null

null

for them

Chaika•7/7/23, 1:01 AM

If you're using wrangler 3.x, the default wrangler dev is local/all local data, have to use wrangler dev --remote to access real KV namespaces/data

0x33•7/7/23, 1:07 AM

That works with the preview KV namespace! Thanks! How do I make it work for production deployment?

0x33•7/7/23, 1:10 AM

I npx wrangler deploynpx wrangler deploy ed the same code. The binding IDs are correct and nothing shows up in the KV. The worker gets hit 155k/24h

Chaika•7/7/23, 1:12 AM

Is your kv namespace id and preview id the same? dev --remote will use preview_id, and deployed worker will use normal id, might be the cause of some confusion.

0x33•7/7/23, 1:15 AM

These are different IDs. I understand that preview has a different purpose to production. There was a warning about that when I tried to run npx wrangler devnpx wrangler dev too.

binding = "HEADERS"
id = "659461f0c2c446b..."
preview_id = "65ee1a3b420c4b..."

binding = "HEADERS"
id = "659461f0c2c446b..."
preview_id = "65ee1a3b420c4b..."

0x33•7/7/23, 1:18 AM

I double checked the IDs and just redeployed the same code. No errors from workers

Chaika•7/7/23, 1:18 AM

Right, my line of thinking was "are you checking the right one"? If your worker simply puts that on every request, you should be able to see those values in the dashboard. Can you see them there? If so, it is just the worker itself not being able to get them later, or something else?

0x33•7/7/23, 1:21 AM

5 records appeared in the KV but the strange thing is that it only contains my 5 curlcurl requests I made after the deploy. Is there a delay in the global worker deployment or is the KV localised?

0x33•7/7/23, 1:21 AM

0x33•7/7/23, 1:22 AM

0x33•7/7/23, 1:23 AM

Chaika•7/7/23, 1:24 AM

KV is global, that kinda sounds like to me you're looking at the preview and not the production namespace, or you have something with your code that is preventing writes. Workers are pretty quick to deploy globally, maybe 10-15s

0x33•7/7/23, 1:26 AM

  async fetch(request, env, ctx) {
// ... snip ...
    await env.HEADERS.put(uuid, data, { expirationTtl: 60 * 60 * 24 * 7 });
// ... snip ...

  async fetch(request, env, ctx) {
// ... snip ...
    await env.HEADERS.put(uuid, data, { expirationTtl: 60 * 60 * 24 * 7 });
// ... snip ...

This is the worker code. It uses export default {export default { syntax. Every request should hit it.

0x33•7/7/23, 1:27 AM

Hmm...

0x33•7/7/23, 1:27 AM

Is it safe to post namespace ids here?

0x33•7/7/23, 1:28 AM

0x33•7/7/23, 1:29 AM

I'm cetain that this is the same

id

id

as in [[kv_namespaces]][[kv_namespaces]] id = id =

0x33•7/7/23, 1:29 AM

Maybe I'm doing something really silly.

0x33•7/7/23, 1:32 AM

Could you tell if this might be the culprit? I'm using a custom domain "without custom domain". I created route and DNS record manually so the requests are possibly routed to the old worker instances and the new code is never executed. I've never actually used the Connect your Worker to a custom domain.Connect your Worker to a custom domain. panel.

0x33•7/7/23, 1:33 AM

When I ran curlcurl I used *.workers.dev*.workers.dev which contains the most fresh deployment.

0x33•7/7/23, 1:39 AM

Yes. That was it!

Chaika•7/7/23, 1:39 AM

Namespace IDs (along other IDs like Account ID, Zone IDs, etc) are not sensitive, although I can't do anything with them, just a community member

Maybe I'm doing something really silly.

Definitely possible you're doing something silly and I can't see it lol

Could you tell if this might be the culprit? I'm using a custom domain "without custom domain". I created route and DNS record manually so the requests are possibly routed to the old worker instances and the new code is never executed. I've never actually used the Connect your Worker to a custom domain. panel.

I believe Old Workers stay around for a bit to handle in flight requests/etc, but don't stay around for that long, and that wouldn't be http route specific

Custom domains are just another way to bind

0x33•7/7/23, 1:39 AM

Do it the Cloudflare way @yoloking9000

0x33•7/7/23, 1:40 AM

Thanks for help @chaika.me . I'm so happy now. The KV is populating.

0x33•7/7/23, 1:41 AM

Chaika•7/7/23, 1:42 AM

Nice! Was just a matter of time. KV itself by nature is eventually consistent, although the dashboard should be very quick since it pulls from the central stores iirc, those account analytics are def a bit delayed. How KV works is a good read if you haven't read it already: https://developers.cloudflare.com/workers/learning/how-kv-works/
Edit: I almost forget, things have changed a bit recently, and there is more caching layers, the recent blog post is a good read as well: https://blog.cloudflare.com/faster-workers-kv-architecture/, KV is pretty cool

vassbence•7/7/23, 10:29 AM

KV cold read times are really bad

vassbence•7/7/23, 10:29 AM

makes me want to switch to DynamoDB as I might get worse read times simple due to location (even if i do like a multi region deployment) but it's still better than cold reads in kv by 3x

vassbence•7/7/23, 10:45 AM

Also, how does KV differ from a CDN (like CloudFront) caching JSON responses? Since both of the solutions go to the origin in case of a cache MISS

Vvassbence Also, how does KV differ from a CDN (like CloudFront) caching JSON responses? Si...

Hello, I’m Allie!•7/7/23, 2:33 PM

You pay a flat fee for transit to the origin, whether it is the same colo, or across the world. Plus, you benefit from pixelcf

’s routing to get to core faster

beelzabub•7/7/23, 3:28 PM

If your key has insanely low cache hit rate (typically means you have a lot of cardinality in the key name) then I agree KV will be a bad fit. In practice though, we find access patterns don't demonstrate high cardinality and thus the high cache hit ratio means you're rarely getting the worst read times. Additionally, with https://blog.cloudflare.com/faster-workers-kv-architecture/, we amortize the cost of cold reads within your region and then globally. You'd have to be doing a read of a totally never before seen key every single time.

Additionally, if you're worried about things getting evicted from the cache because the keys are infrequently accessed, I'd recommend setting a very high cacheTTL. It doesn't alter whether you see the update, but it can help with keeping TTFB down for infrequently accessed keys. The only caveat to consider is in a security context (e.g. if you're storing access tokens, the cacheTTL would map to the revocation TTL you can promise)

Tin Cap•7/12/23, 2:42 AM

Is there a standard practice on using a single vs multiple KV stores?

Say I have two entities to store X and Y. I'm debating between:

1) Having three KV stores: dev, staging, and production. Then using keys prefixes to differentiate "x:<ID>" and "y:<ID>"

2) Having six KV stores: dev-X, dev-Y, etc. Then using simple keys "<ID>".

Approach #1 seems a little messier since you're co-mingling data, but I'm actually having a hard time figuring out why I would care about that.

Approach #2 seems simpler in that I only need to setup one KV store namespace in my app.

James•7/12/23, 4:46 AM

I’d probably go with #2. It’s not going to make much of a difference as you noted, but #2 brings the benefit of being able to completely nuke an environment’s data very quickly by just dropping and recreating that namespace if ever needed.

Unsmart•7/12/23, 4:50 AM

I personally would do #1 helps especially if you want to do somewhat complex queries of related data doing a list by prefix but if you never want to do that it doesn’t really matter then

beelzabub•7/12/23, 2:41 PM

I typically would have dev comingled with staging (maybe with an injected prefix if I'm worried about name conflicts). Whether I do 1 or 2 would depend on the relationship between the keys. If it's the same logical service, I'd probably go with 1. If it's different logical services I'd go with #2. I distinguish a logical service from whether it lives in the same worker by the functionality it's accomplishing (e.g. if you know you have a service hiding within the code of another). In practice though, #1 is often the choice.

Tin Cap•7/12/23, 2:55 PM

Yeah, I'm leaning towards approach 1 for this use case. The data is used by the same service(s) and would get created/destroyed together.

Tin Cap•7/12/23, 2:56 PM

Kinda seems like KV stores are much like logical groupings of data. Keeping data isolated properly, rather than individual datatypes.

Bbeelzabub I typically would have dev comingled with staging (maybe with an injected prefix...

Tin Cap•7/12/23, 2:57 PM

I treat dev and staging differently because dev is a mix of dirty data. I put bad stuff into dev and nuke it all the time. Staging is at least attempting to be as clean as production.

spillere - depinhub.io•7/12/23, 5:45 PM

https://github.com/cloudflare/cloudflare-go/blob/master/workers_kv_example_test.go#L107

I think i'm having an issue to get the accont id, can it be to any domain?

GitHub

cloudflare-go/workers_kv_example_test.go at master · cloudflare/clo...

Go library for the Cloudflare v4 API. Contribute to cloudflare/cloudflare-go development by creating an account on GitHub.

I did get it to work a bit but I still don't understand. I deployed, my wrangler.toml file has worke

Similar Threads

Similar Threads

Similar Threads