when running workers-rs with wrangler on local mode, my code attempts to store a value on the key va

when running workers-rs with wrangler on local mode, my code attempts to store a value on the key value database, (no errors), but when i check the dashboard on cloudflare, there doesn't seem to actually be anything stored

mooonOP•5/22/23, 4:34 PM

anyone know why?

Mmooon when running workers-rs with wrangler on local mode, my code attempts to store a...

Cyb3r-Jak3•5/22/23, 4:34 PM

Local mode doesn’t effect any remote resources

mooonOP•5/22/23, 4:35 PM

ah i see

mooonOP•5/22/23, 4:36 PM

so if i use on my worker with local mode enabled and attempt to add something, it'll just be stored somewhere locally on my system?

Mmooon so if i use `ctx.kv` on my worker with local mode enabled and attempt to add som...

Cyb3r-Jak3•5/22/23, 4:39 PM

Only if you have the persist flag (wrangler 2). Otherwise it is stored in memory and is gone once dev stops

mooonOP•5/22/23, 4:40 PM

alright, thank you

b-fuze (Mike32)•5/22/23, 11:30 PM

Does KV support secondary indexes?

Bb-fuze (Mike32)Does KV support secondary indexes?

Isaac McFadyen•5/23/23, 12:40 AM

No.

Isaac McFadyen•5/23/23, 12:40 AM

You can list keys by prefix but that's it.

IIsaac McFadyen No.

b-fuze (Mike32)•5/23/23, 12:40 AM

Thanks!

b-fuze (Mike32)•5/23/23, 12:41 AM

So to "resort" something I'd have to delete and readd it with a new key right? @Isaac McFadyen | YYZ01

Bb-fuze (Mike32)So to "resort" something I'd have to delete and readd it with a new key right? @...

Isaac McFadyen•5/23/23, 12:41 AM

Yeah pretty much.

b-fuze (Mike32)•5/23/23, 12:41 AM

Coolio

Isaac McFadyen•5/23/23, 12:42 AM

Keep in mind that because KV is eventually consistent if you read a value then change the value with the same key it might take up to 60s to see the new value in other locations

Isaac McFadyen•5/23/23, 12:42 AM

If you change the key, as long as you haven't queried the key before, it should be instant.

Isaac McFadyen•5/23/23, 12:42 AM

If you've queried the key before, even if it didn't exist at the time, it may take up to 60s to see the new value

b-fuze (Mike32)•5/23/23, 12:42 AM

Huh... Nah, this key will have been queried before

Isaac McFadyen•5/23/23, 12:43 AM

In which case, yeah, 60s (or more in very rare cases) to see in other locations.

b-fuze (Mike32)•5/23/23, 12:43 AM

It'll be queried and then immediately updated again

IIsaac McFadyen In which case, yeah, 60s (or more in *very* rare cases) to see in other location...

b-fuze (Mike32)•5/23/23, 12:44 AM

Fair enough, thanks!

b-fuze (Mike32)•5/23/23, 1:30 AM

@Isaac McFadyen | YYZ01 do you know if there's a way to count keys? Something like the method but without returning the actual keys and just a count?

Bb-fuze (Mike32)@Isaac McFadyen | YYZ01 do you know if there's a way to count keys? Something li...

Isaac McFadyen•5/23/23, 1:32 AM

I don't believe so, no, aside from just manually calling and counting that way (which is error-prone).

b-fuze (Mike32)•5/23/23, 1:32 AM

I'll have to do that then. I know that there's eventual consistency involved, but I'll need a count even if it's a rough count

Bb-fuze (Mike32)I'll have to do that then. I know that there's eventual consistency involved, bu...

Isaac McFadyen•5/23/23, 1:33 AM

~~Don't quote me on this - but I don't think calls use the cache, actually, and so won't be subject to the eventual consistency part.~~ See below.

Isaac McFadyen•5/23/23, 1:38 AM

Easy way to test, actually - if you call multiple times in a row, if the calls all take then same time then they don't get cached, if the first is slower than the rest they do get cached.

Isaac McFadyen•5/23/23, 1:43 AM

I am wrong, just tested and KV operations do indeed get cached

Isaac McFadyen•5/23/23, 1:43 AM

Sorry about that.

b-fuze (Mike32)•5/23/23, 1:45 AM

Np, thanks for checking!

MMesorchoo Thanks, any plans to add something like the bulk import with wrangler in the fut...

beelzabub•5/23/23, 1:53 AM

That's more a question for the wrangler channel I think.

IIsaac McFadyen I am wrong, just tested and KV `list` operations do indeed get cached 😄

beelzabub•5/23/23, 1:55 AM

They get cached for the exact same sets of parameters. The cache key is built from the parameters (prefix, limit, or cursor) and cached for max 1 minute (IIRC you can't customize the cache TTL currently).

Bbeelzabub They get cached for the exact same sets of parameters. The cache key is built fr...

b-fuze (Mike32)•5/23/23, 3:09 AM

Do you know if has an upper limit somehow? After 7500 keys it won't return another , but I should have inserted more keys than that

matt•5/23/23, 4:38 AM

the latency on these kv gets/puts seems awfully high no?

Bb-fuze (Mike32)Do you know if `list` has an upper limit somehow? After 7500 keys it won't retur...

beelzabub•5/23/23, 5:22 AM

It shouldn’t. If you’re doing writes concurrently then it’s possible you cached the end of the list and need to wait a minute or so. What’s your namespace?

Mmatt the latency on these kv gets/puts seems awfully high no?

beelzabub•5/23/23, 5:23 AM

Which geographic location is this happening in and are you sure the gets are happening on a key that’s supposed to be already cached? Also is this a change in behavior or a new measurement you’re looking at

Bbeelzabub Which geographic location is this happening in and are you sure the gets are hap...

matt•5/23/23, 5:41 AM

My bad, I thought Region: Earth meant all the data was replicated globally? I’m in Australia so if both reads and writes are going somewhere off-continent that would explain these figures

matt•5/23/23, 5:42 AM

But I am confused about why they would be going off continent?

Mmatt My bad, I thought Region: Earth meant all the data was replicated globally? I’m ...

Hello, I’m Allie!•5/23/23, 7:07 AM

KV core is US/EU iirc, so writes go there. KV reads are cached locally, in the datacenter they are read from

HHello, I’m Allie!KV core is US/EU iirc, so writes go there. KV reads are cached locally, in the d...

beelzabub•5/23/23, 7:54 AM

Correct. https://developers.cloudflare.com/workers/learning/how-kv-works/. So yeah, writes from Australia may take a while. The first read ever for a key will be slow but subsequent reads should be fast. The reads don’t even need to be from the same data center. Anywhere in Australia will populate the cache for all of Australia.

How KV works · Cloudflare Workers docs

Workers KV is a global, low-latency, key-value data store. It stores data in a small number of centralized data centers, then caches that data in …

Bbeelzabub Correct. https://developers.cloudflare.com/workers/learning/how-kv-works/. So ye...

Hello, I’m Allie!•5/23/23, 7:59 AM

Oh, is that new with KV2? And how do the regions work there? Is it by country?

Erisa•5/23/23, 9:01 AM

The page has been there for a long time

EErisa The `How KV works` page has been there for a long time

Hello, I’m Allie!•5/23/23, 9:56 AM

No, I mean the thing where cache is applied over an entire region? The docs still seem to imply that cache is only in a single datacenter:

[M]ore popular values are cached in the data centers they are requested from.

Bbeelzabub That’s a fantastic question. I’ll note that option 2 also has a challenge in tha...

Larry•5/23/23, 10:29 AM

When you say "reminder that you can only list 1k at a time", do you mean that a page is only 1k max, or that all the pages can't return more than 1,000 items?

LLarry When you say "reminder that you can only list 1k at a time", do you mean that a ...

asp•5/23/23, 10:44 AM

That one listing is limited to 1000 keys. You can use the cursor to paginate.

Aasp That one listing is limited to 1000 keys. You can use the cursor to paginate.

Larry•5/23/23, 10:47 AM

Again, I'm not 100% clear. Let's say I have 10,000 keys that match my prefix. I make an initial list() call with that prefix and I'll get back a page plus a cursor. I can use that cursor on followup list() calls to get more pages. When it's all said and done, can I get back all 10,000 keys?

LLarry Again, I'm not 100% clear. Let's say I have 10,000 keys that match my prefix. I ...

asp•5/23/23, 10:47 AM

Yes. First call will list 1-1000. Second call will list 1001-2000, if you specify the cursor. And so on...

Aasp Yes. First call will list 1-1000. Second call will list 1001-2000, if you specif...

Larry•5/23/23, 10:48 AM

Cool. Thanks!

Farhian•5/23/23, 1:54 PM

Hi guys, we started seeing issues with accessing (reading) values from KV. Writing works, while reading seems to fail. The failures only seem to happen when the keys contain a combination of "/ @ +". Any thoughts? Thank you!

Farhian•5/23/23, 1:54 PM

Also, the keys and values are visible in the Portal, but retrieving them from the code fails.