@Isaac McFadyen | YYZ01 do you know if there's a way to count keys? Something like the `list()` meth

@Isaac McFadyen | YYZ01 do you know if there's a way to count keys? Something like the

list()

list()

method but without returning the actual keys and just a count?

Bb-fuze (Mike32)@Isaac McFadyen | YYZ01 do you know if there's a way to count keys? Something li...

Isaac McFadyen•5/23/23, 1:32 AM

I don't believe so, no, aside from just manually calling and counting that way (which is error-prone).

b-fuze (Mike32)OP•5/23/23, 1:32 AM

I'll have to do that then. I know that there's eventual consistency involved, but I'll need a count even if it's a rough count

Bb-fuze (Mike32)I'll have to do that then. I know that there's eventual consistency involved, bu...

Isaac McFadyen•5/23/23, 1:33 AM

~~Don't quote me on this - but I don't think calls use the cache, actually, and so won't be subject to the eventual consistency part.~~ See below.

Isaac McFadyen•5/23/23, 1:38 AM

Easy way to test, actually - if you call multiple times in a row, if the calls all take then same time then they don't get cached, if the first is slower than the rest they do get cached.

Isaac McFadyen•5/23/23, 1:43 AM

I am wrong, just tested and KV operations do indeed get cached

Isaac McFadyen•5/23/23, 1:43 AM

Sorry about that.

b-fuze (Mike32)OP•5/23/23, 1:45 AM

Np, thanks for checking!

MMesorchoo Thanks, any plans to add something like the bulk import with wrangler in the fut...

beelzabub•5/23/23, 1:53 AM

That's more a question for the wrangler channel I think.

IIsaac McFadyen I am wrong, just tested and KV `list` operations do indeed get cached 😄

beelzabub•5/23/23, 1:55 AM

They get cached for the exact same sets of parameters. The cache key is built from the parameters (prefix, limit, or cursor) and cached for max 1 minute (IIRC you can't customize the cache TTL currently).

Bbeelzabub They get cached for the exact same sets of parameters. The cache key is built fr...

b-fuze (Mike32)OP•5/23/23, 3:09 AM

Do you know if has an upper limit somehow? After 7500 keys it won't return another , but I should have inserted more keys than that

matt•5/23/23, 4:38 AM

the latency on these kv gets/puts seems awfully high no?

Bb-fuze (Mike32)Do you know if `list` has an upper limit somehow? After 7500 keys it won't retur...

beelzabub•5/23/23, 5:22 AM

It shouldn’t. If you’re doing writes concurrently then it’s possible you cached the end of the list and need to wait a minute or so. What’s your namespace?

Mmatt the latency on these kv gets/puts seems awfully high no?

beelzabub•5/23/23, 5:23 AM

Which geographic location is this happening in and are you sure the gets are happening on a key that’s supposed to be already cached? Also is this a change in behavior or a new measurement you’re looking at

Bbeelzabub Which geographic location is this happening in and are you sure the gets are hap...

matt•5/23/23, 5:41 AM

My bad, I thought Region: Earth meant all the data was replicated globally? I’m in Australia so if both reads and writes are going somewhere off-continent that would explain these figures

matt•5/23/23, 5:42 AM

But I am confused about why they would be going off continent?

Mmatt My bad, I thought Region: Earth meant all the data was replicated globally? I’m ...

Hello, I’m Allie!•5/23/23, 7:07 AM

KV core is US/EU iirc, so writes go there. KV reads are cached locally, in the datacenter they are read from

HHello, I’m Allie!KV core is US/EU iirc, so writes go there. KV reads are cached locally, in the d...

beelzabub•5/23/23, 7:54 AM

Correct. https://developers.cloudflare.com/workers/learning/how-kv-works/. So yeah, writes from Australia may take a while. The first read ever for a key will be slow but subsequent reads should be fast. The reads don’t even need to be from the same data center. Anywhere in Australia will populate the cache for all of Australia.

How KV works · Cloudflare Workers docs

Workers KV is a global, low-latency, key-value data store. It stores data in a small number of centralized data centers, then caches that data in …

Bbeelzabub Correct. https://developers.cloudflare.com/workers/learning/how-kv-works/. So ye...

Hello, I’m Allie!•5/23/23, 7:59 AM

Oh, is that new with KV2? And how do the regions work there? Is it by country?

Erisa•5/23/23, 9:01 AM

The page has been there for a long time

EErisa The `How KV works` page has been there for a long time

Hello, I’m Allie!•5/23/23, 9:56 AM

No, I mean the thing where cache is applied over an entire region? The docs still seem to imply that cache is only in a single datacenter:

[M]ore popular values are cached in the data centers they are requested from.

Bbeelzabub That’s a fantastic question. I’ll note that option 2 also has a challenge in tha...

Larry•5/23/23, 10:29 AM

When you say "reminder that you can only list 1k at a time", do you mean that a page is only 1k max, or that all the pages can't return more than 1,000 items?

LLarry When you say "reminder that you can only list 1k at a time", do you mean that a ...

asp•5/23/23, 10:44 AM

That one listing is limited to 1000 keys. You can use the cursor to paginate.

Aasp That one listing is limited to 1000 keys. You can use the cursor to paginate.

Larry•5/23/23, 10:47 AM

Again, I'm not 100% clear. Let's say I have 10,000 keys that match my prefix. I make an initial list() call with that prefix and I'll get back a page plus a cursor. I can use that cursor on followup list() calls to get more pages. When it's all said and done, can I get back all 10,000 keys?

LLarry Again, I'm not 100% clear. Let's say I have 10,000 keys that match my prefix. I ...

asp•5/23/23, 10:47 AM

Yes. First call will list 1-1000. Second call will list 1001-2000, if you specify the cursor. And so on...

Aasp Yes. First call will list 1-1000. Second call will list 1001-2000, if you specif...

Larry•5/23/23, 10:48 AM

Cool. Thanks!

Farhian•5/23/23, 1:54 PM

Hi guys, we started seeing issues with accessing (reading) values from KV. Writing works, while reading seems to fail. The failures only seem to happen when the keys contain a combination of "/ @ +". Any thoughts? Thank you!

Farhian•5/23/23, 1:54 PM

Also, the keys and values are visible in the Portal, but retrieving them from the code fails.

Farhian•5/23/23, 2:12 PM

If I make the key shorter and remove the special characters, it works again.

FFarhian Hi guys, we started seeing issues with accessing (reading) values from KV. Writi...

asp•5/23/23, 2:37 PM

What is the error you are getting? Is this happening consistently?

Farhian•5/23/23, 2:37 PM

I can reproduce it 100% of the times

Farhian•5/23/23, 2:37 PM

No error, just that the returned value is null

FFarhian No error, just that the returned value is null

asp•5/23/23, 2:38 PM

Is this through a KV binding?

Farhian•5/23/23, 2:38 PM

yes

Farhian•5/23/23, 2:38 PM

both storing and reading the value

Farhian•5/23/23, 2:38 PM

I could give you a sample key

Farhian•5/23/23, 2:39 PM

here is one: "registrationChallenge/mylongname.something+test1@gmail.com"

Farhian•5/23/23, 2:39 PM

I can see the key and value in the portal, but the code when trying to read it receives null.

Farhian•5/23/23, 2:41 PM

If I change it to "registrationChallenge" (just to test), it works

FFarhian here is one: "registrationChallenge/mylongname.something+test1@gmail.com"

asp•5/23/23, 2:51 PM

Hmm, I am not able to reproduce this. Could you share the code snippet?

Farhian•5/23/23, 2:52 PM

Farhian•5/23/23, 2:53 PM

and this is for setting the value

Farhian•5/23/23, 2:54 PM

FFarhian Click to see attachment

asp•5/23/23, 2:57 PM

Just as a sanity check, could you please try logging the key just before the statement?

asp•5/23/23, 2:57 PM

and also before the put

Farhian•5/23/23, 2:57 PM

yeah I did, copy and pasted in Notepad and compared them

Farhian•5/23/23, 3:00 PM

And this is code that worked until a couple of days ago. I checked everything and nothing was changed in the code in 2 weeks. The only change that we did do was to switched to a paid plan (we were on the free one before) - which should not be causing this

Farhian•5/23/23, 3:01 PM

If it works for you, I could continue investigating on my side and see if I missed anything.

asp•5/23/23, 3:02 PM

If it worked until a couple of days ago and nothing changed with your code, then I'd expect the issue to be with KV:

Farhian•5/23/23, 3:02 PM

yeah, but if it works for you...

HHello, I’m Allie!Yes

Plinpod•5/23/23, 3:51 PM

thanks!