My bad, I thought Region: Earth meant all the data was replicated globally? I’m in Australia so if b

My bad, I thought Region: Earth meant all the data was replicated globally? I’m in Australia so if both reads and writes are going somewhere off-continent that would explain these figures

mattOP•5/23/23, 5:42 AM

But I am confused about why they would be going off continent?

Mmatt My bad, I thought Region: Earth meant all the data was replicated globally? I’m ...

Hello, I’m Allie!•5/23/23, 7:07 AM

KV core is US/EU iirc, so writes go there. KV reads are cached locally, in the datacenter they are read from

HHello, I’m Allie!KV core is US/EU iirc, so writes go there. KV reads are cached locally, in the d...

beelzabub•5/23/23, 7:54 AM

Correct. https://developers.cloudflare.com/workers/learning/how-kv-works/. So yeah, writes from Australia may take a while. The first read ever for a key will be slow but subsequent reads should be fast. The reads don’t even need to be from the same data center. Anywhere in Australia will populate the cache for all of Australia.

How KV works · Cloudflare Workers docs

Workers KV is a global, low-latency, key-value data store. It stores data in a small number of centralized data centers, then caches that data in …

Bbeelzabub Correct. https://developers.cloudflare.com/workers/learning/how-kv-works/. So ye...

Hello, I’m Allie!•5/23/23, 7:59 AM

Oh, is that new with KV2? And how do the regions work there? Is it by country?

Erisa•5/23/23, 9:01 AM

The How KV worksHow KV works page has been there for a long time

EErisa The `How KV works` page has been there for a long time

Hello, I’m Allie!•5/23/23, 9:56 AM

No, I mean the thing where cache is applied over an entire region? The docs still seem to imply that cache is only in a single datacenter:

[M]ore popular values are cached in the data centers they are requested from.

Bbeelzabub That’s a fantastic question. I’ll note that option 2 also has a challenge in tha...

Larry•5/23/23, 10:29 AM

When you say "reminder that you can only list 1k at a time", do you mean that a page is only 1k max, or that all the pages can't return more than 1,000 items?

LLarry When you say "reminder that you can only list 1k at a time", do you mean that a ...

asp•5/23/23, 10:44 AM

That one listing is limited to 1000 keys. You can use the cursor to paginate.

Aasp That one listing is limited to 1000 keys. You can use the cursor to paginate.

Larry•5/23/23, 10:47 AM

Again, I'm not 100% clear. Let's say I have 10,000 keys that match my prefix. I make an initial list() call with that prefix and I'll get back a page plus a cursor. I can use that cursor on followup list() calls to get more pages. When it's all said and done, can I get back all 10,000 keys?

LLarry Again, I'm not 100% clear. Let's say I have 10,000 keys that match my prefix. I ...

asp•5/23/23, 10:47 AM

Yes. First call will list 1-1000. Second call will list 1001-2000, if you specify the cursor. And so on...

Aasp Yes. First call will list 1-1000. Second call will list 1001-2000, if you specif...

Larry•5/23/23, 10:48 AM

Cool. Thanks!

Farhian•5/23/23, 1:54 PM

Hi guys, we started seeing issues with accessing (reading) values from KV. Writing works, while reading seems to fail. The failures only seem to happen when the keys contain a combination of "/ @ +". Any thoughts? Thank you!

Farhian•5/23/23, 1:54 PM

Also, the keys and values are visible in the Portal, but retrieving them from the code fails.

Farhian•5/23/23, 2:12 PM

If I make the key shorter and remove the special characters, it works again.

FFarhian Hi guys, we started seeing issues with accessing (reading) values from KV. Writi...

asp•5/23/23, 2:37 PM

What is the error you are getting? Is this happening consistently?

Farhian•5/23/23, 2:37 PM

I can reproduce it 100% of the times

Farhian•5/23/23, 2:37 PM

No error, just that the returned value is null

FFarhian No error, just that the returned value is null

asp•5/23/23, 2:38 PM

Is this through a KV binding?

Farhian•5/23/23, 2:38 PM

yes

Farhian•5/23/23, 2:38 PM

both storing and reading the value

Farhian•5/23/23, 2:38 PM

I could give you a sample key

Farhian•5/23/23, 2:39 PM

here is one: "registrationChallenge/mylongname.something+test1@gmail.com"

Farhian•5/23/23, 2:39 PM

I can see the key and value in the portal, but the code when trying to read it receives null.

Farhian•5/23/23, 2:41 PM

If I change it to "registrationChallenge" (just to test), it works

FFarhian here is one: "registrationChallenge/mylongname.something+test1@gmail.com"

asp•5/23/23, 2:51 PM

Hmm, I am not able to reproduce this. Could you share the code snippet?

Farhian•5/23/23, 2:52 PM

Farhian•5/23/23, 2:53 PM

and this is for setting the value

Farhian•5/23/23, 2:54 PM

FFarhian Click to see attachment

asp•5/23/23, 2:57 PM

Just as a sanity check, could you please try logging the key just before the

env.KV.get()

env.KV.get()

statement?

asp•5/23/23, 2:57 PM

and also before the put

Farhian•5/23/23, 2:57 PM

yeah I did, copy and pasted in Notepad and compared them

Farhian•5/23/23, 3:00 PM

And this is code that worked until a couple of days ago. I checked everything and nothing was changed in the code in 2 weeks. The only change that we did do was to switched to a paid plan (we were on the free one before) - which should not be causing this

Farhian•5/23/23, 3:01 PM

If it works for you, I could continue investigating on my side and see if I missed anything.

asp•5/23/23, 3:02 PM

If it worked until a couple of days ago and nothing changed with your code, then I'd expect the issue to be with KV:

Farhian•5/23/23, 3:02 PM

yeah, but if it works for you...

HHello, I’m Allie!Yes

Plinpod•5/23/23, 3:51 PM

thanks!

Larry•5/23/23, 6:26 PM

I see comments here and there about the next gen KV. What improvements are expected for that? and when is it anticipated?

LLarry I see comments here and there about the next gen KV. What improvements are expec...

Chaika•5/23/23, 6:32 PM

check out this thread: kv
no eta though

Bbeelzabub It shouldn’t. If you’re doing writes concurrently then it’s possible you cached ...

b-fuze (Mike32)•5/23/23, 6:49 PM

Nvm, turns out I wasn't on the paid plan. I input my payment information and had assumed that it would automatically switch to the paid plan, but only after checking my email I see the notification from CF that it had blocked further requests

b-fuze (Mike32)•5/23/23, 7:09 PM

Now when I run my "key counting" code it's just 1 off from the actual count of inserted items

b-fuze (Mike32)•5/23/23, 7:10 PM

Which is fine for now

FFarhian here is one: "registrationChallenge/mylongname.something+test1@gmail.com"

beelzabub•5/24/23, 4:39 AM

I hadn’t seen this until just now. I’ll take a look tomorrow to see what’s going on.

FFarhian If it works for you, I could continue investigating on my side and see if I miss...

beelzabub•5/24/23, 4:45 AM

Is it possible that the issue is the expirationTtl? Like you’re telling KV to delete the data after 1 minute and if you are hitting the key before it’s written you’ll cache the 404 for a minute meaning the next write would not really be visible… are you sure this used to work without any changes from your side?

HHello, I’m Allie!Oh, is that new with KV2? And how do the regions work there? Is it by country?

beelzabub•5/24/23, 4:48 AM

It’s more based on proximity with coarse regions (eg Europe) being used for now as an MVP. We’re still tweaking things to optimize some of the behavior (Asia in particular isn’t fully optimized)

Bbeelzabub It’s more based on proximity with coarse regions (eg Europe) being used for now ...

Hello, I’m Allie!•5/24/23, 5:27 AM

Is there any way to use this new cache system with Workers(or other products)?

HHello, I’m Allie!Is there any way to use this new cache system with Workers(or other products)?

beelzabub•5/24/23, 5:43 AM

Not yet but we’re exploring that.

CChaika check out this thread: https://discord.com/channels/595317990191398933/874727019...

Larry•5/24/23, 11:08 AM

Thanks! Is there any published analysis (Cloudflare of 3rd party) that characterizes the current TTFB numbers?

Bbeelzabub Is it possible that the issue is the expirationTtl? Like you’re telling KV to de...

beelzabub•5/24/23, 12:48 PM

The only other thought I have is that we do have 1 slight change. We currently don’t upsert on put into the cache. That’s being said, I have no explanation for why the characters being used would change behavior.

Bbeelzabub The only other thought I have is that we do have 1 slight change. We currently d...

beelzabub•5/24/23, 1:59 PM

export default {
  async fetch(request, env, ctx) {
    if (request.headers.get('authorization') !== 'abc') {
      return new Response('not authorized', { status: 401 })
    }
    try {
      const key = "registrationChallenge/bob4@foo.bar"
      const value = Math.random().toString()
      const firstGet = await env.KV.get(key) ?? "<null>"
      await env.KV.put(key, value)
      const secondGet = await env.KV.get(key) ?? "<null>"
      return new Response(JSON.stringify({
        key,
        firstGet,
        updated: value,
        secondGet,
      }, undefined, 2));
    } catch (e) {
      return new Response(e.stack, { status: 500 });
    }
  },
};

export default {
  async fetch(request, env, ctx) {
    if (request.headers.get('authorization') !== 'abc') {
      return new Response('not authorized', { status: 401 })
    }
    try {
      const key = "registrationChallenge/bob4@foo.bar"
      const value = Math.random().toString()
      const firstGet = await env.KV.get(key) ?? "<null>"
      await env.KV.put(key, value)
      const secondGet = await env.KV.get(key) ?? "<null>"
      return new Response(JSON.stringify({
        key,
        firstGet,
        updated: value,
        secondGet,
      }, undefined, 2));
    } catch (e) {
      return new Response(e.stack, { status: 500 });
    }
  },
};

This prints for example:

{
  "key": "registrationChallenge/bob4@foo.bar",
  "firstGet": "<null>",
  "updated": "0.6567038844336677",
  "secondGet": "0.6567038844336677"
}

{
  "key": "registrationChallenge/bob4@foo.bar",
  "firstGet": "<null>",
  "updated": "0.6567038844336677",
  "secondGet": "0.6567038844336677"
}

beelzabub•5/24/23, 2:01 PM

Adding an

expirationTtl

expirationTtl

has no effect. We did start the rollout of the new KV caching infrastructure on Monday which might explain the observed difference but I don't see anything in Sentry, our logging, nor can I reproduce what you're observing...

export default { async fetch(request, env, ctx) { if (request.headers.get('authorization') !== 'abc') { return new Response('not authorized', { status: 401 }) } try { const key = "registrationChallenge/bob4@foo.bar" const value = Math.random().toString() const firstGet = await env.KV.get(key) ?? "<null>" await env.KV.put(key, value) const secondGet = await env.KV.get(key) ?? "<null>" return new Response(JSON.stringify({ key, firstGet, updated: value, secondGet, }, undefined, 2)); } catch (e) { return new Response(e.stack, { status: 500 }); } }, };