Again, I'm not 100% clear. Let's say I have 10,000 keys that match my prefix. I make an initial list

LLarry Again, I'm not 100% clear. Let's say I have 10,000 keys that match my prefix. I ...

A

asp•5/23/23, 10:47 AM

Yes. First call will list 1-1000. Second call will list 1001-2000, if you specify the cursor. And so on...

Aasp Yes. First call will list 1-1000. Second call will list 1001-2000, if you specif...

L

LarryOP•5/23/23, 10:48 AM

Cool. Thanks!

F

Farhian•5/23/23, 1:54 PM

Hi guys, we started seeing issues with accessing (reading) values from KV. Writing works, while reading seems to fail. The failures only seem to happen when the keys contain a combination of "/ @ +". Any thoughts? Thank you!

F

Farhian•5/23/23, 1:54 PM

Also, the keys and values are visible in the Portal, but retrieving them from the code fails.

F

Farhian•5/23/23, 2:12 PM

If I make the key shorter and remove the special characters, it works again.

FFarhian Hi guys, we started seeing issues with accessing (reading) values from KV. Writi...

A

asp•5/23/23, 2:37 PM

What is the error you are getting? Is this happening consistently?

F

Farhian•5/23/23, 2:37 PM

I can reproduce it 100% of the times

F

Farhian•5/23/23, 2:37 PM

No error, just that the returned value is null

FFarhian No error, just that the returned value is null

A

asp•5/23/23, 2:38 PM

Is this through a KV binding?

F

Farhian•5/23/23, 2:38 PM

yes

F

Farhian•5/23/23, 2:38 PM

both storing and reading the value

F

Farhian•5/23/23, 2:38 PM

I could give you a sample key

F

Farhian•5/23/23, 2:39 PM

here is one: "registrationChallenge/mylongname.something+test1@gmail.com"

F

Farhian•5/23/23, 2:39 PM

I can see the key and value in the portal, but the code when trying to read it receives null.

F

Farhian•5/23/23, 2:41 PM

If I change it to "registrationChallenge" (just to test), it works

FFarhian here is one: "registrationChallenge/mylongname.something+test1@gmail.com"

A

asp•5/23/23, 2:51 PM

Hmm, I am not able to reproduce this. Could you share the code snippet?

F

Farhian•5/23/23, 2:52 PM

F

Farhian•5/23/23, 2:53 PM

and this is for setting the value

F

Farhian•5/23/23, 2:54 PM

FFarhian Click to see attachment

A

asp•5/23/23, 2:57 PM

Just as a sanity check, could you please try logging the key just before the

env.KV.get()

env.KV.get()

env.KV.get()

env.KV.get() statement?

A

asp•5/23/23, 2:57 PM

and also before the put

F

Farhian•5/23/23, 2:57 PM

yeah I did, copy and pasted in Notepad and compared them

F

Farhian•5/23/23, 3:00 PM

And this is code that worked until a couple of days ago. I checked everything and nothing was changed in the code in 2 weeks. The only change that we did do was to switched to a paid plan (we were on the free one before) - which should not be causing this

F

Farhian•5/23/23, 3:01 PM

If it works for you, I could continue investigating on my side and see if I missed anything.

A

asp•5/23/23, 3:02 PM

If it worked until a couple of days ago and nothing changed with your code, then I'd expect the issue to be with KV:

F

Farhian•5/23/23, 3:02 PM

yeah, but if it works for you...

HHello, I’m Allie!Yes

P

Plinpod•5/23/23, 3:51 PM

thanks!

L

LarryOP•5/23/23, 6:26 PM

I see comments here and there about the next gen KV. What improvements are expected for that? and when is it anticipated?

LLarry I see comments here and there about the next gen KV. What improvements are expec...

C

Chaika•5/23/23, 6:32 PM

check out this thread: kv
no eta though

Bbeelzabub It shouldn’t. If you’re doing writes concurrently then it’s possible you cached ...

B

b-fuze (Mike32)•5/23/23, 6:49 PM

Nvm, turns out I wasn't on the paid plan. I input my payment information and had assumed that it would automatically switch to the paid plan, but only after checking my email I see the notification from CF that it had blocked further requests

B

b-fuze (Mike32)•5/23/23, 7:09 PM

Now when I run my "key counting" code it's just 1 off from the actual count of inserted items

B

b-fuze (Mike32)•5/23/23, 7:10 PM

Which is fine for now

FFarhian here is one: "registrationChallenge/mylongname.something+test1@gmail.com"

B

beelzabub•5/24/23, 4:39 AM

I hadn’t seen this until just now. I’ll take a look tomorrow to see what’s going on.

FFarhian If it works for you, I could continue investigating on my side and see if I miss...

B

beelzabub•5/24/23, 4:45 AM

Is it possible that the issue is the expirationTtl? Like you’re telling KV to delete the data after 1 minute and if you are hitting the key before it’s written you’ll cache the 404 for a minute meaning the next write would not really be visible… are you sure this used to work without any changes from your side?

HHello, I’m Allie!Oh, is that new with KV2? And how do the regions work there? Is it by country?

B

beelzabub•5/24/23, 4:48 AM

It’s more based on proximity with coarse regions (eg Europe) being used for now as an MVP. We’re still tweaking things to optimize some of the behavior (Asia in particular isn’t fully optimized)

Bbeelzabub It’s more based on proximity with coarse regions (eg Europe) being used for now ...

H

Hello, I’m Allie!•5/24/23, 5:27 AM

Is there any way to use this new cache system with Workers(or other products)?

HHello, I’m Allie!Is there any way to use this new cache system with Workers(or other products)?

B

beelzabub•5/24/23, 5:43 AM

Not yet but we’re exploring that.

CChaika check out this thread: https://discord.com/channels/595317990191398933/874727019...

L

LarryOP•5/24/23, 11:08 AM

Thanks! Is there any published analysis (Cloudflare of 3rd party) that characterizes the current TTFB numbers?

Bbeelzabub Is it possible that the issue is the expirationTtl? Like you’re telling KV to de...

B

beelzabub•5/24/23, 12:48 PM

The only other thought I have is that we do have 1 slight change. We currently don’t upsert on put into the cache. That’s being said, I have no explanation for why the characters being used would change behavior.

Bbeelzabub The only other thought I have is that we do have 1 slight change. We currently d...

B

beelzabub•5/24/23, 1:59 PM

export default {
  async fetch(request, env, ctx) {
    if (request.headers.get('authorization') !== 'abc') {
      return new Response('not authorized', { status: 401 })
    }
    try {
      const key = "registrationChallenge/bob4@foo.bar"
      const value = Math.random().toString()
      const firstGet = await env.KV.get(key) ?? "<null>"
      await env.KV.put(key, value)
      const secondGet = await env.KV.get(key) ?? "<null>"
      return new Response(JSON.stringify({
        key,
        firstGet,
        updated: value,
        secondGet,
      }, undefined, 2));
    } catch (e) {
      return new Response(e.stack, { status: 500 });
    }
  },
};

export default {
  async fetch(request, env, ctx) {
    if (request.headers.get('authorization') !== 'abc') {
      return new Response('not authorized', { status: 401 })
    }
    try {
      const key = "registrationChallenge/bob4@foo.bar"
      const value = Math.random().toString()
      const firstGet = await env.KV.get(key) ?? "<null>"
      await env.KV.put(key, value)
      const secondGet = await env.KV.get(key) ?? "<null>"
      return new Response(JSON.stringify({
        key,
        firstGet,
        updated: value,
        secondGet,
      }, undefined, 2));
    } catch (e) {
      return new Response(e.stack, { status: 500 });
    }
  },
};

export default {
  async fetch(request, env, ctx) {
    if (request.headers.get('authorization') !== 'abc') {
      return new Response('not authorized', { status: 401 })
    }
    try {
      const key = "registrationChallenge/bob4@foo.bar"
      const value = Math.random().toString()
      const firstGet = await env.KV.get(key) ?? "<null>"
      await env.KV.put(key, value)
      const secondGet = await env.KV.get(key) ?? "<null>"
      return new Response(JSON.stringify({
        key,
        firstGet,
        updated: value,
        secondGet,
      }, undefined, 2));
    } catch (e) {
      return new Response(e.stack, { status: 500 });
    }
  },
};

export default {
  async fetch(request, env, ctx) {
    if (request.headers.get('authorization') !== 'abc') {
      return new Response('not authorized', { status: 401 })
    }
    try {
      const key = "registrationChallenge/bob4@foo.bar"
      const value = Math.random().toString()
      const firstGet = await env.KV.get(key) ?? "<null>"
      await env.KV.put(key, value)
      const secondGet = await env.KV.get(key) ?? "<null>"
      return new Response(JSON.stringify({
        key,
        firstGet,
        updated: value,
        secondGet,
      }, undefined, 2));
    } catch (e) {
      return new Response(e.stack, { status: 500 });
    }
  },
};

This prints for example:

{
  "key": "registrationChallenge/bob4@foo.bar",
  "firstGet": "<null>",
  "updated": "0.6567038844336677",
  "secondGet": "0.6567038844336677"
}

{
  "key": "registrationChallenge/bob4@foo.bar",
  "firstGet": "<null>",
  "updated": "0.6567038844336677",
  "secondGet": "0.6567038844336677"
}

{
  "key": "registrationChallenge/bob4@foo.bar",
  "firstGet": "<null>",
  "updated": "0.6567038844336677",
  "secondGet": "0.6567038844336677"
}

{
  "key": "registrationChallenge/bob4@foo.bar",
  "firstGet": "<null>",
  "updated": "0.6567038844336677",
  "secondGet": "0.6567038844336677"
}

B

beelzabub•5/24/23, 2:01 PM

Adding an

expirationTtl

expirationTtl

expirationTtl

expirationTtl has no effect. We did start the rollout of the new KV caching infrastructure on Monday which might explain the observed difference but I don't see anything in Sentry, our logging, nor can I reproduce what you're observing...

B

beelzabub•5/24/23, 2:13 PM

The more complex key works too:

{
  "key": "registrationChallenge/helloworld+1////22///%2f+++...\\<<<>/@test.com:443//",
  "firstGet": "<null>",
  "updated": "0.41442319206749634",
  "secondGet": "0.41442319206749634"
}

{
  "key": "registrationChallenge/helloworld+1////22///%2f+++...\\<<<>/@test.com:443//",
  "firstGet": "<null>",
  "updated": "0.41442319206749634",
  "secondGet": "0.41442319206749634"
}

{
  "key": "registrationChallenge/helloworld+1////22///%2f+++...\\<<<>/@test.com:443//",
  "firstGet": "<null>",
  "updated": "0.41442319206749634",
  "secondGet": "0.41442319206749634"
}

{
  "key": "registrationChallenge/helloworld+1////22///%2f+++...\\<<<>/@test.com:443//",
  "firstGet": "<null>",
  "updated": "0.41442319206749634",
  "secondGet": "0.41442319206749634"
}

Bbeelzabub The more complex key works too: ``` { "key": "registrationChallenge/helloworld...