I'm having an issue with the KV cache. Up until recently (sometime after mid-February?), I was able

P

I was trying to connect my Netlify Edge Function to any KV store and was having trouble finding one that would work (I've even written my own KV JS library in the past); so ended up creating a thin wrapper around Cloudflare's REST API that worked great!

const baseUrl = `https://api.cloudflare.com/client/v4/accounts/${CLOUDFLARE_ACCOUNT}/storage/kv/namespaces/${CLOUDFLARE_NAMESPACE}`;
const headers = { ... };  // Auth stuff

class CloudflareRest {
  EXPIRES = true;

  async get(key) {
    const res = await fetch(`${baseUrl}/values/${key}`, { headers });
    if (res.status === 404) return null; // It does not exist
    const data = await (res.headers.get("content-type").includes("json")
      ? res.json()
      : res.text());
    if (typeof data !== "string") return null;
    return JSON.parse(data);
  }

  async set(key, body, { expires }) {
    const expiration = expires ? `expiration_ttl=${expires}&` : "";
    await fetch(`${baseUrl}/values/${key}?${expiration}`, {
      method: "PUT",
      headers,
      body: JSON.stringify(body),
    });
    return key;
  }

  async keys(prefix) {
    const res = await fetch(`${baseUrl}/keys`, { headers });
    const data = await res.json();
    return data.result
      .map((it) => it.name)
      .filter((key) => key.startsWith(prefix));
  }

  async *iterate(prefix) {
    // ...
  }
}

const store = kv(CloudflareRest);
// console.log(await store.get("hello"));
// console.log(await store.set("hello", { world: "spins" }));
// console.log(await store.get("hello"));

const baseUrl = `https://api.cloudflare.com/client/v4/accounts/${CLOUDFLARE_ACCOUNT}/storage/kv/namespaces/${CLOUDFLARE_NAMESPACE}`;
const headers = { ... };  // Auth stuff

class CloudflareRest {
  EXPIRES = true;

  async get(key) {
    const res = await fetch(`${baseUrl}/values/${key}`, { headers });
    if (res.status === 404) return null; // It does not exist
    const data = await (res.headers.get("content-type").includes("json")
      ? res.json()
      : res.text());
    if (typeof data !== "string") return null;
    return JSON.parse(data);
  }

  async set(key, body, { expires }) {
    const expiration = expires ? `expiration_ttl=${expires}&` : "";
    await fetch(`${baseUrl}/values/${key}?${expiration}`, {
      method: "PUT",
      headers,
      body: JSON.stringify(body),
    });
    return key;
  }

  async keys(prefix) {
    const res = await fetch(`${baseUrl}/keys`, { headers });
    const data = await res.json();
    return data.result
      .map((it) => it.name)
      .filter((key) => key.startsWith(prefix));
  }

  async *iterate(prefix) {
    // ...
  }
}

const store = kv(CloudflareRest);
// console.log(await store.get("hello"));
// console.log(await store.set("hello", { world: "spins" }));
// console.log(await store.get("hello"));

const baseUrl = `https://api.cloudflare.com/client/v4/accounts/${CLOUDFLARE_ACCOUNT}/storage/kv/namespaces/${CLOUDFLARE_NAMESPACE}`;
const headers = { ... };  // Auth stuff

class CloudflareRest {
  EXPIRES = true;

  async get(key) {
    const res = await fetch(`${baseUrl}/values/${key}`, { headers });
    if (res.status === 404) return null; // It does not exist
    const data = await (res.headers.get("content-type").includes("json")
      ? res.json()
      : res.text());
    if (typeof data !== "string") return null;
    return JSON.parse(data);
  }

  async set(key, body, { expires }) {
    const expiration = expires ? `expiration_ttl=${expires}&` : "";
    await fetch(`${baseUrl}/values/${key}?${expiration}`, {
      method: "PUT",
      headers,
      body: JSON.stringify(body),
    });
    return key;
  }

  async keys(prefix) {
    const res = await fetch(`${baseUrl}/keys`, { headers });
    const data = await res.json();
    return data.result
      .map((it) => it.name)
      .filter((key) => key.startsWith(prefix));
  }

  async *iterate(prefix) {
    // ...
  }
}

const store = kv(CloudflareRest);
// console.log(await store.get("hello"));
// console.log(await store.set("hello", { world: "spins" }));
// console.log(await store.get("hello"));

const baseUrl = `https://api.cloudflare.com/client/v4/accounts/${CLOUDFLARE_ACCOUNT}/storage/kv/namespaces/${CLOUDFLARE_NAMESPACE}`;
const headers = { ... };  // Auth stuff

class CloudflareRest {
  EXPIRES = true;

  async get(key) {
    const res = await fetch(`${baseUrl}/values/${key}`, { headers });
    if (res.status === 404) return null; // It does not exist
    const data = await (res.headers.get("content-type").includes("json")
      ? res.json()
      : res.text());
    if (typeof data !== "string") return null;
    return JSON.parse(data);
  }

  async set(key, body, { expires }) {
    const expiration = expires ? `expiration_ttl=${expires}&` : "";
    await fetch(`${baseUrl}/values/${key}?${expiration}`, {
      method: "PUT",
      headers,
      body: JSON.stringify(body),
    });
    return key;
  }

  async keys(prefix) {
    const res = await fetch(`${baseUrl}/keys`, { headers });
    const data = await res.json();
    return data.result
      .map((it) => it.name)
      .filter((key) => key.startsWith(prefix));
  }

  async *iterate(prefix) {
    // ...
  }
}

const store = kv(CloudflareRest);
// console.log(await store.get("hello"));
// console.log(await store.set("hello", { world: "spins" }));
// console.log(await store.get("hello"));

PPaco ・ francisco.io I was trying to connect my Netlify Edge Function to _any_ KV store and was havin...

B

boop•3/21/25, 2:20 PM

Just keep in mind that the API has a limit of 1200 requests per 5 minute period

A

ajgeiss0702•3/22/25, 4:35 AM

anyone else seeing weird issues with kv?

A

ajgeiss0702•3/22/25, 4:35 AM

a bunch of my keys are randomly acting like they don't exist for a while, then show fine for a few minutes, then go back to acting like they don't exist for a while

A

ajgeiss0702•3/22/25, 4:50 AM

like these keys definitely exist

A

ajgeiss0702•3/22/25, 4:50 AM

yet they return null when I try to fetch them from workers

A

ajgeiss0702•3/22/25, 4:52 AM

even in the dashboard, they aren't always there for some reason. They show up most of the time in the dashboard, but not always

A

ajgeiss0702•3/22/25, 4:54 AM

this code has been running fine for almost a year, then suddenly tonight its being weird, which makes me think its a KV issue

Aajgeiss0702 this code has been running fine for almost a year, then suddenly tonight its bei...

I

Isaac McFadyen•3/22/25, 4:58 AM

Can you send the KV namespace ID please? (it's fine/safe to share publicly) Me or someone else can try and escalate if we have the chance.

A

ajgeiss0702•3/22/25, 4:58 AM

kv namespace:

af574b5f161d4d3c938fae885549b417

af574b5f161d4d3c938fae885549b417

af574b5f161d4d3c938fae885549b417

af574b5f161d4d3c938fae885549b417
account:

f55b85c8a963663b11036975203c63c0

f55b85c8a963663b11036975203c63c0

f55b85c8a963663b11036975203c63c0

f55b85c8a963663b11036975203c63c0

I

Isaac McFadyen•3/22/25, 4:58 AM

And when you do a

get

get

on the key are you specifying a cacheTtl? If so, what is it?

IIsaac McFadyen And when you do a `get` on the key are you specifying a cacheTtl? If so, what is...

A

ajgeiss0702•3/22/25, 4:58 AM

initially I did not, however as a troubleshooting step I tried setting it to

60

60 which seemed to not make a difference

I

Isaac McFadyen•3/22/25, 4:59 AM

The fact that the dashboard is also sometimes not returning the keys makes me think it's not your code, so I'll go ahead and see if I can raise this.

A

ajgeiss0702•3/22/25, 4:59 AM

the issue is also occurring across two different workers if that helps

A

ajgeiss0702•3/22/25, 5:00 AM

the issue seems to be happening with the

2025/03/21:snippet

2025/03/21:snippet

2025/03/21:snippet

2025/03/21:snippet and

2025/03/21:videoId

2025/03/21:videoId

2025/03/21:videoId

2025/03/21:videoId keys. Other keys that were written around the same time (

2025/03/21:mainShowStart

2025/03/21:mainShowStart

2025/03/21:mainShowStart

2025/03/21:mainShowStart would have been written within a few minutes of the others) do not have this issue oddly

IIsaac McFadyen The fact that the dashboard is also sometimes not returning the keys makes me th...

O

o7•3/22/25, 5:01 AM

I had a similar issue not too long ago where nothing would return and it would spam list requests. My only fix was recreating the namespace
kv

A

ajgeiss0702•3/22/25, 5:02 AM

im not sure if its the same issue, as listing in my dashbaord appears to work for everything except for these two keys

O

o7•3/22/25, 5:02 AM

ah interesting. still wanted to share it incase it helped

IIsaac McFadyen The fact that the dashboard is also sometimes not returning the keys makes me th...

A

ajgeiss0702•3/22/25, 5:02 AM

thank you!

I

Isaac McFadyen•3/22/25, 5:03 AM

OK I escalated this, I'm heading to sleep but when/if CSUP gets to it I'll ask them to communicate here directly. No promises on timeline though.

A

ajgeiss0702•3/22/25, 5:04 AM

thank you. i'll see if I can work around it for now, as ive got things that depend on reading this data that I want to be processed ASAP

Aajgeiss0702 the issue seems to be happening with the `2025/03/21:snippet` and `2025/03/21:vi...

I

Isaac McFadyen•3/22/25, 5:06 AM

Do you know around when the keys were written?

A

ajgeiss0702•3/22/25, 5:06 AM

a few hours ago, i can get a rough time 1s

A

ajgeiss0702•3/22/25, 5:06 AM

it would have been somewhere around this timestamp:

2025-03-21T23:56:12Z

2025-03-21T23:56:12Z

2025-03-21T23:56:12Z

2025-03-21T23:56:12Z

A

ajgeiss0702•3/22/25, 5:07 AM

probably before, but might have been the same time. i don't remember exactly what order theyre written, but i can look if important

I

Isaac McFadyen•3/22/25, 5:07 AM

I'm wondering if it would perhaps have occurred during this incident earlier https://discord.com/channels/595317990191398933/895794943182909470/1352765245877391421, I didn't think KV and R2 were connected but perhaps they now share some infra internally that might have failed?

Aajgeiss0702 probably before, but might have been the same time. i don't remember exactly wha...

I

Isaac McFadyen•3/22/25, 5:07 AM

No don't worry, just wanted to get an approx. timestamp.

A

ajgeiss0702•3/22/25, 5:14 AM

the earliest it could been written is 23:45 UTC

A

ajgeiss0702•3/22/25, 5:14 AM

which it looks like the r2 issue was resolved by then

A

ajgeiss0702•3/22/25, 5:27 AM

please ping me if there is any movement on this

A

ajgeiss0702•3/22/25, 6:08 AM

it looks like it might be fixed now? its been showing consistently the past few minutes

Aajgeiss0702 it looks like it might be fixed now? its been showing consistently the past few ...

C

cfmicky•3/22/25, 4:48 PM

Hey there, thanks for the update on this. Are you still seeing the issue with the KV keys that you were initially reporting?

Ccfmicky Hey there, thanks for the update on this. Are you still seeing the issue with th...

A

ajgeiss0702•3/22/25, 8:02 PM

it looks like those keys I mentioned are still sometimes not showing up in the dashboard.
but I have a cron task that collects those keys into a single one, and that appears to have read them correctly this time so its no longer interrupting my application

A

ajgeiss0702•3/22/25, 8:02 PM

also i realized I forgot to mention this before, but those keys also have an expiration set (15 days)

A

ajgeiss0702•3/22/25, 8:04 PM

If it helps, here is what this kv namespace is for:
each of those keys with colons are "fragments" of an event. they are fragmented because each piece of info is available at different times and I need to record them. that cron then combines them into a single key after the event for easier reading, so those fragments get an expiration ttl of 15 days (enough so that I can diagnose things if something goes wrong with the collapsing, and small enough so that they don't stick around forever wasting storage)

A

ajgeiss0702•3/22/25, 8:06 PM

this part of the application was made before D1 existed, otherwise I probably would have used D1. Once D1 read replicas are launched, i might consider migrating this specific namespace to d1 instead, since that would probably better fit this usecase

A

ajgeiss0702•3/22/25, 8:11 PM

even though its no longer effecting my application, lmk if theres any other info I can provide if you want to look into preventing this from happening in the future

Z

zecanardOP•3/22/25, 8:18 PM

This is the same issue I’m experiencing kv and I’m still seeing it. I can refresh the Dashboard, and randomly get my old or new KV entries.

T

thomasgauvin•3/24/25, 4:33 PM

We're looking into how to resolve this, it seems like some routing was changed up which is making it take more time for you to see updates in the dashboard

T

Tuấn Nguyễn•3/26/25, 2:27 AM

I am having issue with KV.
I query a record from D1 Database, it took me 15ms, then I cache that record as json in KV.
I read that record from KV with record id, so I expected it should be faster than D1 but it actually took me 500ms, way slower.
Am I doing it wrong?

T

Tuấn Nguyễn•3/26/25, 2:30 AM

TTuấn Nguyễn I am having issue with KV. I query a record from D1 Database, it took me 15ms, t...

T

Tuấn Nguyễn•3/26/25, 3:25 PM

I finally found a bug that continuously write and invalidate the cache. MY BADDDDDD.

Bbanyan My site was too slow, so I had to switch from KV to Cache API. Is it normal for ...

C

Chaika•3/26/25, 3:38 PM

Read status to show this many Not Found results?

Depends entirely on how you're calling KV. If you're using it to serve static assets and people are requesting non-existent paths, not too surprising. If you have a manifest or something in front, making it so every KV get request should always exist, could be a bug in your code.

Your p50 is really high there, how are you using KV? Could you increase cacheTTL/hit the same keys more?

T

thomasgauvin•3/27/25, 3:20 PM

KV is a global key-value store. If there's not heavy traffic on your keys, then they won't be getting cached in all of the Cloudflare locations. When this is the case, then it needs to fetch the result from the central stores for key-value data. See https://developers.cloudflare.com/kv/concepts/how-kv-works/ for an explanation on this

Cloudflare Docs

How KV works · Cloudflare Workers KV docs

KV is a global, low-latency, key-value data store. It stores data in a small number of centralized data centers, then caches that data in Cloudflare's data centers after access.

S

Sam•3/27/25, 8:15 PM

Can we store complex JSON in KV as a value? I have a json that’s 4,5 levels deep and need to know if that’s possible with KV.

SSam Can we store complex JSON in KV as a value? I have a json that’s 4,5 levels deep...

I

Isaac McFadyen•3/27/25, 8:34 PM

At its core, KV stores bytes so any value (including JSON) that can be serialized/deserialized as bytes can be stored in KV.

I'm having an issue with the KV cache. Up until recently (sometime after mid-February?), I was able

Similar Threads

Similar Threads

Similar Threads