Hi, does anyone know if KV supports bulk reads? It seems really inefficient to be reading out 1 key

Ttalkingarmor Hi, does anyone know if KV supports bulk reads? It seems really inefficient to b...

H

It does not, though you can issue reads in parallel, by awaiting them all with

Promise.all

Promise.all

Promise.all

Promise.all

Ttalkingarmor Hi, does anyone know if KV supports bulk reads? It seems really inefficient to b...

D

DaniFoldi•4/1/24, 8:40 AM

bulk reads unfortunately aren't supported by the binding, and not even by the api, since individual values could be up to 25MB in size - if your data is small (eg. fits in key + metadata), you can use

list

list

to get up to 1k keys at a time

HHello, I’m Allie!It does not, though you can issue reads in parallel, by awaiting them all with `...

T

talkingarmorOP•4/1/24, 11:17 AM

Yeah, that's what I am doing, but it feels wrong initiating so many calls to load from KV.

DDaniFoldi bulk reads unfortunately aren't supported by the binding, and not even by the ap...

T

talkingarmorOP•4/1/24, 11:17 AM

The files I have are very small, usually less than 20kb compressed

T

talkingarmorOP•4/1/24, 11:18 AM

But my app requires loading 50 to 100 key value pairsr per function call

Ttalkingarmor The files I have are very small, usually less than 20kb compressed

D

DaniFoldi•4/1/24, 12:15 PM

Yeah metadata is unfortunately 1 or 2kb, can’t remember exactly, so too large, you can either chunk the files or batch them into a single larger kv value

DDaniFoldi Yeah metadata is unfortunately 1 or 2kb, can’t remember exactly, so too large, y...

T

talkingarmorOP•4/1/24, 1:37 PM

Okay thanks, btw, is it save to assume that looking KV and retrieving the values are significantly faster than R2? I haven't written any performance testing, but I assume that it would..

C

Chief•4/1/24, 2:25 PM

Does KV have read/write units? I.e. If I write 10MB into a value, is it the same price as writing 10KB?

CChief Does KV have read/write units? I.e. If I write 10MB into a value, is it the same...

I

Isaac McFadyen•4/1/24, 2:51 PM

Yes, same price. All writes are billed as the same amount.

I

Isaac McFadyen•4/1/24, 2:52 PM

KV has a limit of 25MB per value so that's the limit there, but anything below the limit is billed the same.

U

Unsmart•4/1/24, 5:15 PM

Durable objects is the only product that bills by units instead of just read or write

U

Unsmart•4/1/24, 5:15 PM

Everything else is simple pricing

Ttalkingarmor Okay thanks, btw, is it save to assume that looking KV and retrieving the values...

D

DaniFoldi•4/2/24, 7:34 AM

For values that are cached, retrievals should be much faster. For new reads, they come from one of the core locations. See https://developers.cloudflare.com/kv/reference/how-kv-works/ for how it works

O

OG | Career Fair Chair Assistant•4/3/24, 4:34 PM

Hi, I am having issues assessing my KV namespace. Also I created my worker on vscode but I cannot see it on my cloudflare web account. Could this be why there is a discrepancy in assessing my namespace and if it is or whyever this is happening, could I get assistance from someone here please?

Z

zegevlier•4/3/24, 6:55 PM

No, you can only use KV namespaces from the same account

Y

YokoWasis•4/4/24, 1:21 AM

KV is unbelieveably slow, any problem with cloudflare ?

YYokoWasis KV is unbelieveably slow, any problem with cloudflare ?

C

Chaika•4/4/24, 1:31 AM

no known issues on the status page with kv #cloudflare-status, and it looks ok on my end. What do you mean by "unbelievably slow"?

CChaika no known issues on the status page with kv #cloudflare-status, and it looks ok o...

Y

YokoWasis•4/4/24, 1:35 AM

it took 10 secs for a request that usually completed less than 1 sec

Y

YokoWasis•4/4/24, 1:38 AM

I don't know whether this is caused by workers or kv. The endpoint is basically just "read some kv key" and output the result as json

YYokoWasis I don't know whether this is caused by workers or kv. The endpoint is basically ...

C

Chaika•4/4/24, 1:40 AM

Do you still get 10s latency to it, or was that just one time? I tested the endpoint directly you sent in DM and I get ~50ms to it

CChaika Do you still get 10s latency to it, or was that just one time? I tested the endp...

Y

YokoWasis•4/4/24, 1:41 AM

i stil get it. it's not fixed by 10s, but more than 5sec

Y

YokoWasis•4/4/24, 1:41 AM

I am in Indonesia

C

Chaika•4/4/24, 1:42 AM

KV has two central stores, one in US, one in EU, and then caches at each location for the CacheTTL (which is min/default 60s), so Indonesia is a fair bit away but shouldn't be that high, and should be relatively low after it is cached

O

OG | Career Fair Chair Assistant•4/4/24, 2:00 AM

I am still unable to assess my kv namespace

CChaika KV has two central stores, one in US, one in EU, and then caches at each locatio...

T

Tony Leung•4/4/24, 6:08 AM

so now I know why my API is extremely slow to use KV as JWT's JTI storage. It would be nice to have KV to store their data at APAC - Oceania

OOG | Career Fair Chair Assistant I am still unable to assess my kv namespace

A

andrewp•4/4/24, 1:10 PM

How did you create your namespace? From your workers project repo, run

npx wrangler kv:namespace list

npx wrangler kv:namespace list

npx wrangler kv:namespace list

npx wrangler kv:namespace list and share the output please.

TTony Leung so now I know why my API is extremely slow to use KV as JWT's JTI storage. It wo...

C

Chaika•4/4/24, 1:12 PM

well YokoWasis's specific issue is his ISP was routing him from Indonesia to Europe for a bit, but eventually straightened out, so you shouldn't see as bad latency like he did, but yea it would be nice if there was a central store over there. Durable Objects and Durable Object Storage might be a good alternative, they just launched DOs in Australia yesterday

CChaika well YokoWasis's specific issue is his ISP was routing him from Indonesia to Eur...

T

Tony Leung•4/5/24, 3:49 AM

I'm utilising 60s - 120s of cache to reduce cold fetch from KV. But since my customers are majorly live in APAC, having a roundtrip to US is not ideal...

T

Tony Leung•4/5/24, 3:50 AM

luckily D1 has APAC location so the latency is ok

Ggrimpa There is a way to delete all records from a KV with a single command or API/Work...

H

Hello, I’m Allie!•4/5/24, 4:55 PM

Delete the KV Namespace and then recreate it?

Ggrimpa But is there a way to use the same namespace when recreating? Because I'm using ...

H

Hello, I’m Allie!•4/5/24, 5:05 PM

You would have to patch the Worker with the new ID, but otherwise yes

J

James•4/5/24, 9:50 PM

Yep. Changes may take up to 60 seconds: https://developers.cloudflare.com/kv/reference/how-kv-works/

Cloudflare Docs

How KV works · Cloudflare Workers KV

KV is a global, low-latency, key-value data store. It stores data in a small number of centralized data centers, then caches that data in Cloudflare’s …

L

LuisPeris•4/6/24, 2:10 PM

Hi!
What is the best way (and less intrusive) to use KV in an existing nodeJS project ?

V

Voids•4/6/24, 3:51 PM

where am I going wrong????

addEventListener('fetch', event => {
  event.respondWith(handleRequest(event.request))
})

// Initialize Cloudflare KV Namespace
const kvNamespaceId = '97';
const kvNamespace = KVNamespace(kvNamespaceId);

async function handleRequest(request) {
  // Handle POST request to store data
  if (request.method === 'POST') {
    const requestBody = await request.text();
    // Store data in KV
    await storeData('example_key', requestBody);
    return new Response('Data stored successfully!', { status: 200 });
  }

  // Handle GET request to retrieve data
  if (request.method === 'GET') {
    // Retrieve data from KV
    const data = await retrieveData('example_key');
    if (data) {
      return new Response(data, { status: 200 });
    } else {
      return new Response('No data found for the given key', { status: 404 });
    }
  }

  return new Response('Invalid request method', { status: 405 });
}

async function storeData(key, value) {
  try {
    // Store data in Cloudflare KV
    await kvNamespace.put(key, value);
  } catch (error) {
    console.error('Error storing data:', error);
    throw new Error('Failed to store data');
  }
}

async function retrieveData(key) {
  try {
    // Retrieve data from Cloudflare KV
    const data = await kvNamespace.get(key);
    if (data === null) {
      return null;
    }
    return data;
  } catch (error) {
    console.error('Error retrieving data:', error);
    throw new Error('Failed to retrieve data');
  }
}

addEventListener('fetch', event => {
  event.respondWith(handleRequest(event.request))
})

// Initialize Cloudflare KV Namespace
const kvNamespaceId = '97';
const kvNamespace = KVNamespace(kvNamespaceId);

async function handleRequest(request) {
  // Handle POST request to store data
  if (request.method === 'POST') {
    const requestBody = await request.text();
    // Store data in KV
    await storeData('example_key', requestBody);
    return new Response('Data stored successfully!', { status: 200 });
  }

  // Handle GET request to retrieve data
  if (request.method === 'GET') {
    // Retrieve data from KV
    const data = await retrieveData('example_key');
    if (data) {
      return new Response(data, { status: 200 });
    } else {
      return new Response('No data found for the given key', { status: 404 });
    }
  }

  return new Response('Invalid request method', { status: 405 });
}

async function storeData(key, value) {
  try {
    // Store data in Cloudflare KV
    await kvNamespace.put(key, value);
  } catch (error) {
    console.error('Error storing data:', error);
    throw new Error('Failed to store data');
  }
}

async function retrieveData(key) {
  try {
    // Retrieve data from Cloudflare KV
    const data = await kvNamespace.get(key);
    if (data === null) {
      return null;
    }
    return data;
  } catch (error) {
    console.error('Error retrieving data:', error);
    throw new Error('Failed to retrieve data');
  }
}

addEventListener('fetch', event => {
  event.respondWith(handleRequest(event.request))
})

// Initialize Cloudflare KV Namespace
const kvNamespaceId = '97';
const kvNamespace = KVNamespace(kvNamespaceId);

async function handleRequest(request) {
  // Handle POST request to store data
  if (request.method === 'POST') {
    const requestBody = await request.text();
    // Store data in KV
    await storeData('example_key', requestBody);
    return new Response('Data stored successfully!', { status: 200 });
  }

  // Handle GET request to retrieve data
  if (request.method === 'GET') {
    // Retrieve data from KV
    const data = await retrieveData('example_key');
    if (data) {
      return new Response(data, { status: 200 });
    } else {
      return new Response('No data found for the given key', { status: 404 });
    }
  }

  return new Response('Invalid request method', { status: 405 });
}

async function storeData(key, value) {
  try {
    // Store data in Cloudflare KV
    await kvNamespace.put(key, value);
  } catch (error) {
    console.error('Error storing data:', error);
    throw new Error('Failed to store data');
  }
}

async function retrieveData(key) {
  try {
    // Retrieve data from Cloudflare KV
    const data = await kvNamespace.get(key);
    if (data === null) {
      return null;
    }
    return data;
  } catch (error) {
    console.error('Error retrieving data:', error);
    throw new Error('Failed to retrieve data');
  }
}

addEventListener('fetch', event => {
  event.respondWith(handleRequest(event.request))
})

// Initialize Cloudflare KV Namespace
const kvNamespaceId = '97';
const kvNamespace = KVNamespace(kvNamespaceId);

async function handleRequest(request) {
  // Handle POST request to store data
  if (request.method === 'POST') {
    const requestBody = await request.text();
    // Store data in KV
    await storeData('example_key', requestBody);
    return new Response('Data stored successfully!', { status: 200 });
  }

  // Handle GET request to retrieve data
  if (request.method === 'GET') {
    // Retrieve data from KV
    const data = await retrieveData('example_key');
    if (data) {
      return new Response(data, { status: 200 });
    } else {
      return new Response('No data found for the given key', { status: 404 });
    }
  }

  return new Response('Invalid request method', { status: 405 });
}

async function storeData(key, value) {
  try {
    // Store data in Cloudflare KV
    await kvNamespace.put(key, value);
  } catch (error) {
    console.error('Error storing data:', error);
    throw new Error('Failed to store data');
  }
}

async function retrieveData(key) {
  try {
    // Retrieve data from Cloudflare KV
    const data = await kvNamespace.get(key);
    if (data === null) {
      return null;
    }
    return data;
  } catch (error) {
    console.error('Error retrieving data:', error);
    throw new Error('Failed to retrieve data');
  }
}

J

James•4/6/24, 3:56 PM

Are you using a framework or migrating from another platform?

const kvNamespaceId = '97';
const kvNamespace = KVNamespace(kvNamespaceId);

const kvNamespaceId = '97';
const kvNamespace = KVNamespace(kvNamespaceId);

const kvNamespaceId = '97';
const kvNamespace = KVNamespace(kvNamespaceId);

const kvNamespaceId = '97';
const kvNamespace = KVNamespace(kvNamespaceId);

This isn't a standard way to access a KV binding

J

James•4/6/24, 3:56 PM

See https://developers.cloudflare.com/kv/reference/kv-bindings/ for examples of how you'd normally do this in Workers

JJames Are you using a framework or migrating from another platform? ```js const kvNa...

V

Voids•4/6/24, 3:56 PM

what I want is to send and receive data on my website

J

James•4/6/24, 3:57 PM

Have you run through the code and getting started guide at https://developers.cloudflare.com/kv/get-started/ ? - your code here is very different

JJames Have you run through the code and getting started guide at https://developers.cl...

V

Voids•4/6/24, 4:04 PM

Im so confused

J

James•4/6/24, 4:05 PM

We're happy to help if you can expand on where you're confused

A good place to start would probably be where you got that original code from? Was it some framework?

V

Voids•4/6/24, 4:06 PM

my orginial code was given to me to test with fetching

J

James•4/6/24, 4:06 PM

Given to you by where/what exactly?

V

Voids•4/6/24, 4:06 PM

someone else who was helping me but they went to sleep

Hi, does anyone know if KV supports bulk reads? It seems really inefficient to be reading out 1 key

Similar Threads