Code of your test? Are you using `kv.get(key, { cacheTtl: 60 })` or the cache API?

Code of your test?

Are you using

kv.get(key, { cacheTtl: 60 })

kv.get(key, { cacheTtl: 60 })

kv.get(key, { cacheTtl: 60 })

kv.get(key, { cacheTtl: 60 }) or the cache API?

VVikash Code of your test? Are you using `kv.get(key, { cacheTtl: 60 })` or the cache ...

SK•12/10/23, 1:24 AM

https://gist.github.com/Repo-SK/76775be58020c964c17eae45be8b3104

SK•12/10/23, 1:25 AM

/kv uses kv.get with a cacheTtl value of an hour (3600) and /kv-cache-api uses the cache api

SK•12/10/23, 1:25 AM

for the benchmark simply send requests to both endpoints and extract the latencies from the html returned (I used a python script to send 100 requests to each)

kian•12/10/23, 6:18 PM

As in the read takes that long to fulfill, or that long until you see the new value?

zehawk•12/16/23, 3:09 PM

Where does one see how much storage is being used by KV? I can never get an accurate value. I got billed for 1.69GB while the KV page shows 67KB. So what gives?

Mmatt_d @izdi No, this behavior is an inescapable aspect of WorkersKV.

Advany•12/17/23, 1:00 PM

is it already as fast as 5 seconds? I though it would take up to 60 seconds to get consistent with the write

Zzehawk Where does one see how much storage is being used by KV? I can never get an accu...

Matt Silverlock•12/17/23, 1:03 PM

Please open a Support ticket here.

AAdvany is it already as fast as 5 seconds? I though it would take up to 60 seconds to g...

Matt Silverlock•12/17/23, 2:32 PM

60s for values to refresh (eventual consistency). If you read a key, you will implicitly cache the empty/null value. You can read new keys faster than 60s on first read.

Advany•12/17/23, 2:33 PM

Awesome! Just wanted to verify. Hopefully one day we will see faster refreshes but good for now!

AAdvany is it already as fast as 5 seconds? I though it would take up to 60 seconds to g...

Hello, I’m Allie!•12/17/23, 2:39 PM

This also assumes you haven’t set a higher CacheTTL

HHello, I’m Allie!This also assumes you haven’t set a higher CacheTTL

Advany•12/17/23, 3:07 PM

Yeah a while back there was talk about a new version of KV that would refresh faster even with a higher CacheTTL. There where talks of 15 seconds

AAdvany Yeah a while back there was talk about a new version of KV that would refresh fa...

Hello, I’m Allie!•12/17/23, 3:28 PM

Yeah, that was v2, but it got rolled back a while ago

bonald•12/18/23, 2:55 AM

16ms latency for kv , is this chat real? was looking the test redis vs kv . i guess theres no test to do. is this real?

kian•12/18/23, 2:56 AM

For a cached read? Sounds right to me.

Bbonald 16ms latency for kv , is this chat real? was looking the test redis vs kv . i gu...

Advany•12/18/23, 8:41 AM

a kv is cached locally using Cache. Thats why second reads on the same location are faster

HHello, I’m Allie!Yeah, that was v2, but it got rolled back a while ago

Advany•12/18/23, 8:41 AM

aaaah good to know they are still working on that!

Befus•12/20/23, 11:50 AM

How would I store an image inside KV? Not using cloudflare images

BBefus How would I store an image inside KV? Not using cloudflare images

Isaac McFadyen•12/20/23, 12:01 PM

You'd probably encode it as an ArrayBuffer. Keep in mind that will buffer the image into the Worker's RAM, but I don't think there's any way to stream directly to/from KV.

Isaac McFadyen•12/20/23, 12:01 PM

Oh actually, there is!

Isaac McFadyen•12/20/23, 12:01 PM

You can directly write a ReadableStream: https://developers.cloudflare.com/kv/api/write-key-value-pairs/

Isaac McFadyen•12/20/23, 12:02 PM

So you'd look at your request, see if it's an image that you want to upload (based on whatever factors you want), then do a

env.KV_BINDING.put("imagename", request.body)

env.KV_BINDING.put("imagename", request.body)

Isaac McFadyen•12/20/23, 12:03 PM

And then to read it back you can do a

const image = env.KV_BINDING.get("imagename", { type: "stream" });

const image = env.KV_BINDING.get("imagename", { type: "stream" });

(notice the stream part to get back as a stream)

Befus•12/20/23, 12:04 PM

Damn I'm going to try this, I will let you know!

Isaac McFadyen•12/20/23, 12:05 PM

Good luck!

IIsaac McFadyen Good luck!

Befus•12/20/23, 1:35 PM

export async function POST(req: NextRequest) {
  const formData = await req.formData();
  const file = formData.get("file") as File;
  console.log(file);
  console.log(file.type);
  console.log(file.name);
  // check if incoming data is a file and an image extension
  if (!file || !file.type.startsWith("image/")) {
    return new Response(JSON.stringify("error"), {
      headers: {
        "content-type": "application/json",
      },
    });
  }

  // if it is a file, upload the name of it to KV
  const { GALLERY } = process.env;
  const response = await GALLERY.put(file.name, file as unknown as ReadableStream<Uint8Array>);

  return new Response(JSON.stringify(response), {
    headers: {
      "content-type": "application/json",
    },
  });
}

export async function POST(req: NextRequest) {
  const formData = await req.formData();
  const file = formData.get("file") as File;
  console.log(file);
  console.log(file.type);
  console.log(file.name);
  // check if incoming data is a file and an image extension
  if (!file || !file.type.startsWith("image/")) {
    return new Response(JSON.stringify("error"), {
      headers: {
        "content-type": "application/json",
      },
    });
  }

  // if it is a file, upload the name of it to KV
  const { GALLERY } = process.env;
  const response = await GALLERY.put(file.name, file as unknown as ReadableStream<Uint8Array>);

  return new Response(JSON.stringify(response), {
    headers: {
      "content-type": "application/json",
    },
  });
}

I'm new to using KV, I seem to be getting the type error that GALLERY.put is not a function. But it does exist and I'm pretty sure it would work on production but I am testing locally now using npx cf-bindings-proxy

Isaac McFadyen•12/20/23, 1:42 PM

Hmm you're using Next?

Isaac McFadyen•12/20/23, 1:43 PM

You might want to ask in #next-on-pages, I haven't used Next at all and don't know how the bindings are supposed to work.

Isaac McFadyen•12/20/23, 1:43 PM

Also you shouldn't need to cast using Typescript - if you have the file as a File then it should be insertable by

file.readable

file.readable

I think?

Befus•12/20/23, 1:51 PM

Yea I'm using next, I will try to ask in the other channel.

There doesn't seem to be a file.readable but there is file.stream()?

IIsaac McFadyen And then to read it back you can do a `const image = env.KV_BINDING.get("imagena...

Befus•12/20/23, 2:51 PM

How would I get all images using .list?

BBefus Yea I'm using next, I will try to ask in the other channel. There doesn't seem ...

Isaac McFadyen•12/20/23, 2:58 PM

That's probably it, yeah.

BBefus How would I get all images using .list?

Isaac McFadyen•12/20/23, 2:59 PM

list

list

only shows you the keys. You need to run

get

get

on each key to get the image.

Befus•12/20/23, 3:06 PM

Happy me :)

Original message was deleted

Chaika•12/21/23, 12:36 AM

Web Assets, Cloudflare Pages uses KV for all uploaded assets. Lots of reads, not many writes

Original message was deleted

Chaika•12/21/23, 12:39 AM

I wouldn't think so. It depends on size and your budget I suppose, but R2 has a single location whereas KV has two central stores (one in NA, one in EU) and a built in caching layer.
For your own use, R2 may be cheaper though, at the cost of latency.

Chaika•12/21/23, 12:41 AM

There's a nice post on how KV works here:
https://developers.cloudflare.com/kv/learning/how-kv-works/
and overall storage options on CF
https://developers.cloudflare.com/workers/platform/storage-options/

Chaika•12/21/23, 12:42 AM

You can cache it via a custom domain, but each bucket still only lives in one region/location

Chaika•12/21/23, 12:44 AM

KV has some other cool things with Auto expiring keys via ExpirationTTL. You can have something be cached for x time and automatically be deleted to be repopulated. KV also as noted on that page can be used for configuration, like flags you change to activate features on your website

EdFletcher•12/21/23, 2:24 AM

you needn't even think of them as either/or. KV and R2 can work well together. for example say you have large binary blobs ("files") with associated structured metadata: you can store the metadata in KV with a pointer to the blob in R2. similarly with D1, you can have pointers to PKs (etc.) in a D1 DB from KV to help link K/V-style storage with relational storage.

Original message was deleted

EdFletcher•12/21/23, 2:51 AM

you bet, i'm glad it was helpful! and in fact that wasn't a contrived example either, i do those things in a set of workers i have open-sourced so you're welcome to take a gander at the code if it can be further helpful https://github.com/image-yinyang/

moody 💭•12/24/23, 2:39 PM

i could really use kv batch requests. i'm probably up to 300 requests per api call. and suspect making that many requests has a 1s+ latency overhead

Mmoody 💭i could really use kv batch requests. i'm probably up to 300 requests per api ca...

Matt Silverlock•12/24/23, 4:28 PM

What are you fetching that requires that many calls per eyeball/client request? Have you considered packing them into a larger value (and thus fewer keys) - ?

A batch read API won’t immediately solve performance issues as there isn’t a guarantee all of the keys in a batch would be hot (cached) or co-located in storage. Your batch would always be limited by the slowest read, identical to batch reads in most storage systems.

MMatt Silverlock What are you fetching that requires that many calls per eyeball/client request? ...

moody 💭•12/24/23, 4:29 PM

it would at least save cpu time

export async function POST(req: NextRequest) { const formData = await req.formData(); const file = formData.get("file") as File; console.log(file); console.log(file.type); console.log(file.name); // check if incoming data is a file and an image extension if (!file || !file.type.startsWith("image/")) { return new Response(JSON.stringify("error"), { headers: { "content-type": "application/json", }, }); } // if it is a file, upload the name of it to KV const { GALLERY } = process.env; const response = await GALLERY.put(file.name, file as unknown as ReadableStream<Uint8Array>); return new Response(JSON.stringify(response), { headers: { "content-type": "application/json", }, }); }

Code of your test? Are you using `kv.get(key, { cacheTtl: 60 })` or the cache API?

Similar Threads

Similar Threads

Similar Threads