Cloudflare Developers•3y ago

<https://test.blueradar.net/>

https://test.blueradar.net/

SKOP•12/7/23, 2:57 AM

there are 2 routes, one fetches the data from KV, and the other tries the cache first and fallbacks to kv

SKOP•12/7/23, 2:57 AM

I ran a simple python script that hit each route 100 times and recorded the latencies to get some summary stats

SKOP•12/7/23, 2:57 AM

UUnsmart A cache will make things faster but if the data is already cached it cant make t...

SKOP•12/7/23, 2:57 AM

it turns out that it does actually make it faster

SKOP•12/7/23, 2:58 AM

sure not by a whole lot but but it does

SKOP•12/7/23, 2:58 AM

the last time I tested this KV was performing much worse (around 50-100ms) so I'm super happy that it's working heaps better now

Unsmart•12/7/23, 3:02 AM

I mean if you think it helps you can do it sure no one going to stop you

SKOP•12/7/23, 3:03 AM

but yeah, when I refactor this I'll switch to r2 + cache api instead of layering it on kv, that was just placeholder to get it working

SSK <https://test.blueradar.net/>

Vikash•12/9/23, 3:46 PM

Code of your test?

Are you using kv.get(key, { cacheTtl: 60 })kv.get(key, { cacheTtl: 60 }) or the cache API?

VVikash Code of your test? Are you using `kv.get(key, { cacheTtl: 60 })` or the cache ...

SKOP•12/10/23, 1:24 AM

https://gist.github.com/Repo-SK/76775be58020c964c17eae45be8b3104

SKOP•12/10/23, 1:25 AM

/kv uses kv.get with a cacheTtl value of an hour (3600) and /kv-cache-api uses the cache api

SKOP•12/10/23, 1:25 AM

for the benchmark simply send requests to both endpoints and extract the latencies from the html returned (I used a python script to send 100 requests to each)

kian•12/10/23, 6:18 PM

As in the read takes that long to fulfill, or that long until you see the new value?

zehawk•12/16/23, 3:09 PM

Where does one see how much storage is being used by KV? I can never get an accurate value. I got billed for 1.69GB while the KV page shows 67KB. So what gives?

Mmatt_d @izdi No, this behavior is an inescapable aspect of WorkersKV.

Advany•12/17/23, 1:00 PM

is it already as fast as 5 seconds? I though it would take up to 60 seconds to get consistent with the write

Zzehawk Where does one see how much storage is being used by KV? I can never get an accu...

Matt Silverlock•12/17/23, 1:03 PM

Please open a Support ticket here.

AAdvany is it already as fast as 5 seconds? I though it would take up to 60 seconds to g...

Matt Silverlock•12/17/23, 2:32 PM

60s for values to refresh (eventual consistency). If you read a key, you will implicitly cache the empty/null value. You can read new keys faster than 60s on first read.

Advany•12/17/23, 2:33 PM

Awesome! Just wanted to verify. Hopefully one day we will see faster refreshes but good for now!

AAdvany is it already as fast as 5 seconds? I though it would take up to 60 seconds to g...

Hello, I’m Allie!•12/17/23, 2:39 PM

This also assumes you haven’t set a higher CacheTTL

HHello, I’m Allie!This also assumes you haven’t set a higher CacheTTL

Advany•12/17/23, 3:07 PM

Yeah a while back there was talk about a new version of KV that would refresh faster even with a higher CacheTTL. There where talks of 15 seconds

AAdvany Yeah a while back there was talk about a new version of KV that would refresh fa...

Hello, I’m Allie!•12/17/23, 3:28 PM

Yeah, that was v2, but it got rolled back a while ago

bonald•12/18/23, 2:55 AM

16ms latency for kv , is this chat real? was looking the test redis vs kv . i guess theres no test to do. is this real?

kian•12/18/23, 2:56 AM

For a cached read? Sounds right to me.

Bbonald 16ms latency for kv , is this chat real? was looking the test redis vs kv . i gu...

Advany•12/18/23, 8:41 AM

a kv is cached locally using Cache. Thats why second reads on the same location are faster

HHello, I’m Allie!Yeah, that was v2, but it got rolled back a while ago

Advany•12/18/23, 8:41 AM

aaaah good to know they are still working on that!

Befus•12/20/23, 11:50 AM

How would I store an image inside KV? Not using cloudflare images

BBefus How would I store an image inside KV? Not using cloudflare images

Isaac McFadyen•12/20/23, 12:01 PM

You'd probably encode it as an ArrayBuffer. Keep in mind that will buffer the image into the Worker's RAM, but I don't think there's any way to stream directly to/from KV.

Isaac McFadyen•12/20/23, 12:01 PM

Oh actually, there is!

Isaac McFadyen•12/20/23, 12:01 PM

You can directly write a ReadableStream: https://developers.cloudflare.com/kv/api/write-key-value-pairs/

Isaac McFadyen•12/20/23, 12:02 PM

So you'd look at your request, see if it's an image that you want to upload (based on whatever factors you want), then do a

env.KV_BINDING.put("imagename", request.body)

env.KV_BINDING.put("imagename", request.body)

Isaac McFadyen•12/20/23, 12:03 PM

And then to read it back you can do a

const image = env.KV_BINDING.get("imagename", { type: "stream" });

const image = env.KV_BINDING.get("imagename", { type: "stream" });

(notice the stream part to get back as a stream)

Befus•12/20/23, 12:04 PM

Damn I'm going to try this, I will let you know!

Isaac McFadyen•12/20/23, 12:05 PM

Good luck!

IIsaac McFadyen Good luck!

Befus•12/20/23, 1:35 PM

export async function POST(req: NextRequest) {
  const formData = await req.formData();
  const file = formData.get("file") as File;
  console.log(file);
  console.log(file.type);
  console.log(file.name);
  // check if incoming data is a file and an image extension
  if (!file || !file.type.startsWith("image/")) {
    return new Response(JSON.stringify("error"), {
      headers: {
        "content-type": "application/json",
      },
    });
  }

  // if it is a file, upload the name of it to KV
  const { GALLERY } = process.env;
  const response = await GALLERY.put(file.name, file as unknown as ReadableStream<Uint8Array>);

  return new Response(JSON.stringify(response), {
    headers: {
      "content-type": "application/json",
    },
  });
}

export async function POST(req: NextRequest) {
  const formData = await req.formData();
  const file = formData.get("file") as File;
  console.log(file);
  console.log(file.type);
  console.log(file.name);
  // check if incoming data is a file and an image extension
  if (!file || !file.type.startsWith("image/")) {
    return new Response(JSON.stringify("error"), {
      headers: {
        "content-type": "application/json",
      },
    });
  }

  // if it is a file, upload the name of it to KV
  const { GALLERY } = process.env;
  const response = await GALLERY.put(file.name, file as unknown as ReadableStream<Uint8Array>);

  return new Response(JSON.stringify(response), {
    headers: {
      "content-type": "application/json",
    },
  });
}

I'm new to using KV, I seem to be getting the type error that GALLERY.put is not a function. But it does exist and I'm pretty sure it would work on production but I am testing locally now using npx cf-bindings-proxy

Isaac McFadyen•12/20/23, 1:42 PM

Hmm you're using Next?

Isaac McFadyen•12/20/23, 1:43 PM

You might want to ask in #next-on-pages, I haven't used Next at all and don't know how the bindings are supposed to work.

Isaac McFadyen•12/20/23, 1:43 PM

Also you shouldn't need to cast using Typescript - if you have the file as a File then it should be insertable by

file.readable

file.readable

I think?

Befus•12/20/23, 1:51 PM

Yea I'm using next, I will try to ask in the other channel.

There doesn't seem to be a file.readable but there is file.stream()?

IIsaac McFadyen And then to read it back you can do a `const image = env.KV_BINDING.get("imagena...

Befus•12/20/23, 2:51 PM

How would I get all images using .list?

BBefus Yea I'm using next, I will try to ask in the other channel. There doesn't seem ...

Isaac McFadyen•12/20/23, 2:58 PM

That's probably it, yeah.

BBefus How would I get all images using .list?

Isaac McFadyen•12/20/23, 2:59 PM

list

list

only shows you the keys. You need to run

get

get

on each key to get the image.

Befus•12/20/23, 3:06 PM

Happy me :)

Original message was deleted

Chaika•12/21/23, 12:36 AM

Web Assets, Cloudflare Pages uses KV for all uploaded assets. Lots of reads, not many writes

export async function POST(req: NextRequest) { const formData = await req.formData(); const file = formData.get("file") as File; console.log(file); console.log(file.type); console.log(file.name); // check if incoming data is a file and an image extension if (!file || !file.type.startsWith("image/")) { return new Response(JSON.stringify("error"), { headers: { "content-type": "application/json", }, }); } // if it is a file, upload the name of it to KV const { GALLERY } = process.env; const response = await GALLERY.put(file.name, file as unknown as ReadableStream<Uint8Array>); return new Response(JSON.stringify(response), { headers: { "content-type": "application/json", }, }); }

<https://test.blueradar.net/>

Similar Threads

Similar Threads

Similar Threads