I am trying to store images, should I store them directly into a KV or would it be better to store i

I am trying to store images, should I store them directly into a KV or would it be better to store in R2 and save the link to it in a KV

BBefus I am trying to store images, should I store them directly into a KV or would it ...

arch•12/27/23, 11:13 AM

Probably best to save to R2 and reference them in KV. Storing large blobs in any database is generally not advised.

Aarch Probably best to save to R2 and reference them in KV. Storing large blobs in any...

Isaac McFadyen•12/27/23, 2:47 PM

I would normally agree with you, but KV is designed to handle blobs up to 25MiB and will be a fair bit faster than R2 if you do

(and more expensive so depends on priorities I guess)

Isaac McFadyen•12/27/23, 2:47 PM

That's how Pages stores all assets

IIsaac McFadyen I would normally agree with you, but KV is designed to handle blobs up to 25MiB ...

arch•12/27/23, 3:28 PM

True! In my head I always avoid putting files in any database/kv store because then my code has to deal with retrieving, potentially decoding and then serving it up instead of offloading it to the/a static file CDN (although you might still want a worker for generating a signed S3 URL).

Aarch True! In my head I always avoid putting files in any database/kv store because t...

Isaac McFadyen•12/27/23, 3:48 PM

Makes sense, yeah

kian•12/30/23, 10:10 AM

It isn’t available to anyone currently, it was pulled.

Matt Silverlock•12/30/23, 12:10 PM

There won’t be any “API” changes - “KV 2.0” was only a name folks really used here. We’re working on the underlying storage improvements but I think “2.0” implies breaking changes / a migration / etc, which isn’t the case (and would be painful).

James•12/30/23, 2:26 PM

To expand on Matt’s message and to eliminate any confusion, “KV 2.0” wasn’t really a name just used here. The primary architect behind it tweeted about it, and it was on the beta signups at bare minimum.

I do agree that 2.0 is probably a bad name to describe underlying changes that will improve things but incur no breaking behaviour for end-users, but that’s not really what’s being discussed.

The initial changes were rolled back after KV experienced a large number of sequential incidents around the changes, and it was decided that was enough was enough. You can read the details of that here: https://blog.cloudflare.com/workers-kv-restoring-reliability

As for when to expect those changes again, or other underlying storage improvements, there’s no public info here, but I imagine any updates will be shared here and on the blog when ready. Hopefully we’ll hear something in the new year!

kian•12/30/23, 2:26 PM

A lot of confusion stems from the fact that https://developers.cloudflare.com/kv/learning/how-kv-works/ still talks about the KV 2.0 architecture.

Will•1/2/24, 8:15 AM

is it faster to retrive the KV data thru binding or thru rest api?

and how long usually it takes, to retrive data thru binding? I tried it myself, and I got 200+ ms

it's slower than getting the data from another endpoint hosted in aws EC2(70ms)
the size of the data is the same

Will•1/2/24, 9:30 AM

I tried with rest API, I got 2000ms

Will•1/2/24, 9:31 AM

so in order from the fastest to the slowest :

Api hosted in my ec2
KV thru binding === get the JSON from R2
KV thru rest API

Chaika•1/2/24, 4:50 PM

API goes back to core, which is either US West/EU or just US West, not sure but eitherway not optimal. Using a worker is much better as its the local dc.
Still though, KV's main benefit is cache. First request may be slow, espec if you're far away from one of the two central stores in us/eu, but future requests should be cached
https://developers.cloudflare.com/kv/learning/how-kv-works/

How KV works · Cloudflare Workers KV

KV is a global, low-latency, key-value data store. It stores data in a small number of centralized data centers, then caches that data in Cloudflare’s …

Will•1/3/24, 12:32 AM

thanks guys @celestial @Chaika @MattD | WorkersKV,Queues
I will try again with warm request and compare it to api hosted in my ec2.
hopefully, it can manage to be faster.

Will•1/3/24, 12:43 AM

let's say my list of key are :

business:u1-type:paying-sex:male
business:u1-type:paying-sex:female
business:u1-type:free-sex:male
business:u1-type:free-sex:female
business:u2-type:paying-sex:male
business:u2-type:paying-sex:female

is there anyway, let's say I just want to filter u1 and male?

Original message was deleted

Will•1/3/24, 1:14 AM

thank you

I initially plan to use postgre for this, but just want to try if worker KV can do this too.
D1, is no for a moment, cause it's still in beta, and I am the only developer

so no time if I had to refactor my code if somehow the realease version of D1 is dramatically different than the beta

Original message was deleted

Will•1/3/24, 1:45 AM

I think I found a way to do this with KV.
I just regex filter the key first before returning the response

kian•1/4/24, 9:54 PM

Seems fine for me, on 4.20231218.0

Emo•1/6/24, 12:35 AM

hello everyone! could someone help me understand when one would go for kv versus going for durable object? it's not clear for me

EEmo hello everyone! could someone help me understand when one would go for kv versus...

Cyb3r-Jak3•1/6/24, 12:59 AM

There is https://developers.cloudflare.com/workers/platform/storage-options/
KV is better for read heavy work loads and is eventually consistent, which means a write from one DC can take time before reaching a different one.
DO are strongly consistent and do transactional storage so they are better if you need to share the same data across the globe.
Also depending on your data then D1 is also an option

Choosing a data or storage product. · Cloudflare Workers docs

Storage and database options available on Cloudflare's developer platform.

Original message was deleted

zegevlier•1/6/24, 1:53 PM

You're in the #kv channel, I feel like #durable-objects would fit better here.
Durable objects are serverless. There are no non-serverless durable objects, at least not that Cloudflare offers.

Yongyi•1/7/24, 4:17 PM

hi what does it means for storage / account. Does that mean even if you are on cloudflare workers paid plan, your limit is 1GB?

kian•1/7/24, 4:35 PM

That's for the free plan.

kian•1/7/24, 4:35 PM

kian•1/7/24, 4:35 PM

If you scroll the table to the right, you'll see the Paid plan limits.

Yongyi•1/8/24, 12:53 AM

Thank you. I definitely missed that

Zef•1/10/24, 11:37 AM

It seems batch operations (such as batch get and put) are not available via bindings for CF workers, the docs recommend using the KV API (HTTP API), but performing a fetch to e.g. https://api.cloudflare.com/client/v4/...https://api.cloudflare.com/client/v4/... from a Cloudflare worker seems pretty slow (600ms+ latency). Is there a shortcut/faster way to call the cloudflare APIs from a worker?

ZZef It seems batch operations (such as batch get and put) are not available via bind...

DaniFoldi•1/10/24, 12:03 PM

unfortunately there isn't any better way, and I'm not sure if smart placement would be of much help either :/

Zef•1/10/24, 12:03 PM

I'll try to just run a bunch of parallel gets via bindings then and see how that performs

ZZef I'll try to just run a bunch of parallel gets via bindings then and see how that...

DaniFoldi•1/10/24, 12:09 PM

workers allow up to 6 parallel I/O operations, so I recommend using something like https://flareutils.pages.dev/promisequeue/ if you're planning to do many (say 100)

PromiseQueue | Flareutils

Small Utilities and little goodies that make developing with Cloudflare easier and faster.

DDaniFoldi workers allow up to 6 parallel I/O operations, so I recommend using something li...

Zef•1/10/24, 12:19 PM

Nice, thanks, I'll do that

Luciano•1/10/24, 9:02 PM

Question: What is the performance of listing KV values by a prefix in comparison to a database with an index such as D1?

Julian•1/11/24, 7:42 PM

Can I access KV from outside of Workers? I need to write API Keys from my backend

BefusOP•1/13/24, 9:02 AM

Can I save a PDF in a KV?

zegevlier•1/13/24, 11:20 AM

Sure! You can save any kind of data in KV, as long as it's under 25mb

Zzegevlier Sure! You can save any kind of data in KV, as long as it's under 25mb

BefusOP•1/13/24, 12:00 PM

Thank you!

ramesh prasad•1/13/24, 12:37 PM

Hello I am using nextjs, @cloudflare/next-on-pages. Since i heard static pages assets are deployed on KV by cloudflare which has hard limit of 25mb. What can i do if i hit the 25mb limit. Can i increase this limit?

Rramesh prasad Hello I am using nextjs, @cloudflare/next-on-pages. Since i heard static pages a...

Matt Silverlock•1/13/24, 2:14 PM

Per asset (per file). Are you running into this limit? https://developers.cloudflare.com/pages/platform/limits/#file-size

Limits · Cloudflare Pages docs

Below are limits observed by the Cloudflare Free plan. For more details on removing these limits, refer to the Cloudflare plans page.

Rramesh prasad Hello I am using nextjs, @cloudflare/next-on-pages. Since i heard static pages a...

Matt Silverlock•1/13/24, 2:14 PM

If you have single files that are larger than 25MB, you should store them in R2 and link to them from your app.

I am trying to store images, should I store them directly into a KV or would it be better to store i

Similar Threads

Similar Threads

Similar Threads