What are you fetching that requires that many calls per eyeball/client request? Have you considered

What are you fetching that requires that many calls per eyeball/client request? Have you considered packing them into a larger value (and thus fewer keys) - ?

A batch read API won’t immediately solve performance issues as there isn’t a guarantee all of the keys in a batch would be hot (cached) or co-located in storage. Your batch would always be limited by the slowest read, identical to batch reads in most storage systems.

MMatt Silverlock What are you fetching that requires that many calls per eyeball/client request? ...

moody 💭•12/24/23, 4:29 PM

it would at least save cpu time

Mmoody 💭it would at least save cpu time

Matt SilverlockOP•12/24/23, 4:29 PM

Network calls don’t count towards CPU time on the Standard usage model.

moody 💭•12/24/23, 4:29 PM

constructing and firing 300 calls has some non-negligible overhead

moody 💭•12/24/23, 4:29 PM

i'd imagine

moody 💭•12/24/23, 4:30 PM

vs creating an array of 300 keys and firing one call

moody 💭•12/24/23, 4:31 PM

and i'm sure there is overhead in the backend for each call as well

Mmoody 💭constructing and firing 300 calls has some non-negligible overhead

Matt SilverlockOP•12/24/23, 4:31 PM

That is minimal vs the performance / latency issue you called out, which is why I’m suggesting you re-think your key design. Batch reads are not a panacea.

moody 💭•12/24/23, 4:34 PM

the use case is, i have a request that requires several megabytes of data from a postgresql database. i cache as much as i can in the worker memory, but for a fresh worker, i have to get all that data out of postgres.

i have 1 query that tells me all the keys that have new data (something like last updated timestamp), and then another query that gets the new data for all the keys that have seen updates.

my postgres database started getting bandwidth throttled (blue line) because of the workers fetching so much data from it. so i started storing the data for each key + last updated in kv and fetch from kv first

moody 💭•12/24/23, 4:36 PM

it is perfect for kv or redis because it's write-once (key+last-updated-ts w/ ttl)->data.

MMatt Silverlock That is minimal vs the performance / latency issue you called out, which is why ...

moody 💭•12/24/23, 4:39 PM

i think a batch read request that only reads from the most local region cache and then asynchronously requests copying to that datacenter would be ok

moody 💭•12/24/23, 5:52 PM

the other part of the latency is doing a put of each of the keys that wasnt found in the store. batch read and write are pretty important i think

MMatt Silverlock That is minimal vs the performance / latency issue you called out, which is why ...

moody 💭•12/26/23, 1:04 PM

curious if you have any ideas of how i can improve my architecture

moody 💭•12/26/23, 1:05 PM

the batch endpoint will definitely speed things up and separately i think it's also not great that multiple requests will simultaneously update the cache

MMatt Silverlock That is minimal vs the performance / latency issue you called out, which is why ...

moody 💭•12/26/23, 4:44 PM

i think i might try using the request response cache for the data since i don't need it replicated across regions

jacob•12/26/23, 10:54 PM

Is it normal for put() to take several minutes? I executed it with seemingly no errors or issues maybe 5 minutes ago or so and they still don't show up on the cloudflare dashboard

kian•12/26/23, 10:58 PM

How did you call put()? In a published Worker?

Kkian How did you call put()? In a published Worker?

jacob•12/26/23, 10:58 PM

The worker is published though this code is running locally

kian•12/26/23, 10:59 PM

That’s expected then, local has a local emulation of storage.

kian•12/26/23, 10:59 PM

If you want to run it on the edge and talk to real resources, add --remote

jacob•12/26/23, 10:59 PM

Oh I see, that actually sounds quite useful. Thanks!

Befus•12/27/23, 10:51 AM

I am trying to store images, should I store them directly into a KV or would it be better to store in R2 and save the link to it in a KV

BBefus I am trying to store images, should I store them directly into a KV or would it ...

arch•12/27/23, 11:13 AM

Probably best to save to R2 and reference them in KV. Storing large blobs in any database is generally not advised.

Aarch Probably best to save to R2 and reference them in KV. Storing large blobs in any...

Isaac McFadyen•12/27/23, 2:47 PM

I would normally agree with you, but KV is designed to handle blobs up to 25MiB and will be a fair bit faster than R2 if you do

(and more expensive so depends on priorities I guess)

Isaac McFadyen•12/27/23, 2:47 PM

That's how Pages stores all assets

IIsaac McFadyen I would normally agree with you, but KV is designed to handle blobs up to 25MiB ...

arch•12/27/23, 3:28 PM

True! In my head I always avoid putting files in any database/kv store because then my code has to deal with retrieving, potentially decoding and then serving it up instead of offloading it to the/a static file CDN (although you might still want a worker for generating a signed S3 URL).

Aarch True! In my head I always avoid putting files in any database/kv store because t...

Isaac McFadyen•12/27/23, 3:48 PM

Makes sense, yeah

kian•12/30/23, 10:10 AM

It isn’t available to anyone currently, it was pulled.

Matt SilverlockOP•12/30/23, 12:10 PM

There won’t be any “API” changes - “KV 2.0” was only a name folks really used here. We’re working on the underlying storage improvements but I think “2.0” implies breaking changes / a migration / etc, which isn’t the case (and would be painful).

James•12/30/23, 2:26 PM

To expand on Matt’s message and to eliminate any confusion, “KV 2.0” wasn’t really a name just used here. The primary architect behind it tweeted about it, and it was on the beta signups at bare minimum.

I do agree that 2.0 is probably a bad name to describe underlying changes that will improve things but incur no breaking behaviour for end-users, but that’s not really what’s being discussed.

The initial changes were rolled back after KV experienced a large number of sequential incidents around the changes, and it was decided that was enough was enough. You can read the details of that here: https://blog.cloudflare.com/workers-kv-restoring-reliability

As for when to expect those changes again, or other underlying storage improvements, there’s no public info here, but I imagine any updates will be shared here and on the blog when ready. Hopefully we’ll hear something in the new year!

kian•12/30/23, 2:26 PM

A lot of confusion stems from the fact that https://developers.cloudflare.com/kv/learning/how-kv-works/ still talks about the KV 2.0 architecture.

Will•1/2/24, 8:15 AM

is it faster to retrive the KV data thru binding or thru rest api?

and how long usually it takes, to retrive data thru binding? I tried it myself, and I got 200+ ms

it's slower than getting the data from another endpoint hosted in aws EC2(70ms)
the size of the data is the same

Will•1/2/24, 9:30 AM

I tried with rest API, I got 2000ms

Will•1/2/24, 9:31 AM

so in order from the fastest to the slowest :

Api hosted in my ec2
KV thru binding === get the JSON from R2
KV thru rest API

Chaika•1/2/24, 4:50 PM

API goes back to core, which is either US West/EU or just US West, not sure but eitherway not optimal. Using a worker is much better as its the local dc.
Still though, KV's main benefit is cache. First request may be slow, espec if you're far away from one of the two central stores in us/eu, but future requests should be cached
https://developers.cloudflare.com/kv/learning/how-kv-works/

How KV works · Cloudflare Workers KV

KV is a global, low-latency, key-value data store. It stores data in a small number of centralized data centers, then caches that data in Cloudflare’s …

Will•1/3/24, 12:32 AM

thanks guys @celestial @Chaika @MattD | WorkersKV,Queues
I will try again with warm request and compare it to api hosted in my ec2.
hopefully, it can manage to be faster.

Will•1/3/24, 12:43 AM

let's say my list of key are :

business:u1-type:paying-sex:male
business:u1-type:paying-sex:female
business:u1-type:free-sex:male
business:u1-type:free-sex:female
business:u2-type:paying-sex:male
business:u2-type:paying-sex:female

is there anyway, let's say I just want to filter u1 and male?

Original message was deleted

Will•1/3/24, 1:14 AM

thank you

I initially plan to use postgre for this, but just want to try if worker KV can do this too.
D1, is no for a moment, cause it's still in beta, and I am the only developer

so no time if I had to refactor my code if somehow the realease version of D1 is dramatically different than the beta

Original message was deleted

Will•1/3/24, 1:45 AM

I think I found a way to do this with KV.
I just regex filter the key first before returning the response

kian•1/4/24, 9:54 PM

Seems fine for me, on 4.20231218.0

What are you fetching that requires that many calls per eyeball/client request? Have you considered

Similar Threads