hello everyone! could someone help me understand when one would go for kv versus going for durable o

hello everyone! could someone help me understand when one would go for kv versus going for durable object? it's not clear for me

EEmo hello everyone! could someone help me understand when one would go for kv versus...

Cyb3r-Jak3•1/6/24, 12:59 AM

There is https://developers.cloudflare.com/workers/platform/storage-options/
KV is better for read heavy work loads and is eventually consistent, which means a write from one DC can take time before reaching a different one.
DO are strongly consistent and do transactional storage so they are better if you need to share the same data across the globe.
Also depending on your data then D1 is also an option

Choosing a data or storage product. · Cloudflare Workers docs

Storage and database options available on Cloudflare's developer platform.

Original message was deleted

zegevlier•1/6/24, 1:53 PM

You're in the #kv channel, I feel like #durable-objects would fit better here.
Durable objects are serverless. There are no non-serverless durable objects, at least not that Cloudflare offers.

Yongyi•1/7/24, 4:17 PM

hi what does it means for storage / account. Does that mean even if you are on cloudflare workers paid plan, your limit is 1GB?

kian•1/7/24, 4:35 PM

That's for the free plan.

kian•1/7/24, 4:35 PM

kian•1/7/24, 4:35 PM

If you scroll the table to the right, you'll see the Paid plan limits.

Yongyi•1/8/24, 12:53 AM

Thank you. I definitely missed that

Zef•1/10/24, 11:37 AM

It seems batch operations (such as batch get and put) are not available via bindings for CF workers, the docs recommend using the KV API (HTTP API), but performing a fetch to e.g. https://api.cloudflare.com/client/v4/...https://api.cloudflare.com/client/v4/... from a Cloudflare worker seems pretty slow (600ms+ latency). Is there a shortcut/faster way to call the cloudflare APIs from a worker?

ZZef It seems batch operations (such as batch get and put) are not available via bind...

DaniFoldi•1/10/24, 12:03 PM

unfortunately there isn't any better way, and I'm not sure if smart placement would be of much help either :/

Zef•1/10/24, 12:03 PM

I'll try to just run a bunch of parallel gets via bindings then and see how that performs

ZZef I'll try to just run a bunch of parallel gets via bindings then and see how that...

DaniFoldi•1/10/24, 12:09 PM

workers allow up to 6 parallel I/O operations, so I recommend using something like https://flareutils.pages.dev/promisequeue/ if you're planning to do many (say 100)

PromiseQueue | Flareutils

Small Utilities and little goodies that make developing with Cloudflare easier and faster.

DDaniFoldi workers allow up to 6 parallel I/O operations, so I recommend using something li...

Zef•1/10/24, 12:19 PM

Nice, thanks, I'll do that

Luciano•1/10/24, 9:02 PM

Question: What is the performance of listing KV values by a prefix in comparison to a database with an index such as D1?

Julian•1/11/24, 7:42 PM

Can I access KV from outside of Workers? I need to write API Keys from my backend

Befus•1/13/24, 9:02 AM

Can I save a PDF in a KV?

zegevlier•1/13/24, 11:20 AM

Sure! You can save any kind of data in KV, as long as it's under 25mb

Zzegevlier Sure! You can save any kind of data in KV, as long as it's under 25mb

Befus•1/13/24, 12:00 PM

Thank you!

ramesh prasad•1/13/24, 12:37 PM

Hello I am using nextjs, @cloudflare/next-on-pages. Since i heard static pages assets are deployed on KV by cloudflare which has hard limit of 25mb. What can i do if i hit the 25mb limit. Can i increase this limit?

Rramesh prasad Hello I am using nextjs, @cloudflare/next-on-pages. Since i heard static pages a...

Matt Silverlock•1/13/24, 2:14 PM

Per asset (per file). Are you running into this limit? https://developers.cloudflare.com/pages/platform/limits/#file-size

Limits · Cloudflare Pages docs

Below are limits observed by the Cloudflare Free plan. For more details on removing these limits, refer to the Cloudflare plans page.

Rramesh prasad Hello I am using nextjs, @cloudflare/next-on-pages. Since i heard static pages a...

Matt Silverlock•1/13/24, 2:14 PM

If you have single files that are larger than 25MB, you should store them in R2 and link to them from your app.

atgctg•1/13/24, 6:14 PM

Looking at key-value benchmarks right now and found it surprising to see Cloudflare's KV latencies being so high. Is there something that these benchmarks overlook?
https://upstash.com/blog/edgecaching-benchmark
https://deno.com/blog/comparing-deno-kv

MMatt Silverlock If you have _single_ files that are larger than 25MB, you should store them in R...

ramesh prasad•1/13/24, 7:48 PM

thanks brother

ramesh prasad•1/13/24, 7:50 PM

And yeah I have an usecase to cover. After every 1000 worker invocation I need to hit an api to external system. how can i do that? Any idea on this? Workers will as usual get data from user, the data will be stored on lets say d1. And after 1000 data I get from user, I need to push that data to external system. Any idea here?

X•1/13/24, 7:52 PM

please manage every excitement. help me 1st.

Aatgctg Looking at key-value benchmarks right now and found it surprising to see Cloudfl...

Unsmart•1/13/24, 8:10 PM

I would say KV is by no means the fastest database in the world but I would take these comparisons with a grain of salt as both those companies have a vested interest in making their db looking better.

I would test KV latency for yourself the latency will depend how close you are to the origin and if the value is cached or not. For me a cache miss on KV takes roughly ~300ms and a cache hit takes roughly ~40ms.

UUnsmart I would say KV is by no means the fastest database in the world but I would take...

atgctg•1/13/24, 10:04 PM

thanks for the reply!

Unsmart•1/15/24, 5:48 PM

dear lord that code is very cursed KEKW

Unsmart•1/15/24, 5:55 PM

I'm very confused how they got a p99 of 700ms for doing a single query and get on DynamoDB also....

My work has serveral routes at p99 of 500ms or less that do at least 2-3x the amount of queries lol

Unsmart•1/15/24, 5:57 PM

DynamoDB is an interesting DB though where more read load produces lower latency lol so 500ms when barely any reads and 200ms p99 at peak

Unsmart•1/15/24, 6:04 PM

Also interesting to note that the Deno comparison is doing a list followed by a get for every service but then their own only does a list so very unfair comparison

Unsmart•1/15/24, 9:39 PM

I dont think theres much reason to benchmark them just pick the one you want to use. If you already use cloudflare and the feature set of KV is good enough for your use case I dont see any reason to use something outside of cloudflare. Picking based off of benchmarks is only really relevant when you are cloudflare scale imo

Unsmart•1/15/24, 9:44 PM

The p99 latency I measured was for DynamoDB not KV and I just looked at the AWS Lambda latency provided by cloudwatch.

Unsmart•1/15/24, 9:46 PM

The somewhat equivalent on cloudflare would be going to your worker and looking at execution duration but I'd imagine you probably have a lot of routes on a single worker. And the graph also shows in GB-sec instead of milliseconds.

Julian•1/17/24, 12:08 AM

One of my collegues made an SDK for Cloudflare KV API in Java! https://github.com/segu23/cloudflare-sdk-kv

GitHub

GitHub - segu23/cloudflare-sdk-kv

Contribute to segu23/cloudflare-sdk-kv development by creating an account on GitHub.

mackenzie•1/18/24, 8:31 AM

#real noob# question. I have a worker/DO that writes some status to KV. I'd like to write a small we app (and later mobile app) to visualize it....are there any examples for how to go about doing that, using cloud flare or other (using rust if possible)

mackenzie•1/18/24, 8:32 AM

Possible with pages?

mackenzie•1/18/24, 8:33 AM

(later I might want to subscribe to a Pub/Sub topic, if I get Beta access)

kp1197•1/20/24, 11:54 PM

Hi, I am setting up local development, and I have a worker which has a KV binding. Is it possible for me to develop locally, while connecting to and accessing the values for KV namespace as stored in my cloudflare account? i.e.; npx wrangler login then npx wranger dev --remote

kp1197•1/20/24, 11:54 PM

My overall goal is to follow principle of least access and not have any copies of the values in my KV namespace stored in any way in my local, even in a command buffer

kian•1/20/24, 11:57 PM

Yes, that's what remote development is

kp1197•1/21/24, 12:07 AM

Hmmm... any idea what this error message is about?

kp1197•1/21/24, 12:07 AM

notes: [
{
text: 'Uncaught ReferenceError: CONFIG is not defined\n' +
' at index.js:28:26\n' +
' [code: 10021]'
}
],
location: undefined,
kind: 'error',
code: 10021
}

kp1197•1/21/24, 12:08 AM

Preceeded by: 'A request to the Cloudflare API' ... 'failed'

kp1197•1/21/24, 12:09 AM

My toml looks like this: kv_namespaces = [
{ binding = "CONFIG", id = "<I_AINT_TELLIN>" }
]

Cyb3r-Jak3•1/21/24, 12:24 AM

What does your code look like?

kp1197•1/21/24, 12:36 AM

It has a line which references CONFIG in the top level scope

kp1197•1/21/24, 12:36 AM

I.e.; global

hello everyone! could someone help me understand when one would go for kv versus going for durable o

Similar Threads

Similar Threads

Similar Threads