Cloudflare Developers•3y ago

again, might just be my experience on a bad day with a bad region

SKOP•11/25/23, 9:20 AM

this matches what upstash found in their much more scientific benchmark https://upstash.com/blog/edgecaching-benchmark

HHermes Alves Hi there, Is there any nodejs SDK to consume data from Cloudflare KV? I want to ...

Hermes Alves•11/26/23, 5:32 AM

Do you query through the rest api endpoint, since it doesn't have an SDK?

VVikash Any chance to consider this feature to Filter/search KV by metadata tags - htt...

Vikash•11/27/23, 1:03 PM

@HardlyHolidayin’ Any chance?

VVikash @HardlyHolidayin’ Any chance?

Hello, I’m Allie!•11/27/23, 1:04 PM

I don't work for Cloudflare, so I can't comment. Also, pinging people won't get you an answer any faster

VVikash Any chance to consider this feature to Filter/search KV by metadata tags - htt...

DaniFoldi•11/27/23, 5:19 PM

Without knowledge of too many internal details, I highly doubt that multi-dimension indexing is something that KV would support in the foreseeable future, it's meant to be a key-value store. If you need more advanced queries, #durable-objects or #d1-beta might be more suitable

captain•11/27/23, 9:36 PM

was thre any KV hickups today?

Unsmart•11/27/23, 9:44 PM

everything seems fine for me

UUnsmart everything seems fine for me

captain•11/27/23, 9:55 PM

its fine for me now too, but around 8:30 to 9:30 when I was using something i had built it kept giving me old data,maybe it was a hickup somewhere else idk

captain•11/27/23, 9:56 PM

i have a 3rd party site with embedded google tag which then references some css data in a worker I created

captain•11/27/23, 9:56 PM

and that data is stored in KV

captain•11/27/23, 9:56 PM

but its working now...so moving on

DDaniFoldi Without knowledge of too many internal details, I highly doubt that multi-dimens...

Vikash•11/28/23, 8:10 AM

thanks for confirming.

Mmono_cron Hey! I have a few questions regarding the data returned from https://developers....

Matt Silverlock•11/28/23, 1:50 PM

Let me take a look.

sten•11/28/23, 5:35 PM

const key = "userId123;date;2023-01-01;postId;456";const key = "userId123;date;2023-01-01;postId;456";

I guess this is not a good key structure for posts?
I am thinking of when I am suppose to delete a post and I only have the postId, then I have to iterate the whole DB to find a key that matches this specific key by splitting each key and looking on index[2] .

Lilitu_Kali•12/3/23, 4:33 PM

Hey all, not sure if this is a "me" issue, so checking in - has anyone else been experiencing 500 errors related to kv storage today? A worker route was working then after manually adding maybe a dozen keys max via the dashboard I started getting 500s. Not sure if they were related or it was just a coincidence. The worker basically reads through the list of keys then for each key reads it value and outputs a json formatted list of the content.

Lilitu_Kali•12/3/23, 4:34 PM

The path where the Worker is triggered is https://kol.linzinha.is/players

Lilitu_Kali•12/3/23, 5:19 PM

so weird, I was seing a different error which doesn't seem to matter but I copied and pasted the exact same code in and resaved the worker, and the 500s went away

Aries•12/4/23, 3:37 AM

https://files.aries.fyi/2023/12/03/Code_5be27562613e7080.png

Aries•12/4/23, 3:38 AM

for (let index = 0; index < 10; index++) {
  await new Promise((resolve) => setTimeout(resolve, 1000));
  await getKey(versionKey);
}

for (let index = 0; index < 10; index++) {
  await new Promise((resolve) => setTimeout(resolve, 1000));
  await getKey(versionKey);
}

Aries•12/4/23, 3:38 AM

how does that make any sense?

Aries•12/4/23, 3:41 AM

ah, next.js decided to automatically cache it.

Aries•12/4/23, 3:41 AM

thanks next.js

Aries•12/4/23, 3:42 AM

wait hold on.

Aries•12/4/23, 3:42 AM

still an issue?
https://files.aries.fyi/2023/12/03/Code_9dd93c6dc0d365c3.png

kian•12/4/23, 4:10 AM

KV is eventually consistent, and caches per-colo. This looks like normal behaviour.

Varna•12/5/23, 3:44 PM

Does data get removed from KV if not accessed for a long time? i.e. 1 year.

VVarna Does data get removed from KV if not accessed for a long time? i.e. 1 year.

Hello, I’m Allie!•12/5/23, 3:45 PM

No. As long as you are willing to pay, it will stay

Chaika•12/5/23, 3:46 PM

(Unless you set an Expiration TTL on it, of course)

Chaika•12/5/23, 3:46 PM

If you wanted to, it could. Otherwise it's a full data store. After a year it wouldn't be cached anywhere but it would still be at the two central stores

w•12/5/23, 10:53 PM

Would Cloudflare KV be secure enough to store encryption keys? I'm trying to store many sets of data with each one being encrypted so thinking of using AWS KMS to store a master encryption key and storing multiple Data Encryption Keys in Cloudflare KV that are encrypted and decrypted using the AWS KMS key before use each time. Since I need to encrypt and decrypt frequently, KMS is a bit too price prohibitive for use entirely.

Santosh•12/6/23, 3:55 PM

Quick question: In case I update a KV value behind the scenes prior to its expiration (say using a DO), would the clients who are using the KV see it upon the next fetch or is the expectation that clients using the KV see the updated value within max of 60 seconds??

SSantosh Quick question: In case I update a KV value behind the scenes prior to its expir...

Hello, I’m Allie!•12/6/23, 3:56 PM

Clients will see it whenever the edge colo nearest to them pulls from the origin(or, if they are in the same colo as updater, then basically instantly)

Hello, I’m Allie!•12/6/23, 3:57 PM

So, if you have a cacheTtl of a week, then it could take up to a week to show an update

Santosh•12/6/23, 3:58 PM

Would I not be able to influence the read using this?

env.NAMESPACE.get(key, { cacheTtl: 3600 });

env.NAMESPACE.get(key, { cacheTtl: 3600 });

. I understand the write has a ttl as well and the read ttl has a min of 60secs

SSK there is no mistake, kv is in-fact not negligible latency like you might assume

Santosh•12/6/23, 4:53 PM

Interesting that you mentioned Cache API, what use cases would require to use KV vs Cache API(given thats local to the Data center that the caches it locally)? Just curious. Would use cases that require heavier processing for making the "object" available require KV? What kind of latencies have you seen with cache API?

SSantosh Interesting that you mentioned Cache API, what use cases would require to use KV...

Unsmart•12/6/23, 4:56 PM

The Cache API is just a cache and can be evicted at any time. KV basically has a very basic API similar to cache but its durable storage.

So Cache API if you just want a very basic cache that can be evicted at any time. KV if you want a more durable storage API with low latency (KV automatically uses the Cache API in front of it).

Santosh•12/6/23, 4:57 PM

A simple config read from R2, should we consider that more suitable for Cache API vs KV?

Unsmart•12/6/23, 4:58 PM

Yeah I would use Cache API and not KV for that fastnod

No need to have 2 durable storage layers.

Unsmart•12/6/23, 5:00 PM

KV may be a better fit for configs than R2 given KV will have lower latency but its also added cost so depends if you care that much about getting that slight edge in latency by having 2 origins vs 1 before the cache.

Unsmart•12/6/23, 5:02 PM

I will say though with KV only having 1 extra origin I do almost never see a reason to use KV instead of R2 + Cache API

If it had like 3 or 4 origins it might start to make more sense for truly latency critical things

UUnsmart I will say though with KV only having 1 extra origin I do almost never see a rea...

Santosh•12/6/23, 5:04 PM

Can you pl explain "1 extra origin" concept? Not sure I understand what you mean by 3/4 origins.

UUnsmart KV may be a better fit for configs than R2 given KV will have lower latency but ...

Santosh•12/6/23, 5:06 PM

It appears like KV latency is jumping all over even after Cache (theoretically KV within colo already uses cache ??) from the comments above ?Using plain vanilla Cache API -

Unsmart•12/6/23, 5:06 PM

KV stores data in 2 locations. One in the US, one in the EU.

R2 stores data in 1 location (location is selected at creation time).

So for reads that miss the cache KV can be slightly faster because it might have the data stored closer to the user. With only having 1 extra location the data is stored though the added benefit is pretty minimal IMO. But if KV stored data in like 3-4 regions across the world the latency boost for cache misses could be pretty noticeable.

Unsmart•12/6/23, 5:08 PM

Throwing the Cache API in front of KV will not really change the latency of KV given KV already caches data.

Santosh•12/6/23, 5:09 PM

What about using Cache alone without using KV api? WOuld that change the latency to access the data?

Unsmart•12/6/23, 5:10 PM

It would yes unless whatever source you are getting the data from already uses cloudflare cache in some way.

Unsmart•12/6/23, 5:11 PM

A cache will make things faster but if the data is already cached it cant make things faster than that.

again, might just be my experience on a bad day with a bad region

Similar Threads

Similar Threads

Similar Threads