hi, who can shed some light on the mysterious 1000 operations per invocation limit in KV? I have a w

Kkika hi, who can shed some light on the mysterious 1000 operations per invocation lim...

K

kikaOP•3/16/23, 9:39 PM

My impression is that every call to the queue handler is one invocation, so I shouldn't hit any limits

Original message was deleted

K

kikaOP•3/16/23, 9:45 PM

well, I read about this limit in the KV documentation https://developers.cloudflare.com/workers/platform/limits/#kv-limits

Limits · Cloudflare Workers docs

You can request adjustments to limits that conflict with your project goals by contacting Cloudflare. To increase a limit, complete the Limit Increase …

Original message was deleted

K

kikaOP•3/16/23, 9:48 PM

tbh, I don't see how this helps. does this somehow mean that all invocations of the queue handler is one invocation?

Original message was deleted

K

kikaOP•3/16/23, 9:49 PM

so every time queue handler is called it's one invocation. for every invocation i have a budget of 1000 calls to KV. but somehow this is not true. I write 10 times 100 items and boom - Exceeded Resources

Original message was deleted

K

kikaOP•3/16/23, 9:50 PM

10 times calling queue event handler is one invocation?

Original message was deleted

K

kikaOP•3/16/23, 9:52 PM

which means I can initiate 1000 calls to KV. But I can only make 1000 total across 10 invocations.

K

kikaOP•3/16/23, 9:54 PM

#[event(queue)]
pub async fn queue(batch: MessageBatch<QData>, env: Env, ctx: Context) -> Result<()> {
    try_setup_logger();
    utils::set_panic_hook();
    ctx.wait_until(async move {
        let kv = KvStorage::new(env.kv("data").unwrap(), env.queue("dataq").unwrap());
        info!("Batch: got {} messages", batch.messages().unwrap().len());
        for msg in batch.messages().unwrap() {
            let data: QData = msg.body;
            kv.write_kv(&data.key, &data.value).await;
        }
    });
    Ok(())
}

#[event(queue)]
pub async fn queue(batch: MessageBatch<QData>, env: Env, ctx: Context) -> Result<()> {
    try_setup_logger();
    utils::set_panic_hook();
    ctx.wait_until(async move {
        let kv = KvStorage::new(env.kv("data").unwrap(), env.queue("dataq").unwrap());
        info!("Batch: got {} messages", batch.messages().unwrap().len());
        for msg in batch.messages().unwrap() {
            let data: QData = msg.body;
            kv.write_kv(&data.key, &data.value).await;
        }
    });
    Ok(())
}

#[event(queue)]
pub async fn queue(batch: MessageBatch<QData>, env: Env, ctx: Context) -> Result<()> {
    try_setup_logger();
    utils::set_panic_hook();
    ctx.wait_until(async move {
        let kv = KvStorage::new(env.kv("data").unwrap(), env.queue("dataq").unwrap());
        info!("Batch: got {} messages", batch.messages().unwrap().len());
        for msg in batch.messages().unwrap() {
            let data: QData = msg.body;
            kv.write_kv(&data.key, &data.value).await;
        }
    });
    Ok(())
}

#[event(queue)]
pub async fn queue(batch: MessageBatch<QData>, env: Env, ctx: Context) -> Result<()> {
    try_setup_logger();
    utils::set_panic_hook();
    ctx.wait_until(async move {
        let kv = KvStorage::new(env.kv("data").unwrap(), env.queue("dataq").unwrap());
        info!("Batch: got {} messages", batch.messages().unwrap().len());
        for msg in batch.messages().unwrap() {
            let data: QData = msg.body;
            kv.write_kv(&data.key, &data.value).await;
        }
    });
    Ok(())
}

K

kikaOP•3/16/23, 9:54 PM

KvStorage is my thin wrapper around KV and Queue

Original message was deleted

K

kikaOP•3/16/23, 9:57 PM

okay, thanks, makes sense

K

kikaOP•3/16/23, 9:58 PM

but logging shows that everything works as it's supposed to work - I'm getting 10 log lines of "Batch: got 100 messages"

Original message was deleted

H

Hello, I’m Allie!•3/18/23, 10:29 AM

Depending on where you are in the world, maybe, though it is a bit on the high end

Original message was deleted

H

Hello, I’m Allie!•3/18/23, 10:44 AM

A VPN shouldn’t make a difference. Odd…

Original message was deleted

E

Erisa•3/18/23, 1:48 PM

Uncached reads come from the central data stores in Europe and US, so if you're far from those then that might be expected in a worst case scenario https://developers.cloudflare.com/workers/learning/how-kv-works/
The reason you get faster hits after is because it ends up cached and serves a warm read from your local datacenter instead. The cache by default lasts 60 seconds but can be increased at read time with the cacheTtl option

B

beelzabub•3/20/23, 5:13 AM

This will get better with KV2 btw where you a) will still get a warm read after 1 minute instead of a cold read b) you can set a really long cacheTTL to get a hot read without altering that writes are invalidated within 1 minute. If you want, feel free to sign up for the Beta.

Bbeelzabub This will get better with KV2 btw where you a) will still get a warm read after ...

K

kikaOP•3/22/23, 5:38 PM

which one is KV2? https://developers.cloudflare.com/workers/platform/betas/

Betas · Cloudflare Workers docs

These are the current alphas and betas relevant to the Cloudflare Workers platform.

U

Unsmart•3/22/23, 5:40 PM

none of those

K

kikaOP•3/22/23, 8:41 PM

then how to "sign up for the Beta"?

K

kian•3/22/23, 8:51 PM

https://forms.gle/d1yrVjxpQNFSYYYf6

Google Docs

KV 2.0 Beta Sign Up

KV 2.0 will be rolling out over the next month or so. Submit interest to get into an early beta before we turn this on globally.

B

beelzabub•3/23/23, 6:21 PM

The sign up is closed and we're rolled out to everyone that had been on that list. I'm not doing any more piecemeal. I'm going to try to get to 10% of all KV objects today (100% account enrollment for the long tail of accounts).

K

kikaOP•3/23/23, 10:00 PM

the data value in the KV has an expiration TTL, but what about the key and metadata? if I

list()

list()

list()

list() the keys when the values had expired already, will the list operation also incur a penalty because of the roundtrip to the "home" datacenter?

B

beelzabub•3/23/23, 11:13 PM

The TTL applies to the entire object. Lists are cached so as long as the input parameters are the same, the same result will be returned each time.

Bbeelzabub The sign up is closed and we're rolled out to everyone that had been on that lis...

B

beelzabub•3/23/23, 11:19 PM

We're now at 100% rollout for ~25% of all KV traffic. We are seeing some small performance regression at the tail latency. We're going to work on seeing what we can do to bring that back down in the coming weeks. Hope everyone enjoys the better consistency.

Bbeelzabub We're now at 100% rollout for ~25% of all KV traffic. We are seeing some small p...

B

beelzabub•3/23/23, 11:24 PM

PSA: If you are using KV2, feel free to set the

cacheTTL

cacheTTL

cacheTTL

cacheTTL to longer than 1 minute if it's not a security-sensitive application. That will give you hot reads for longer without impacting how long it takes to notice a write. How can you tell if you're using KV2? I don't really know but basically you can just ask me. The answer is probably yes unless you're an ENT customer with more than ~500 RPS.

Bbeelzabub The TTL applies to the entire object. Lists are cached so as long as the input p...

K

kikaOP•3/24/23, 12:11 AM

thanks! if I update the metadata on a key, should I use cache API to invalidate it, or it will happen somehow automagically?

Kkika thanks! if I update the metadata on a key, should I use cache API to invalidate ...

B

beelzabub•3/24/23, 12:50 AM

I don’t understand the question. What cache API?

Bbeelzabub PSA: If you are using KV2, feel free to set the `cacheTTL` to longer than 1 minu...

B

beelzabub•3/24/23, 12:50 AM

We’ve turned it back off outside of the beta list. We were seeing some problems that need some investigation

M

micromashor•3/24/23, 1:03 AM

does anyone know the differences in latency figures for R2 vs KV with hot reads? I'd assume KV would be faster

Mmicromashor does anyone know the differences in latency figures for R2 vs KV with hot reads?...

C

Cyb3r-Jak3•3/24/23, 1:06 AM

KV is faster

CCyb3r-Jak3 KV is faster

M

micromashor•3/24/23, 1:13 AM

thanks

B

beelzabub•3/24/23, 5:20 AM

In terms of actual numbers, it depends but typically KV is tens of ms or less (working on bringing that down) globally whereas R2 is best case ~30ms if you’re very very lucky and probably closer to hundreds of ms from arbitrary locations.

M

micromashor•3/24/23, 5:24 AM

I have a blurb of text that changes daily on the homepage of this website, so with that info, I changed it to cache the current day's blurb in KV just to reduce the load time a bit and it's noticeably better

M

micromashor•3/24/23, 5:27 AM

looks like pulling from R2 was taking 109ms on average and KV is averaging 4ms

S

SK•3/24/23, 6:27 AM

hey, I've got a question about KV wording in the docs:

Workers KV is an eventually consistent system, meaning that reads will sometimes reflect an older state of the system. While writes will often be visible globally immediately, it can take up to 60 seconds before reads in all edge locations are guaranteed to see the new value.

Workers KV is an eventually consistent system, meaning that reads will sometimes reflect an older state of the system. While writes will often be visible globally immediately, it can take up to 60 seconds before reads in all edge locations are guaranteed to see the new value.

Workers KV is an eventually consistent system, meaning that reads will sometimes reflect an older state of the system. While writes will often be visible globally immediately, it can take up to 60 seconds before reads in all edge locations are guaranteed to see the new value.

Workers KV is an eventually consistent system, meaning that reads will sometimes reflect an older state of the system. While writes will often be visible globally immediately, it can take up to 60 seconds before reads in all edge locations are guaranteed to see the new value.

S

SK•3/24/23, 6:29 AM

does that mean if I write a new key, it will be visible to other workers really quickly (as in a few seconds max)? but if I update an existing key then the new value may take up to 60 seconds for it to propagate through?

S

SK•3/24/23, 6:30 AM

I'm planning on using kv to deduplicate content, so each worker would first check if some key exists before continuing, is this a valid use case?

S

SK•3/24/23, 6:30 AM

thanks!

Mmicromashor looks like pulling from R2 was taking 109ms on average and KV is averaging 4ms

H

Hello, I’m Allie!•3/24/23, 8:13 AM

Note too that this isn’t entirely fair to R2, since it doesn’t cache responses

K

Kirk Ouimet•3/24/23, 1:33 PM

Hi team - is there any way for me to access my production KV directly from my dev machine? Wanting to do some operations locally without pushing a worker

E

Erisa•3/24/23, 1:35 PM

If you install wrangler:

  wrangler kv:namespace                🗂️  Interact with your Workers KV Namespaces
  wrangler kv:key                      🔑 Individually manage Workers KV key-value pairs
  wrangler kv:bulk                     💪 Interact with multiple Workers KV key-value pairs at once

  wrangler kv:namespace                🗂️  Interact with your Workers KV Namespaces
  wrangler kv:key                      🔑 Individually manage Workers KV key-value pairs
  wrangler kv:bulk                     💪 Interact with multiple Workers KV key-value pairs at once

  wrangler kv:namespace                🗂️  Interact with your Workers KV Namespaces
  wrangler kv:key                      🔑 Individually manage Workers KV key-value pairs
  wrangler kv:bulk                     💪 Interact with multiple Workers KV key-value pairs at once

  wrangler kv:namespace                🗂️  Interact with your Workers KV Namespaces
  wrangler kv:key                      🔑 Individually manage Workers KV key-value pairs
  wrangler kv:bulk                     💪 Interact with multiple Workers KV key-value pairs at once

E

Erisa•3/24/23, 1:35 PM

Commands:
  wrangler kv:key put <key> [value]  Writes a single key/value pair to the given namespace.
  wrangler kv:key list               Outputs a list of all keys in a given namespace.
  wrangler kv:key get <key>          Reads a single value by key from the given namespace.
  wrangler kv:key delete <key>       Removes a single key value pair from the given namespace.

Commands:
  wrangler kv:key put <key> [value]  Writes a single key/value pair to the given namespace.
  wrangler kv:key list               Outputs a list of all keys in a given namespace.
  wrangler kv:key get <key>          Reads a single value by key from the given namespace.
  wrangler kv:key delete <key>       Removes a single key value pair from the given namespace.

Commands:
  wrangler kv:key put <key> [value]  Writes a single key/value pair to the given namespace.
  wrangler kv:key list               Outputs a list of all keys in a given namespace.
  wrangler kv:key get <key>          Reads a single value by key from the given namespace.
  wrangler kv:key delete <key>       Removes a single key value pair from the given namespace.

Commands:
  wrangler kv:key put <key> [value]  Writes a single key/value pair to the given namespace.
  wrangler kv:key list               Outputs a list of all keys in a given namespace.
  wrangler kv:key get <key>          Reads a single value by key from the given namespace.
  wrangler kv:key delete <key>       Removes a single key value pair from the given namespace.

K

Kirk Ouimet•3/24/23, 2:01 PM

Thanks Erisa! Sorry I should have clarified, accessing prod KV from a local worker in wrangler

W

Walshy•3/24/23, 2:16 PM

wrangler dev

wrangler dev

will use the remote KV namespace of your choosing

W

Walshy•3/24/23, 2:16 PM

you cannot access it with

--local

--local

however

SSK does that mean if I write a new key, it will be visible to other workers really ...

B

beelzabub•3/24/23, 2:20 PM

The question is a bit vague. To other workers in the exact same colo? Yes. To other colos world wide? No. That takes time. But creating a key and updating it are identical operations. When you ask for an object and there isn’t one, the 404 is cached just like the value would.

SSK I'm planning on using kv to deduplicate content, so each worker would first chec...

B

beelzabub•3/24/23, 2:27 PM

It might be, it might not be. You need to think about what your storage needs invariant assumptions are. “Deduplicating content” is vague enough that it covers a lot of different techniques and ultimate purposes with different performance preferences for certain operation speeds. When you say deduplication, I start thinking what is the cardinality of the key space you are reading from? If you’re doing content based duplication like writing key per chunk of the body contents, are you basically going to get a cold read on every block and therefore caching isn’t going to help? You will probably run into rate limiting we will impose on you.

Bbeelzabub I don’t understand the question. What cache API?

K

kikaOP•3/24/23, 3:21 PM

this one https://developers.cloudflare.com/workers/runtime-apis/cache/

Cache · Cloudflare Workers docs

The Cache API allows fine grained control of reading and writing from the Cloudflare edge network cache.

HHello, I’m Allie!Note too that this isn’t entirely fair to R2, since it doesn’t cache responses

M

micromashor•3/24/23, 3:31 PM

either way I was just looking for a faster solution to speed things up

WWalshy you cannot access it with `--local` however

D

DaniFoldi•3/24/23, 4:03 PM

with

--experimental-local

--experimental-local

--experimental-local

--experimental-local, you can, I think the flag is

--experimental-local-remote-kv

--experimental-local-remote-kv

--experimental-local-remote-kv

--experimental-local-remote-kv

Kkika this one https://developers.cloudflare.com/workers/runtime-apis/cache/

B

beelzabub•3/24/23, 4:48 PM

KV and Cache are totally separate APIs. If you're combining cache and KV, that's something you have to manage. The caching that KV uses is internal to its service. KV.delete will delete the key. KV.put will update the value associated with a key. KV.get will retrieve a value consistent with backend storage and it's usually a cached read.

Bbeelzabub KV and Cache are totally separate APIs. If you're combining cache and KV, that's...

K

kikaOP•3/24/23, 5:01 PM

I'm not combining, it wasn't clear to me what "cache" did you mean. Thanks for the clarification. What happens if I do KV.put with one metadata and then repeat the same KV.put with the same key and value, but different metadata? Will KV update the metadata? Will metadata get updated on the edge locations where it was cached after reads?

hi, who can shed some light on the mysterious 1000 operations per invocation limit in KV? I have a w

Similar Threads

Similar Threads

Similar Threads