Cloudflare Developers•2y ago

I know, but I still don't know how to get first and last one since [0] and [length-1] don't work

Bbz_008 I know, but I still don't know how to get first and last one since [0] and [leng...

Hello, I’m Allie!•2/6/24, 8:23 AM

You have to run list multiple times if you have more than 1k keys

bz_008OP•2/6/24, 8:23 AM

yeah, that's another issue

bz_008OP•2/6/24, 8:25 AM

KV looks a little hard to use to me..

Bbz_008 yeah, that's another issue

Hello, I’m Allie!•2/6/24, 8:27 AM

Like this:

async function getFirstAndLast(KV: KVNamespace) {
    let listResult = await KV.list();
    const first = listResult.keys[0];
    while(!listResult.list_complete) {
        listResult = await KV.list({cursor: listResult.cursor});
    }
    const last = listResult.keys[listResult.keys.length - 1];
    return { first, last };
}

async function getFirstAndLast(KV: KVNamespace) {
    let listResult = await KV.list();
    const first = listResult.keys[0];
    while(!listResult.list_complete) {
        listResult = await KV.list({cursor: listResult.cursor});
    }
    const last = listResult.keys[listResult.keys.length - 1];
    return { first, last };
}

HHello, I’m Allie!Like this: ```ts async function getFirstAndLast(KV: KVNamespace) { let listR...

bz_008OP•2/6/24, 8:35 AM

Cool, thanks for the help

Santosh•2/8/24, 5:03 PM

Guys, I have stored a large object in KV. Definitely not 25MB limit being reached, and write it using Json.stringify. When I read it back, ocassionally Json.parse throws Unterminated string error. How do I handle this. Seems like this happens with large valiues.

Santosh•2/8/24, 5:03 PM

It does appear like JSON is not the right format for KV for large objects. What do I do?

Santosh•2/8/24, 5:05 PM

Is Json the only format to write to KV? Can someone throw some light here? This is currently happening locally, not sure if we will see this on CF>

zegevlier•2/8/24, 5:05 PM

Any reason you're not just putting the object directly into KV, instead of converting it to JSON first?

Santosh•2/8/24, 5:05 PM

Thats what the current documentation shows.

Santosh•2/8/24, 5:06 PM

Is it possible to write raw object type to KV?

zegevlier•2/8/24, 5:06 PM

I thought it was but looking at the docs I might be misremembering

zegevlier•2/8/24, 5:06 PM

let me check

Zzegevlier Any reason you're not just putting the object directly into KV, instead of conve...

Hello, I’m Allie!•2/8/24, 5:06 PM

KV only takes strings/ArrayBuffers no?

Hello, I’m Allie!•2/8/24, 5:07 PM

ReadableStreams that are stored as an ArrayBuffer

zegevlier•2/8/24, 5:09 PM

Yea I was misremembering, I thought both DOs and KV supported that but it seems only DOs do

Zzegevlier Yea I was misremembering, I thought both DOs and KV supported that but it seems ...

Hello, I’m Allie!•2/8/24, 5:10 PM

Yeah. DOs were the first to use v8.serialize. This also means that KV doesn't support any of the more complex objects

SSantosh Is Json the only format to write to KV? Can someone throw some light here? This ...

zegevlier•2/8/24, 5:10 PM

You can write to KV using any format you want, you just need to serialize it first. JSON should work though, unless you go over the 25mb limit

zegevlier•2/8/24, 5:11 PM

Are you sure the JSON stringified object also does not hit that limit?

Santosh•2/8/24, 5:21 PM

yes, its like 2 MB when I pull the data from another server prior to keeping it in KV

Santosh•2/8/24, 5:21 PM

I used postman to veriify the graphql response

Santosh•2/8/24, 5:22 PM

Can I get some alternatives to Json that works on Cloudflare worker?

Ttoinbis Hey, where can I read about KV scaling strategies? My values are simple - object...

Hello, I’m Allie!•2/10/24, 5:17 AM

I would guess that the falloff happens in the billions/trillions of keys, if ever

Hello, I’m Allie!•2/10/24, 5:17 AM

Since Pages runs on KV

Hello, I’m Allie!•2/10/24, 5:18 AM

I don’t think performance is particularly tied to the number of keys

Ttoinbis Thanks @HardlyWorkin' ! What is the performance tied to then?:)

Hello, I’m Allie!•2/10/24, 5:21 AM

Latency to the core datacenters mostly, when the key you are asking for isn’t cached

Ttoinbis Interesting! I wonder -is this aspect of KV documented anywhere?

Hello, I’m Allie!•2/10/24, 5:23 AM

That it is highly scalable?

Hello, I’m Allie!•2/10/24, 5:23 AM

Or…

Ttoinbis I mean, which are the core data centers, how do I know which values are cached a...

Hello, I’m Allie!•2/10/24, 5:30 AM

There are two cores, one in North America, one in Europe. Don’t think the actual locations are public

Hello, I’m Allie!•2/10/24, 5:31 AM

And afaik you can’t really tell if a value is stale, other than that it isn’t updated for 60+ seconds

Alvee•2/10/24, 6:15 AM

for my multitenant saas I want to store usage for each tenant usage of workers requests having only 1 worker main backend api. want to track the uages of teach tenant using their StoreId (ecommerce saas). Saving request count on each request (using hono js btw) to KV is a good idea or should I store them in a mysql or is this complete madness any better solutions ?

Alvee•2/10/24, 6:38 AM

how does cloudflare store workers request count. :blob_sweat:

AAlvee how does cloudflare store workers request count. :blob_sweat:

Hello, I’m Allie!•2/10/24, 6:46 AM

I would guess it is stored in CF's internal version of AnalyticsEngine

AAlvee for my multitenant saas I want to store usage for each tenant usage of workers r...

Hello, I’m Allie!•2/10/24, 6:46 AM

I would probably go with a Durable Object per customer. With KV, you aren't guaranteed atomic writes, so increments could be overwritten

HHello, I’m Allie!I would probably go with a Durable Object per customer. With KV, you aren't guar...

Alvee•2/10/24, 7:14 AM

another concern of mine is people dos attack from localhost or unknown origins. is it possible to only allow traffic from tenant website domain in workers. IG maybe I have to set this up in my api domain security WAF /?

AAlvee another concern of mine is people dos attack from localhost or unknown origins. ...

Hello, I’m Allie!•2/10/24, 7:16 AM

You can check the origin header, but that does not prevent someone from spoofing said header. Not too difficult to do if you are using curl

HHello, I’m Allie!You can check the origin header, but that does not prevent someone from spoofing...

Alvee•2/10/24, 7:18 AM

I mean checking inside worker mean request already made. no point in there

Alvee•2/10/24, 7:18 AM

in the domain zone level or something is I want

AAlvee in the domain zone level or something is I want

Hello, I’m Allie!•2/10/24, 7:18 AM

Yeah, I meant I think you can check it at the WAF level

HHello, I’m Allie!You can check the origin header, but that does not prevent someone from spoofing...

Alvee•2/10/24, 8:06 AM

you mean this one? people can bypass this with curl or not. I am thinking of only allow tenant domain in here

AAlvee you mean this one? people can bypass this with curl or not. I am thinking of onl...

Hello, I’m Allie!•2/10/24, 8:06 AM

Are you using CF for SaaS to allow tenants to use their own domains?

Hello, I’m Allie!•2/10/24, 8:07 AM

Or do all requests go through your domain?

HHello, I’m Allie!Are you using CF for SaaS to allow tenants to use their own domains?

Alvee•2/10/24, 8:08 AM

tenant will buy their own domains and host their frontend in that domain where ever they want and I will provide api / database full backend as saas with endposint for making their ecommerce stuff. I will just white list their domains so onl y from tenant domains I get traffic and all other are blocked. cuz or else people will spam my worker endpoints

I know, but I still don't know how to get first and last one since [0] and [length-1] don't work

Similar Threads

Similar Threads

Similar Threads