need a suggestion: Need to have a counter but using kv can be an issue as its say only 1 write per s

M

Mrinank•5/31/24, 8:49 PM

Why kv doesn't work in typescript code. I am not getting error in js files, having problem with env in ts

MMrinank Why kv doesn't work in typescript code. I am not getting error in js files, havi...

M

Mrinank•5/31/24, 8:55 PM

Cannot read properties of undefined (reading 'put')

MMrinank Why kv doesn't work in typescript code. I am not getting error in js files, havi...

M

Matt Silverlock•6/2/24, 9:39 PM

Can you share the code, including how you have defined your bindings?

MMatt Silverlock Can you share the code, including how you have defined your bindings?

M

Mrinank•6/3/24, 7:29 PM

Its working now

M

Mrinank•6/3/24, 7:30 PM

sharing the code for others

name = "newcron"
main = "src/index.ts"
compatibility_date = "2024-05-29"


[triggers]
crons = ["* */12 * * *"]

[[kv_namespaces]]
binding = "KV"
id = "b6783170bcde4c6f9d9e6152dc8d1027"

[ai]
binding = "AI"

name = "newcron"
main = "src/index.ts"
compatibility_date = "2024-05-29"


[triggers]
crons = ["* */12 * * *"]

[[kv_namespaces]]
binding = "KV"
id = "b6783170bcde4c6f9d9e6152dc8d1027"

[ai]
binding = "AI"

name = "newcron"
main = "src/index.ts"
compatibility_date = "2024-05-29"


[triggers]
crons = ["* */12 * * *"]

[[kv_namespaces]]
binding = "KV"
id = "b6783170bcde4c6f9d9e6152dc8d1027"

[ai]
binding = "AI"

name = "newcron"
main = "src/index.ts"
compatibility_date = "2024-05-29"


[triggers]
crons = ["* */12 * * *"]

[[kv_namespaces]]
binding = "KV"
id = "b6783170bcde4c6f9d9e6152dc8d1027"

[ai]
binding = "AI"

A

AKStriker98•6/4/24, 10:06 PM

Is there a way to deal with the graveyard that is created from unread items being deleted by the TTL? Had about 5 GB of unread data expire and now we cannot get to the valid data with list because it’s stuck churning through the graveyard. We don’t want to delete and recreate the kv store since it’s tied to multiple apis inputting data to them and being read by workers

K

King Kazu•6/6/24, 4:48 AM

Im needing some help with kv fetching. I am using BetterKV for fetching a rarely-updated value (but pretty important to have latest value with semi-low latency). This value is actually stored in my db and passed into kv and cached for performance (i believe kv w/ caching is faster for global users but I need to still actually do performance testing). Ideally, I want the

cacheTtl

cacheTtl

cacheTtl

cacheTtl and

kvCacheTtl

kvCacheTtl

kvCacheTtl

kvCacheTtl to be high for low latency, but I dont want to have to wait for

kvCacheTtl

kvCacheTtl

kvCacheTtl

kvCacheTtl to expire to fetch the updated value if needed. Am I thinking about this right and do I have to just bite the bullet and have low cache ttl values?

Rraj.js need a suggestion: Need to have a counter but using kv can be an issue as its sa...

N

Nob•6/6/24, 1:05 PM

go with redis

Rraj.js need a suggestion: Need to have a counter but using kv can be an issue as its sa...

I

Isaac McFadyen•6/6/24, 7:36 PM

KV isn't suited to a counter. KV is eventually consistent so you might not see updates at other locations for up to 60s, meaning your counter will easily fall behind/be invalid - for example, if I get + increment + set the counter, and you do it 15s later, you might not see my increment so it would be lost.

I

Isaac McFadyen•6/6/24, 7:36 PM

I'd recommend something like Durable Objects or Redis if you need consistency like a counter.

A

AstroDev•6/7/24, 2:45 PM

Hi - I'm thinking of using KV to debounce a webhook - is there a way to trigger a worker when a value with a TTL is deleted from KV?

AAstroDev Hi - I'm thinking of using KV to debounce a webhook - is there a way to trigger ...

T

thomasgauvin•6/7/24, 3:22 PM

What do you mean by this? It's not possible to trigger a worker when a value expires from TTL, but if you have more context around the problem I might be able to help come up with the right solution

Tthomasgauvin What do you mean by this? It's not possible to trigger a worker when a value exp...

A

AstroDev•6/7/24, 3:27 PM

Thanks Thomas - What I need to do is collect all webhook requests in a given X minute period - and emit one request. Just looking at Durable Objects - that has an alarm function - this sounds like a better option as I can schedule an alarm in future in X minutes?

T

thomasgauvin•6/7/24, 3:50 PM

Yes, Durable Objects seems like a better fit for the job, since you will be able to manage prior state (the calls you want to debounce) and hold/act upon the most recent event after debounce. I'm assuming here that you aren't able to debounce client side since the webhook is a feature you provide to your users?

Tthomasgauvin Yes, Durable Objects seems like a better fit for the job, since you will be able...

A

AstroDev•6/7/24, 3:58 PM

Yeah - this would be a nice feature to add to Pages - the ability to debounce the deployment hook for static sites - that are triggered by CMS save buttons. Some CMSes let you override the button to debouce it within the CMS some don't.

Is there documentation for this: import { DurableObject } from "cloudflare:workers";

Some examples on the CF site use it, some don't.

T

thomasgauvin•6/7/24, 4:27 PM

I don't have the answer for the import statement off the top of my head (the discrepancy might be something DO folks might be more aware of).

I DM'd you for more info about your setup, I'm curious how your users are using CMS', how you are using Pages, and then where you layer fits in

A

AKStriker98•6/7/24, 5:57 PM

Is there a way that a bulk get can be implemented for cloudflare KVs? Like in a way that can keep the limit but only require one call to limit the number of requests that are required to be sent for large data querying? For example, I want to create a bulk endpoint for a large number of lookups. Currently, my client is held up by making thousands of requests which are held up by browser limitations. If I can bundle them in say 200-300 requests to a bulk endpoint it would save my client very precious time

Eengrtitus It appears making an http request is faster than getting a record from kv. Our ...

N

Nob•6/7/24, 11:26 PM

yh kv seem slow, i removed it from my website and its faster now

C

Chaika•6/8/24, 7:45 PM

Well should be a fair bit faster if the key is cached/not having to pull from a central store

Eengrtitus It appears making an http request is faster than getting a record from kv. Our ...

C

Chaika•6/8/24, 7:48 PM

If you're just using KV as cache, you don't need to await the put to move onto your next action. You can do ctx.waitUntil(KV.put(...)); and it will not block the response and let it finish before killing the request context.

Kv only has two central stores (US, EU), so espec in some places like APAC cold reads will be expensive. If your API isn't that slow or expensive, you could consider just using the Cache API https://developers.cloudflare.com/workers/runtime-apis/cache/ to help out a bit, it's always going to be fast (only a few ms) and make sure to use waitUntil when putting into it as well to not have to block the client just for caching

A

AKStriker98•6/11/24, 10:13 PM

I’ve been noticing that my get requests to a kv store have had the latency gradually increasing overtime. This latency changes depending

Inside of the worker code edit page, when my worker is run, all 30ms of the wall click time utilized is caused by my get request to a kv storage after 5 requests, before then I’m seeing up to 400ms of time waiting for the get requests to complete.

On my end client is interacting with a worker, the median response time is sitting at 200ms after 300 requests.

With a curl request directly to the kv store, it is averaging 250-300ms (calculated from rapid succession of 150 requests).

I remember when I last benchmarked my worker I was averaging about a 50-100 ms delay per request. I have not changed anything since last year and it processes about 200 requests per second daily.

Has there been any cause behind this level of degradation in Cloudflare KV response time?

M

Mrinank•6/12/24, 1:42 PM

pages-helpfull stack Nextjs on pages with hono | Sometimes Work, Sometimes Not

Someone check this, KV is sometimes working, sometimes not.
I tried with KV Api also but it always works in localhost but doesnt work in production

MMrinank https://discord.com/channels/595317990191398933/1250368975104249866 Someone che...

T

thomasgauvin•6/12/24, 1:49 PM

Replied in the thread, seems Next.js related rather than KV

E

esselungo•6/14/24, 2:11 PM

kv is really slow to propagate when i have my backend reading it it takes sometimes 3 minutes to get a new value. should i keep using KV or is there some solution?

U

unnoq•6/20/24, 9:58 AM

Hey folks, I've just created the

typed-kv

typed-kv

typed-kv

typed-kv.

It is a tiny TypeScript wrapper for Cloudflare KV with all the necessary features.

Check it out here: https://www.npmjs.com/package/typed-kv

npm

typed-kv

TypeScript wrapper for Cloudflare KV. Latest version: 0.0.4, last published: 14 minutes ago. Start using typed-kv in your project by running

npm i typed-kv

npm i typed-kv

npm i typed-kv

npm i typed-kv. There are no other projects in the npm registry using typed-kv.

T

thomasgauvin•6/20/24, 3:19 PM

Cool stuff! Nice extra features on top!

M

maj•6/20/24, 4:25 PM

memory cache wouldn't support expiration?

Mmaj memory cache wouldn't support expiration?

U

unnoq•6/22/24, 6:41 AM

Yes, but is it necessary on serverless environments? I mainly created this feature to reduce redundant calls to Cloudflare KV.

C

Chinoman10•6/23/24, 5:00 PM

For some reason my 'puts' aren't working...

wrangler.toml

wrangler.toml

wrangler.toml

wrangler.toml

...

[[kv_namespaces]]
    binding = "OAUTH_KV"
    id = "..."

...

...

[[kv_namespaces]]
    binding = "OAUTH_KV"
    id = "..."

...

...

[[kv_namespaces]]
    binding = "OAUTH_KV"
    id = "..."

...

...

[[kv_namespaces]]
    binding = "OAUTH_KV"
    id = "..."

...

The .ts file which has the .put

The .ts file which has the .put

The .ts file which has the .put

The .ts file which has the .put

export const onRequestGet: PagesFunction<Env> = async ({ request, env }) => {
    const { OAUTH_KV } = env;

...

try {
        const dateStamp = new Date().toUTCString();
        await OAUTH_KV.put(`smt:${user_id}`, longToken, {
            expirationTtl: expiresIn,
            metadata: {
                "added_in": dateStamp,
            },
        });
        console.log(`Stored long-lived token for user ${user_id}.`);
    } catch (e) {
        console.log(`Error storing smt long-lived token: ${e}`);
    }

export const onRequestGet: PagesFunction<Env> = async ({ request, env }) => {
    const { OAUTH_KV } = env;

...

try {
        const dateStamp = new Date().toUTCString();
        await OAUTH_KV.put(`smt:${user_id}`, longToken, {
            expirationTtl: expiresIn,
            metadata: {
                "added_in": dateStamp,
            },
        });
        console.log(`Stored long-lived token for user ${user_id}.`);
    } catch (e) {
        console.log(`Error storing smt long-lived token: ${e}`);
    }

export const onRequestGet: PagesFunction<Env> = async ({ request, env }) => {
    const { OAUTH_KV } = env;

...

try {
        const dateStamp = new Date().toUTCString();
        await OAUTH_KV.put(`smt:${user_id}`, longToken, {
            expirationTtl: expiresIn,
            metadata: {
                "added_in": dateStamp,
            },
        });
        console.log(`Stored long-lived token for user ${user_id}.`);
    } catch (e) {
        console.log(`Error storing smt long-lived token: ${e}`);
    }

export const onRequestGet: PagesFunction<Env> = async ({ request, env }) => {
    const { OAUTH_KV } = env;

...

try {
        const dateStamp = new Date().toUTCString();
        await OAUTH_KV.put(`smt:${user_id}`, longToken, {
            expirationTtl: expiresIn,
            metadata: {
                "added_in": dateStamp,
            },
        });
        console.log(`Stored long-lived token for user ${user_id}.`);
    } catch (e) {
        console.log(`Error storing smt long-lived token: ${e}`);
    }

C

Chinoman10•6/23/24, 5:02 PM

I get the "Stored long-lived token" text, but on the dashboard I see nothing.
As you can see

I have 0 writes.

CChinoman10 For some reason my 'puts' aren't working... `wrangler.toml` ``` ... [[kv_names...

K

kian•6/23/24, 5:06 PM

If this is in pages dev then it’s local and doesn’t write to your actual namespace

Kkian If this is in pages dev then it’s local and doesn’t write to your actual namespa...

C

Chinoman10•6/23/24, 5:07 PM

I figured it could be that, so I already pushed (deployed) it and it still doesn't work.

Kkian If this is in pages dev then it’s local and doesn’t write to your actual namespa...

C

Chinoman10•6/23/24, 5:09 PM

Seeing this on the dashboard...

Does that mean I have to specify two namespaces for the two different environments?

C

Chinoman10•6/23/24, 5:10 PM

Here's a more complete version of my current toml:

name="pages-name"
pages_build_output_dir = "dist"
compatibility_date = "2024-06-12"

[[kv_namespaces]]
    binding = "OAUTH_KV"
    id = "..."

[dev]
    port = 3000

[env.production]
    [env.production.vars]
        PUBLIC_ENVIRONMENT = "production"
        SOME_VAR = "BLA BLA"

[env.preview]
    [env.preview.vars]
        PUBLIC_ENVIRONMENT = "preview"
        SOME_VAR = "BLA BLA"

name="pages-name"
pages_build_output_dir = "dist"
compatibility_date = "2024-06-12"

[[kv_namespaces]]
    binding = "OAUTH_KV"
    id = "..."

[dev]
    port = 3000

[env.production]
    [env.production.vars]
        PUBLIC_ENVIRONMENT = "production"
        SOME_VAR = "BLA BLA"

[env.preview]
    [env.preview.vars]
        PUBLIC_ENVIRONMENT = "preview"
        SOME_VAR = "BLA BLA"

name="pages-name"
pages_build_output_dir = "dist"
compatibility_date = "2024-06-12"

[[kv_namespaces]]
    binding = "OAUTH_KV"
    id = "..."

[dev]
    port = 3000

[env.production]
    [env.production.vars]
        PUBLIC_ENVIRONMENT = "production"
        SOME_VAR = "BLA BLA"

[env.preview]
    [env.preview.vars]
        PUBLIC_ENVIRONMENT = "preview"
        SOME_VAR = "BLA BLA"

name="pages-name"
pages_build_output_dir = "dist"
compatibility_date = "2024-06-12"

[[kv_namespaces]]
    binding = "OAUTH_KV"
    id = "..."

[dev]
    port = 3000

[env.production]
    [env.production.vars]
        PUBLIC_ENVIRONMENT = "production"
        SOME_VAR = "BLA BLA"

[env.preview]
    [env.preview.vars]
        PUBLIC_ENVIRONMENT = "preview"
        SOME_VAR = "BLA BLA"

So perhaps I have to declare the KV under the [env.production] and [env.preview] as well?

E

ehrenmann•6/23/24, 6:03 PM

Hi, i have 2 workers that are spread on multiple domains. I save data in d1 and read kv values on each request and im afraid the limits will not be enough for me (cpu time, reads and writes and size pf the D1). Are there limits if i purchase workers for platforms?

W

Walshy•6/23/24, 6:13 PM

Workers Paid is what you'd want

CPU time on Paid is 30 seconds, reads and writes are charged if you go over the included usage.
D1's size however cannot be increased, the recommendation there is sharding into multiple DBs

Kkian If this is in pages dev then it’s local and doesn’t write to your actual namespa...

C

Chinoman10•6/23/24, 10:04 PM

Hey Kian, any idea what could be wrong with what I shared earlier?

CChinoman10 Hey Kian, any idea what could be wrong with what I shared earlier?

C

Chinoman10•6/24/24, 9:14 AM

Found it. It was my suspicion... the namespace needs to be declared under each env...

T

thomasgauvin•6/24/24, 7:34 PM

This isn't a feature that KV provides currently, can you provide more information on the use case? We're collecting more info in the area, interested in hearing about the level of queryability you're looking to get vs D1

C

Chinoman10•6/24/24, 8:19 PM

I've been getting highly inconsistent experiences with KV unfortunately...
I know that KV is "eventually consistent", but even after several hours of an entry being added (put), I still get null when trying to obtain those keys... Sometimes even hours later.

I can understand a few minutes delay for newly created values, and I can understand it takes much longer for changes/deletes (unless
expires
expires
expires
expires is set, in which case I'd assume it doesn't take long to disappear after), but not being able to 'GET' the value on a Pages Function, several days after it's already been

put

put

put

put (and is shown on the dashboard), feels a lot like a bug and not a "that's how it is".

CChinoman10 I've been getting highly inconsistent experiences with KV unfortunately... I kno...

T

thomasgauvin•6/24/24, 8:57 PM

That does seem highly unusual (ie not something that is intended behavior). Do you specify a cacheTtl on these read operations? Are you accessing via the binding or using the API? Are you verifying from the deployed Worker? How are you verifying that your values are not getting updated (via a print in Worker logs, etc.)?

Tthomasgauvin That does seem highly unusual (ie not something that is intended behavior). Do y...

C

Chinoman10•6/24/24, 10:24 PM

Thanks for replying Thomas

I don't specify a cache, no.

I'm accessing via the Binding (declared via

wrangler.toml

wrangler.toml

wrangler.toml

wrangler.toml) on a Pages project (with a Function). I can see that when I run wrangler pages locally it binds correctly (if it didn't, it wouldn't be able to fetch any value anyway, but it can, albeit only one).

If you'd like I can tell you the endpoint where the function is deployed to and you can test on your own browser (both the put and the read); right now it doesn't have any auth (yet). Just send me a DM and I'll reply

(or open a private thread here if you have that permission)

need a suggestion: Need to have a counter but using kv can be an issue as its say only 1 write per s

Similar Threads

Similar Threads

Similar Threads