even if it's somehow valid syntax, it's a very odd way of doing an if/else

IIan even if it's somehow valid syntax, it's a very odd way of doing an if/else

A

Yeah, I was just prototyping, so was not really looking into making it look good. But you are right about that, it does look odd

IIan even if it's somehow valid syntax, it's a very odd way of doing an if/else

A

Aditya Mathur•7/2/24, 3:46 AM

Nope, this was not it.

AAditya Mathur Hello! I am using `WorkerKV` for some experiment and this is my code ```ts awai...

A

Aditya Mathur•7/2/24, 3:52 AM

Hey @Community Champion guys, sorry for pinning you like this but I am stuck on this for a week. Does anyone knows what is wrong in this?

A

andrew_nyr•7/2/24, 4:00 AM

?pings

F

Flare•7/2/24, 4:00 AM

Please do not ping community members for non-moderation reasons. Doing so will not solve your issue faster and will make people less likely to want to help you.

A

Aditya Mathur•7/2/24, 4:20 AM

okay

I

IanOP•7/2/24, 5:07 AM

@Aditya Mathur are you sure you meant to use

expiration

expiration

(seconds since the epoch) or

expirationTtl

expirationTtl

(seconds from now) ? Because I saw your

this.windowMs = 60_000 // 1 minute

this.windowMs = 60_000 // 1 minute

comment, but you also have

expirationTtl: this.windowMs

expirationTtl: this.windowMs

and that would be wrong; it would be 60000 seconds.

IIan @Aditya Mathur are you sure you meant to use `expiration` (seconds since the epo...

A

Aditya Mathur•7/2/24, 5:13 AM

Yeah, you are right about that but it is commented, so it is not getting executed. This is a rate limiter, so the problem is every time a user requests the server I have to update the

totalHits

totalHits

but the expiration time remains that same, thats why it is stored in

resetTime

resetTime

. So when I am updating it, I am passing the

expiration

expiration

again.

I

IanOP•7/2/24, 5:14 AM

In the docs

As of January 2022, expiration targets that are less than 60 seconds into the future are not supported. This is true for both expiration methods.

I

IanOP•7/2/24, 5:16 AM

@Aditya Mathur when you read the KV, doesn't it also contain the

expiration

expiration

attribute?

I

IanOP•7/2/24, 5:17 AM

If so, just merge a default payload with a payload that it found...

A

Aditya Mathur•7/2/24, 5:18 AM

So what you are saying is, by updating the entry I am not overwriting the

expiration

expiration

?

I

IanOP•7/2/24, 5:19 AM

yeah, don't bother updating expiration since it's basically "expires at Time", while the expirationTtl is "expires in X"

I

IanOP•7/2/24, 5:20 AM

expiration is relative to a fixed point in time, the epoch. while expirationTtl is relative to right now.

I

IanOP•7/2/24, 5:21 AM

so do this

have a default payload
get the KV
merge it with the default payload using spread operations
put the KV

A

Aditya Mathur•7/2/24, 5:22 AM

Okay let me try this

AAditya Mathur Okay let me try this

I

IanOP•7/2/24, 5:24 AM

Something like this

const defaultPayload = {
  hits: 1,
  expiration: new Date(new Date().getTime() + 60_000),
}

const record = await env.NAMESPACE.get('key', { type: 'json' })

const payload = {
  ...defaultPayload,
  ...(record ?? {}), // if record is nullish, spread an empty object in its place
}

await env.NAMESPACE.put('key', payload, { expiration: payload.expiration })

const defaultPayload = {
  hits: 1,
  expiration: new Date(new Date().getTime() + 60_000),
}

const record = await env.NAMESPACE.get('key', { type: 'json' })

const payload = {
  ...defaultPayload,
  ...(record ?? {}), // if record is nullish, spread an empty object in its place
}

await env.NAMESPACE.put('key', payload, { expiration: payload.expiration })

const defaultPayload = {
  hits: 1,
  expiration: new Date(new Date().getTime() + 60_000),
}

const record = await env.NAMESPACE.get('key', { type: 'json' })

const payload = {
  ...defaultPayload,
  ...(record ?? {}), // if record is nullish, spread an empty object in its place
}

await env.NAMESPACE.put('key', payload, { expiration: payload.expiration })

const defaultPayload = {
  hits: 1,
  expiration: new Date(new Date().getTime() + 60_000),
}

const record = await env.NAMESPACE.get('key', { type: 'json' })

const payload = {
  ...defaultPayload,
  ...(record ?? {}), // if record is nullish, spread an empty object in its place
}

await env.NAMESPACE.put('key', payload, { expiration: payload.expiration })

IIan Something like this ```typescript const defaultPayload = { hits: 1, expirati...

A

Aditya Mathur•7/2/24, 5:35 AM

No Luck

A

Aditya Mathur•7/2/24, 5:36 AM

Here is the code -

async increment(key: string): Promise<ClientRateLimitInfo> {
    const keyWithPrefix = this.prefixKey(key);

    const defaultPayload = {
      totalHits: 1,
      resetTime: new Date(Date.now() + this.windowMs),
    }

    // @ts-ignore
    const record: Required<ClientRateLimitInfo> | null = await this.namespace.get<Required<ClientRateLimitInfo>>(
      keyWithPrefix,
      "json",
    );

    const payload = { ...defaultPayload, ...(record ? { totalHits: record.totalHits + 1, resetTime: new Date(record.resetTime) } : {}) }

    console.log("Payload - ", payload, " Expire - ", payload.resetTime.toLocaleString(), " Current - ", new Date().toLocaleString())

    await this.namespace.put(keyWithPrefix, JSON.stringify(payload), {
      expirationTtl: !record ? payload.resetTime.getTime() / 1000 : undefined,
    });

    return payload;
  }

async increment(key: string): Promise<ClientRateLimitInfo> {
    const keyWithPrefix = this.prefixKey(key);

    const defaultPayload = {
      totalHits: 1,
      resetTime: new Date(Date.now() + this.windowMs),
    }

    // @ts-ignore
    const record: Required<ClientRateLimitInfo> | null = await this.namespace.get<Required<ClientRateLimitInfo>>(
      keyWithPrefix,
      "json",
    );

    const payload = { ...defaultPayload, ...(record ? { totalHits: record.totalHits + 1, resetTime: new Date(record.resetTime) } : {}) }

    console.log("Payload - ", payload, " Expire - ", payload.resetTime.toLocaleString(), " Current - ", new Date().toLocaleString())

    await this.namespace.put(keyWithPrefix, JSON.stringify(payload), {
      expirationTtl: !record ? payload.resetTime.getTime() / 1000 : undefined,
    });

    return payload;
  }

async increment(key: string): Promise<ClientRateLimitInfo> {
    const keyWithPrefix = this.prefixKey(key);

    const defaultPayload = {
      totalHits: 1,
      resetTime: new Date(Date.now() + this.windowMs),
    }

    // @ts-ignore
    const record: Required<ClientRateLimitInfo> | null = await this.namespace.get<Required<ClientRateLimitInfo>>(
      keyWithPrefix,
      "json",
    );

    const payload = { ...defaultPayload, ...(record ? { totalHits: record.totalHits + 1, resetTime: new Date(record.resetTime) } : {}) }

    console.log("Payload - ", payload, " Expire - ", payload.resetTime.toLocaleString(), " Current - ", new Date().toLocaleString())

    await this.namespace.put(keyWithPrefix, JSON.stringify(payload), {
      expirationTtl: !record ? payload.resetTime.getTime() / 1000 : undefined,
    });

    return payload;
  }

async increment(key: string): Promise<ClientRateLimitInfo> {
    const keyWithPrefix = this.prefixKey(key);

    const defaultPayload = {
      totalHits: 1,
      resetTime: new Date(Date.now() + this.windowMs),
    }

    // @ts-ignore
    const record: Required<ClientRateLimitInfo> | null = await this.namespace.get<Required<ClientRateLimitInfo>>(
      keyWithPrefix,
      "json",
    );

    const payload = { ...defaultPayload, ...(record ? { totalHits: record.totalHits + 1, resetTime: new Date(record.resetTime) } : {}) }

    console.log("Payload - ", payload, " Expire - ", payload.resetTime.toLocaleString(), " Current - ", new Date().toLocaleString())

    await this.namespace.put(keyWithPrefix, JSON.stringify(payload), {
      expirationTtl: !record ? payload.resetTime.getTime() / 1000 : undefined,
    });

    return payload;
  }

A

Aditya Mathur•7/2/24, 5:59 AM

Is there a working example of using

WorkerKV

WorkerKV

with

hono

hono

hono

hono which uses

expiration

expiration

or

expirationTtl

expirationTtl

?

AAditya Mathur Is there a working example of using `WorkerKV` with `hono` which uses `expiratio...

I

IanOP•7/2/24, 6:14 AM

If you're using hono just use https://github.com/rhinobase/hono-rate-limiter

GitHub

GitHub - rhinobase/hono-rate-limiter: Rate Limit middleware for Hon...

Rate Limit middleware for Hono Web Server . Contribute to rhinobase/hono-rate-limiter development by creating an account on GitHub.

IIan If you're using hono just use https://github.com/rhinobase/hono-rate-limiter

A

Aditya Mathur•7/2/24, 6:14 AM

That's interesting, I am the creator of that lib

AAditya Mathur That's interesting, I am the creator of that lib 🤣

I

IanOP•7/2/24, 6:15 AM

haha, so you are

A

Aditya Mathur•7/2/24, 6:15 AM

I have added support for cloudflare and the

WorkerKV

WorkerKV

is giving me some trouble

AAditya Mathur I have added support for cloudflare and the `WorkerKV` is giving me some trouble

I

IanOP•7/2/24, 6:17 AM

I'm curious why you're storing the

resetTime

resetTime

if there's already an

expiration

expiration

field?

IIan I'm curious why you're storing the `resetTime` if there's already an `expiration...

A

Aditya Mathur•7/2/24, 6:20 AM

This is what I have concluded, we need to update the

expiration

expiration

when we update the value, if you not it's not gonna expire.

AAditya Mathur This is what I have concluded, we need to update the `expiration` when we update...

I

IanOP•7/2/24, 6:21 AM

Sure, but when you

get('key')

get('key')

get('key')

get('key') it returns an object with the

expiration

expiration

field, right? So if it already has that field, you can re-write it back without altering it.

I

IanOP•7/2/24, 6:22 AM

When you're altering it, you're taking the

expiration

expiration

which is fixed in time, and keep adding to it each time the request happens. is that what you want? for the expiration to be moved forward?

A

Aditya Mathur•7/2/24, 6:22 AM

If you update the expiration on each request, you will then be putting values less than 60 seconds which will not be getting registered

A

Aditya Mathur•7/2/24, 6:22 AM

All I am concluding from this is WorkerKV cannot be used for rate limiting in general because the API is not build for this purpose. It is more for having global config for env or feature flags

I

IanOP•7/2/24, 6:22 AM

it could be; I have a lateral solution...

A

Aditya Mathur•7/2/24, 6:23 AM

I am listening

I

IanOP•7/2/24, 6:23 AM

make a record for each request with a ttl of 15 minutes.

I

IanOP•7/2/24, 6:23 AM

then list and count all the records on each request to see if they exceed the limit.

I

IanOP•7/2/24, 6:24 AM

the maximum number of records you'll have is the

limit

limit

limit

limit. so not very many.

I

IanOP•7/2/24, 6:24 AM

and you never have to write to them after the first

put()

put()

; you fire and foget them until the next request where you

list()

list()

and count.

I

IanOP•7/2/24, 6:25 AM

basically use the namespace as a "leaky bucket"

IIan then list and count all the records on each request to see if they exceed the li...

A

Aditya Mathur•7/2/24, 6:26 AM

Yes you can manually work this out i.e. without the use of

expiration

expiration

prop. But at the end of the day it will be too expensive to use this solution for rate limiting.

IIan and you never have to write to them after the first `put()`; you fire and foget ...

A

Aditya Mathur•7/2/24, 6:27 AM

Because the limit is 1,000/day. Check this https://developers.cloudflare.com/kv/platform/pricing/

Cloudflare Docs

Pricing · Cloudflare Workers KV

Workers KV is included in both the Free and Paid Workers plans.

I

IanOP•7/2/24, 6:29 AM

You could use D1 https://developers.cloudflare.com/d1/platform/pricing/

Cloudflare Docs

Pricing · Cloudflare D1 docs

D1 bills based on:

I

IanOP•7/2/24, 6:29 AM

There's also durable objects.

IIan There's also durable objects.

A

Aditya Mathur•7/2/24, 6:29 AM

Yes, that is what I am thinking about to drop the support for

WorkerKV

WorkerKV

and use

DurableObjects

DurableObjects

DurableObjects

DurableObjects

AAditya Mathur Yes, that is what I am thinking about to drop the support for `WorkerKV` and use...

I

IanOP•7/2/24, 6:31 AM

the kv plan for paid customers still makes sense.

AAditya Mathur Yes, that is what I am thinking about to drop the support for `WorkerKV` and use...

I

IanOP•7/2/24, 6:33 AM

https://developers.cloudflare.com/durable-objects/examples/build-a-rate-limiter/ here's an actual example of building a rate limiter with durable objects

IIan https://developers.cloudflare.com/durable-objects/examples/build-a-rate-limiter/...

A

Aditya Mathur•7/2/24, 6:34 AM

Nice, will be building something like this then

I

IanOP•7/2/24, 6:35 AM

https://community.cloudflare.com/t/creating-a-rate-limiter-using-workers-kv/319256 there is also this forum post about using WorkerKV for rate limiting

I

IanOP•7/2/24, 6:36 AM

KV is not really a good idea here due to it’s eventually consistent nature. You can however make a rate-limit from Durable Objects 272, which some others have already made from too.

IIan > KV is not really a good idea here due to it’s eventually consistent nature. Yo...

A

Aditya Mathur•7/2/24, 6:36 AM

Yeah

A

Aditya Mathur•7/2/24, 6:37 AM

Now I know why is that

V

vyastj•7/2/24, 10:05 AM

Hello

Can someone help me here or point to correct channel ?

workers-ai

even if it's somehow valid syntax, it's a very odd way of doing an if/else

Similar Threads

Similar Threads

Similar Threads