Is there anything cheaper than `kv` for super simple storage on Workers? Basically, we have a boolea

Is there anything cheaper than

kv

kv

for super simple storage on Workers? Basically, we have a boolean that we need to toggle occasionally. The server is doing 10s-100s of millions of lightweight requests per day so KV is surprisingly expensive for this use case. We end up using an environment variable and flipping that variable every time we want to change the value.

Obviously, our current solution is not ideal as it requires a re-deploy to change environment variables. Any better solutions that are a bit cheaper than

kv

kv

DaveJOP•2/17/23, 7:02 PM

Perhaps we could put a cache layer in front of

kv

kv

and then we could invalidate every minute or so? Would that be a good solution?

DaveJOP•2/17/23, 7:03 PM

But you could automate the task of changing it quite easily

We need to re-deploy every time we change an

env

env

right? We can't do it on the fly?

Original message was deleted

DaveJOP•2/17/23, 7:06 PM

Nice! Cache is basically free right? Or, to be more correct, we just pay for the increased duration of the function.

DaveJOP•2/17/23, 7:07 PM

Cool,

kv

kv

with a cache ttl seems like the right solution. Any thing I'm missing?

DaveJOP•2/17/23, 7:08 PM

Perfect. Thanks @skye_31, appreciate your help.

VVitaVee if no expiration is given when storing an object in KV, will it remain stored th...

Hello, I’m Allie!•2/19/23, 2:49 PM

Indefinitely

DDaveJ Is there anything cheaper than `kv` for super simple storage on Workers? Basical...

JustinNoel•2/20/23, 5:49 PM

Perhaps D1? I know it's in alpha. If not that, then maybe PlanetScale? Their "Scaler" plan would give you 100 billion row reads/mo.

Sspongebhav Hello there, My KV looks something like this Key - apikey, Value - true/false ...

Hello, I’m Allie!•2/23/23, 8:45 AM

A Durable Object maybe?

Sspongebhav Hello there, My KV looks something like this Key - apikey, Value - true/false ...

Matt Silverlock•2/23/23, 1:24 PM

What do you want to do with the count? Just report on it? Prevent further access?

Matt Silverlock•2/23/23, 1:25 PM

Analytics Engine could be a good fit if you just want to report on it, but I would not include the key itself - maybe an ID that represents the key. https://developers.cloudflare.com/analytics/analytics-engine/get-started/

Get started with Workers Analytics Engine · Cloudflare Analytics docs

There are four steps to get started with Workers Analytics Engine:

johtso•2/23/23, 2:28 PM

Is kv suitable for session storage? https://remix.run/docs/en/v1/utils/sessions#createworkerskvsessionstorage-cloudflare-workers

johtso•2/23/23, 2:29 PM

Doesn't the eventual consistency make it unsuitable for something like this?

Jjohtso Doesn't the eventual consistency make it unsuitable for something like this?

Hello, I’m Allie!•2/23/23, 2:32 PM

No, because generally a session cookie is stored on device. And since a device generally connects to the same datacenter, updates appear instantly*

johtso•2/23/23, 2:34 PM

I'm just worried that someone might have something funky going on with their internet connection (eg. vpn stuff) and then I get strange heisenbugs

johtso•2/23/23, 2:34 PM

I'm making assumptions about where their requests are going to come from

Jjohtso I'm just worried that someone might have something funky going on with their int...

Hello, I’m Allie!•2/23/23, 2:36 PM

Even then, unless you have multiple clients with the same session making changes, it should still generally be consistent

johtso•2/23/23, 2:36 PM

It's also a little frustrating because I don't really need all the edge stuff.. people using my site are all going to be from one country

DaniFoldi•2/23/23, 2:36 PM

The only time I would actively be concerned about missing the presence of data is if I access some data that's undefined, it's cached for at least a minute, create the data, and then I have to wait a minute for the changes to appear

johtso•2/23/23, 2:37 PM

But I guess there's no way to say, always store the data here

DaniFoldi•2/23/23, 2:37 PM

for example someone tries to sign in, I check if their account exists in KV, return nope, they sign up, and they have to wait a minute for the account to "appear"

HHello, I’m Allie!Even then, unless you have multiple clients with the same session making changes...

johtso•2/23/23, 2:38 PM

I'm thinking the user does something after logging in, it tries to fetch their session to make sure they're authenticated, and it can't find it so logs them out. Is that not how it behaves?

johtso•2/23/23, 2:39 PM

Or does it only returned cached data, not cached lack of data

johtso•2/23/23, 2:40 PM

So it would always return the correct data when a key is newly created

johtso•2/23/23, 2:40 PM

Just there might be a delay logging someone out

johtso•2/23/23, 2:46 PM

So if I try to read a newly created key from a different colo to the one it was created in, it will return as not existing?

johtso•2/23/23, 2:46 PM

I was optimistic that this is only an issue if there was a previous cached version of that key

johtso•2/23/23, 2:47 PM

Hmm

johtso•2/23/23, 2:48 PM

So this problem would only happen if I check if the user is already logged in from Colo A, log them in on Colo B and then check again on Colo A

johtso•2/23/23, 2:49 PM

Which shouldn't happen as the session key will be randomly generated, so won't have been checked before being written

johtso•2/23/23, 2:52 PM

possibly more of an issue if I'm using the session to store information key to the functioning of the app that changes whenever the user performs an action

johtso•2/23/23, 2:53 PM

then any changing of colo will cause weird stuff to happen

johtso•2/23/23, 2:53 PM

in that case probably better to use a durable object

johtso•2/23/23, 2:54 PM

it would only take them doing something after switching colo, and then the connection switching back for bad things to happen

johtso•2/23/23, 2:56 PM

still a little unnerving to have that possibility though

Louis Deconinck•2/23/23, 10:08 PM

I have a CSV file with 3 million rows. I want to store these one time and then later read the results from this. Is KV well suited for this and how can I get my data in KV?

johtso•2/23/23, 10:17 PM

@louisdck how are you expecting to query the data? just simple key lookups?

johtso•2/23/23, 10:23 PM

is the data static?

Louis Deconinck•2/23/23, 10:26 PM

Yes, it is static data (chess puzzles). I want to query the data through a key lookup. I see there are only 1000 free writes a day. How can I write multiple lines in 1 write opertaion?

Louis Deconinck•2/23/23, 10:28 PM

However, now that I think of it, each chess puzzle is also put in a category. It would be nice if I could also query using these categories. For example, return me 5 random chess puzzles from the endgame category.

Louis Deconinck•2/23/23, 10:29 PM

This is an example row of my csv:
000rO,3R4/8/K7/pB2b3/1p6/1P2k3/3p4/8 w - - 4 58,a6a5 e5c7 a5b4 c7d8,1039,80,85,61,crushing endgame fork short,https://lichess.org/tzeeBEc2#115,

Louis Deconinck•2/23/23, 10:29 PM

The '000rO' is a unique id for the chess puzzle.

johtso•2/23/23, 10:31 PM

@louisdck you can do some quite nice stuff using a sqlite database and static hosting https://github.com/phiresky/sql.js-httpvfs

Is there anything cheaper than `kv` for super simple storage on Workers? Basically, we have a boolea

Similar Threads

Similar Threads

Similar Threads