Writes are replicated when you write and its just a last writer wins and if there happens to be a mi

Writes are replicated when you write and its just a last writer wins and if there happens to be a mismatch between the data sometimes the read can push through a replication fix where it chooses one. But you are also only supposed to write once per second to a specific key so if your use case involves higher write throughput than that you probably should not be using KV.

TigersWay•8/6/23, 11:32 PM

Could someone confirm that the dashboard numbers (operations) are for the last 24 hours but the limits are counted from 0:00 UTC?
Thanks

TigersWay•8/6/23, 11:38 PM

And if true, do we have anyway to know what have been used during the last period(s) / 0:00 - 23:59 UTC?

Murder Chicken•8/7/23, 1:19 PM

Is there any way to bulk fetch KV values in a single request from a worker?

TTigersWay And if true, do we have anyway to know what have been used during the last perio...

Cyb3r-Jak3•8/7/23, 1:34 PM

You should be able to use the GraphQL API to get that data

UUnsmart Writes are replicated when you write and its just a last writer wins and if ther...

leostera•8/7/23, 5:38 PM

i'm actually only writing once per key, but every now and then I'll need to write N things and a summary of what those N things are, like this:

entries = [
  {id: `entry:1`, body: "..."},
  {id: `entry:2`, body: "..."},
  {id: `entry:3`, body: "..."},
]
summary = { id: `summary:1`, entries: [`entry:1`, `entry:2`, `entry:3`] }

entries = [
  {id: `entry:1`, body: "..."},
  {id: `entry:2`, body: "..."},
  {id: `entry:3`, body: "..."},
]
summary = { id: `summary:1`, entries: [`entry:1`, `entry:2`, `entry:3`] }

leostera•8/7/23, 5:40 PM

what I still need to try is if the following read pattern will force replication of all the entries in the summary:

let summary = await fetchSummary(`summary-1`);
for entry in summary.entries {
  let entry = await fetchEntry(entry);
}

let summary = await fetchSummary(`summary-1`);
for entry in summary.entries {
  let entry = await fetchEntry(entry);
}

leostera•8/7/23, 5:47 PM

In the docs I also saw this:

One pattern is to send all of your writes for a given KV key through a corresponding instance of a Durable Object

but i'm not sure how this would look like in code?

leostera•8/7/23, 6:00 PM

yes

leostera•8/7/23, 6:00 PM

immutable as well

leostera•8/7/23, 6:00 PM

so there may be subsequent rquests to

summary:1

summary:1

but it will never change once its written

leostera•8/7/23, 6:00 PM

(same for

entry:1

entry:1

, 2, 3, etc)

leostera•8/7/23, 6:01 PM

cool!

leostera•8/7/23, 6:01 PM

will give this a shot soon

leostera•8/7/23, 6:01 PM

thanks @skye_31!

jax•8/7/23, 10:47 PM

I made a VSCode extension to interface with KV right from VSCode that y'all might like. Has some benefits over the web UI like the a single uniform interface for Local, Preview, and Production environments, and the ability to view and edit metadata and expiration times. Anyways, it's free. https://marketplace.visualstudio.com/items?itemName=jaaxxx.cloudflare-vscode-devtools

Cloudflare KV (unofficial) - Visual Studio Marketplace

Extension for Visual Studio Code - Interface with Cloudflare products (for now just KV, more to come?)

kian•8/8/23, 7:54 PM

R2 has https://developers.cloudflare.com/r2/api/s3/api/

S3 API compatibility · Cloudflare R2 docs

R2 implements the S3 API to allow users and their applications to migrate with ease. When comparing to AWS S3, Cloudflare has removed some API …

Iizdi Are you querying KV through another Cloudflare Worker? I wonder if there is a ...

jax•8/8/23, 10:14 PM

No it's shelling out to the CLI. The latest versions of wrangler allow the

--local

--local

flag.

jax•8/8/23, 10:14 PM

You can always access KV via the HTTP API, that's what wrangler does.

kewbish•8/8/23, 10:15 PM

^ here's the endpoint for reference

UnsmartOP•8/9/23, 4:28 PM

The KV dash really needs some love

Its extremely slow to load because it tries to fetch all the values on a page and theres no batch get (I think showing metadata and giving an option to view value is better tbh). It also never shows all the values because a few of the keys always error out

Also really annoying when you want to try and delete some of the values because it reloads the view (but doesnt even get the next page its just the same values minues the deleted one). Would also be really nice if we could select multiple keys that we want to delete in one go.

ajgeiss0702•8/9/23, 5:51 PM

it appears that metadata is already sent to the browser, its just not displayed

UnsmartOP•8/9/23, 5:54 PM

I'm aware... im saying it should display the metadata instead of running 30 gets that are slow to return

ajgeiss0702•8/9/23, 5:54 PM

i agree. i was actually coming here to say basically the same thing, but saw you had already said it
apologies if it came out the wrong way

sty•8/10/23, 7:39 AM

cloudflare does not support bulk writes from within a worker

cloudflare does not support bulk writes from within a worker

but you can do bulk put with cli or API:

You can write more than one key-value pair at a time with Wrangler or via the API
. The bulk API can accept up to 10,000 KV pairs at once

You can write more than one key-value pair at a time with Wrangler or via the API
. The bulk API can accept up to 10,000 KV pairs at once

so whats stopping me from putting my cf api key into worker env variable and and using api inside the worker? like this:

curl --request PUT \
  --url https://api.cloudflare.com/client/v4/accounts/account_identifier/storage/kv/namespaces/namespace_identifier/bulk \
  --header 'Authorization: Bearer undefined' \
  --header 'Content-Type: application/json' \
  --data '[
  {
    "base64": false,
    "expiration": 1578435000,
    "expiration_ttl": 300,
    "key": "My-Key",
    "metadata": {
      "someMetadataKey": "someMetadataValue"
    },
    "value": "Some string"
  },
...//more objects for bulk put
]'

curl --request PUT \
  --url https://api.cloudflare.com/client/v4/accounts/account_identifier/storage/kv/namespaces/namespace_identifier/bulk \
  --header 'Authorization: Bearer undefined' \
  --header 'Content-Type: application/json' \
  --data '[
  {
    "base64": false,
    "expiration": 1578435000,
    "expiration_ttl": 300,
    "key": "My-Key",
    "metadata": {
      "someMetadataKey": "someMetadataValue"
    },
    "value": "Some string"
  },
...//more objects for bulk put
]'

will i get banned if i do this?

sty•8/10/23, 7:43 AM

does a single bulk put count towards 1 daily put request out of 1000 total alloted per day? or does bulk put still count all the objects in the data ?
like if the bulk put has 400 objects, does it count towards 1 req out of 1000? or 400/1000 ?

sty•8/10/23, 7:45 AM

and last question:

Writes to different keys    1,000 writes per day

Writes to different keys    1,000 writes per day

the kv limit says 1000 writes per dat to

different

different

keys
so that means the limit doesn't apply to the same key?

Ssty `cloudflare does not support bulk writes from within a worker` but you can do b...

DaniFoldi•8/10/23, 7:45 AM

there's nothing stopping you, however note that API operations are limited to 1200/5 minutes, globally per key

Ssty and last question: `Writes to different keys 1,000 writes per day` the kv li...

DaniFoldi•8/10/23, 7:46 AM

it counts as 1 request for the worker, but 400 writes to KV, out of the 1000 per day you get in the free plan

DDaniFoldi it counts as 1 request for the worker, but 400 writes to KV, out of the 1000 per...

sty•8/10/23, 7:46 AM

ok that makes sense. thanks for the clarification.
do you know if the 1000 per day applies if you are updating the same key over and over?

Ssty ok that makes sense. thanks for the clarification. do you know if the 1000 per d...

DaniFoldi•8/10/23, 7:47 AM

I haven't tested this myself, but it would make sense to me if it did, you get 1000 writes per day, same key or not

sty•8/10/23, 7:49 AM

i will test it out.
Because limit page https://developers.cloudflare.com/workers/platform/limits/
clearly says

Writes to different keys    1,000 writes per day

Writes to different keys    1,000 writes per day

so if true, then i can basically write dynamic data to the single key over and over without having to worry about free tier rate limits as a hack sort of. do you think its worth it if true?

Limits · Cloudflare Workers docs

You can request adjustments to limits that conflict with your project goals by contacting Cloudflare. To request an increase to a limit, complete the …

UUnsmart The KV dash really needs some love 😅 Its extremely slow to load because it tr...

jax•8/10/23, 2:20 PM

That's why I made the extension btw.. it supports bulk delete

and doesn't rerun the list operation each time. and is metadata-first.

jax•8/10/23, 2:29 PM

also you can search/filter by metadata.

UnsmartOP•8/10/23, 2:55 PM

Can you tell us what you are getting?

UnsmartOP•8/10/23, 2:59 PM

Are all operations from the worker failing for you or just some?

DDeclan PUT ops are making it, in the background. But the GET ops error out, so it seems...

Matt Silverlock•8/10/23, 3:05 PM

Did you originally opt into the KV 2.0 beta? Can you share an account ID?

curl --request PUT \ --url https://api.cloudflare.com/client/v4/accounts/account_identifier/storage/kv/namespaces/namespace_identifier/bulk \ --header 'Authorization: Bearer undefined' \ --header 'Content-Type: application/json' \ --data '[ { "base64": false, "expiration": 1578435000, "expiration_ttl": 300, "key": "My-Key", "metadata": { "someMetadataKey": "someMetadataValue" }, "value": "Some string" }, ...//more objects for bulk put ]'

Writes are replicated when you write and its just a last writer wins and if there happens to be a mi

Similar Threads