`cloudflare does not support bulk writes from within a worker` but you can do bulk put with cli or

cloudflare does not support bulk writes from within a worker

cloudflare does not support bulk writes from within a worker

but you can do bulk put with cli or API:

You can write more than one key-value pair at a time with Wrangler or via the API
. The bulk API can accept up to 10,000 KV pairs at once

You can write more than one key-value pair at a time with Wrangler or via the API
. The bulk API can accept up to 10,000 KV pairs at once

so whats stopping me from putting my cf api key into worker env variable and and using api inside the worker? like this:

curl --request PUT \
  --url https://api.cloudflare.com/client/v4/accounts/account_identifier/storage/kv/namespaces/namespace_identifier/bulk \
  --header 'Authorization: Bearer undefined' \
  --header 'Content-Type: application/json' \
  --data '[
  {
    "base64": false,
    "expiration": 1578435000,
    "expiration_ttl": 300,
    "key": "My-Key",
    "metadata": {
      "someMetadataKey": "someMetadataValue"
    },
    "value": "Some string"
  },
...//more objects for bulk put
]'

curl --request PUT \
  --url https://api.cloudflare.com/client/v4/accounts/account_identifier/storage/kv/namespaces/namespace_identifier/bulk \
  --header 'Authorization: Bearer undefined' \
  --header 'Content-Type: application/json' \
  --data '[
  {
    "base64": false,
    "expiration": 1578435000,
    "expiration_ttl": 300,
    "key": "My-Key",
    "metadata": {
      "someMetadataKey": "someMetadataValue"
    },
    "value": "Some string"
  },
...//more objects for bulk put
]'

curl --request PUT \
  --url https://api.cloudflare.com/client/v4/accounts/account_identifier/storage/kv/namespaces/namespace_identifier/bulk \
  --header 'Authorization: Bearer undefined' \
  --header 'Content-Type: application/json' \
  --data '[
  {
    "base64": false,
    "expiration": 1578435000,
    "expiration_ttl": 300,
    "key": "My-Key",
    "metadata": {
      "someMetadataKey": "someMetadataValue"
    },
    "value": "Some string"
  },
...//more objects for bulk put
]'

curl --request PUT \
  --url https://api.cloudflare.com/client/v4/accounts/account_identifier/storage/kv/namespaces/namespace_identifier/bulk \
  --header 'Authorization: Bearer undefined' \
  --header 'Content-Type: application/json' \
  --data '[
  {
    "base64": false,
    "expiration": 1578435000,
    "expiration_ttl": 300,
    "key": "My-Key",
    "metadata": {
      "someMetadataKey": "someMetadataValue"
    },
    "value": "Some string"
  },
...//more objects for bulk put
]'

will i get banned if i do this?

styOP•8/10/23, 7:43 AM

does a single bulk put count towards 1 daily put request out of 1000 total alloted per day? or does bulk put still count all the objects in the data ?
like if the bulk put has 400 objects, does it count towards 1 req out of 1000? or 400/1000 ?

styOP•8/10/23, 7:45 AM

and last question:

Writes to different keys    1,000 writes per day

Writes to different keys    1,000 writes per day

the kv limit says 1000 writes per dat to

different

different

keys
so that means the limit doesn't apply to the same key?

Ssty `cloudflare does not support bulk writes from within a worker` but you can do b...

DaniFoldi•8/10/23, 7:45 AM

there's nothing stopping you, however note that API operations are limited to 1200/5 minutes, globally per key

Ssty and last question: `Writes to different keys 1,000 writes per day` the kv li...

DaniFoldi•8/10/23, 7:46 AM

it counts as 1 request for the worker, but 400 writes to KV, out of the 1000 per day you get in the free plan

DDaniFoldi it counts as 1 request for the worker, but 400 writes to KV, out of the 1000 per...

styOP•8/10/23, 7:46 AM

ok that makes sense. thanks for the clarification.
do you know if the 1000 per day applies if you are updating the same key over and over?

Ssty ok that makes sense. thanks for the clarification. do you know if the 1000 per d...

DaniFoldi•8/10/23, 7:47 AM

I haven't tested this myself, but it would make sense to me if it did, you get 1000 writes per day, same key or not

styOP•8/10/23, 7:49 AM

i will test it out.
Because limit page https://developers.cloudflare.com/workers/platform/limits/
clearly says

Writes to different keys    1,000 writes per day

Writes to different keys    1,000 writes per day

so if true, then i can basically write dynamic data to the single key over and over without having to worry about free tier rate limits as a hack sort of. do you think its worth it if true?

Limits · Cloudflare Workers docs

You can request adjustments to limits that conflict with your project goals by contacting Cloudflare. To request an increase to a limit, complete the …

UUnsmart The KV dash really needs some love 😅 Its extremely slow to load because it tr...

jax•8/10/23, 2:20 PM

That's why I made the extension btw.. it supports bulk delete

and doesn't rerun the list operation each time. and is metadata-first.

jax•8/10/23, 2:29 PM

also you can search/filter by metadata.

Unsmart•8/10/23, 2:55 PM

Can you tell us what you are getting?

Unsmart•8/10/23, 2:59 PM

Are all operations from the worker failing for you or just some?

DDeclan PUT ops are making it, in the background. But the GET ops error out, so it seems...

Matt Silverlock•8/10/23, 3:05 PM

Did you originally opt into the KV 2.0 beta? Can you share an account ID?

DDeclan @Matt yep. I have DM'ed my a/c id

Matt Silverlock•8/10/23, 3:08 PM

Got it, thanks. We're going to disable KV 2.0 for testers for the time being.

Kai•8/10/23, 4:06 PM

when sending a batched KV write request, what is the pricing? one write / key or one write / requests?

KKai when sending a batched KV write request, what is the pricing? one write / key or...

DaniFoldi•8/10/23, 4:08 PM

each entry in the batch counts as a write

Kai•8/10/23, 4:08 PM

oh well sad

leostera•8/10/23, 8:36 PM

how come wrangler doesn't let me create kv namespaces with the same restrictions as the web ui? I created most of my ns on the dashboard so they have dashes

leostera•8/10/23, 8:45 PM

does listing by prefix force replication of bucket data or does it work only with bucket-local data?

leostera•8/10/23, 8:58 PM

what's the easiest way to clean up my kv stores?

short of manually listing/bulk-deleting all the keys in each of them

James•8/10/23, 9:03 PM

if you don't care about the ID changing, you could just delete the namespace and recreate it

James•8/10/23, 9:03 PM

otherwise unfortunately you'll have to bulk list/delete things

leostera•8/10/23, 9:03 PM

yeah, i started doing that but i have many kvs and i think i'm gonna need to wipe 'em regularly until I nail the data design

leostera•8/10/23, 9:04 PM

cool, no worries, thanks!

leostera•8/10/23, 10:07 PM

i made a few namespaces in the console, named: pachadb-{kind}-storepachadb-{kind}-store for several things

but wranlger kv:namespace createwranlger kv:namespace create only lets me do

s and it'll prefix things with worker-worker-.

So calling wrangler kv:namespace create pachadb_{kind}_storewrangler kv:namespace create pachadb_{kind}_store got me kv ns named worker-pachadb_{kind}_storeworker-pachadb_{kind}_store

CCharlie Hey @leostera we should probably make that experience more consistent. What ar...

Matt Silverlock•8/11/23, 12:35 AM

https://github.com/cloudflare/workers-sdk/blob/9d8509e08acf082604ca896b4ab9ad5c05ae7505/packages/wrangler/src/kv/index.ts#L61-L68

GitHub

workers-sdk/packages/wrangler/src/kv/index.ts at 9d8509e08acf082604...

Home to Wrangler, the CLI for Cloudflare Workers® - cloudflare/workers-sdk

chientrm.com•8/11/23, 12:40 PM

my image stored in KVKV as base64 encoded string, how can I leverage the browser to treat them as normal images so that I can make the cache happen?

If I embed the base64 string in my rest response, it won't be cache.

chientrm.com•8/11/23, 1:01 PM

Done

export const GET = (async ({ params, locals }) => {
  const { id } = params,
    value = await locals.KV.get(memeTemplateFileKey(id)),
    buffer = Buffer.from(value!.split(',')[1], 'base64');
  return new Response(buffer, { headers: { 'Content-Type': 'image/png' } });
}) satisfies RequestHandler;

export const GET = (async ({ params, locals }) => {
  const { id } = params,
    value = await locals.KV.get(memeTemplateFileKey(id)),
    buffer = Buffer.from(value!.split(',')[1], 'base64');
  return new Response(buffer, { headers: { 'Content-Type': 'image/png' } });
}) satisfies RequestHandler;

leostera•8/11/23, 3:26 PM

@chientrm can you store the raw binary image on r2 instead? that + a public domain should get you all the caching you need on the browsers end

andrewp•8/11/23, 3:50 PM

@chientrm have you tried setting cache directives? https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control

Cache-Control - HTTP | MDN

The Cache-Control HTTP header field holds directives (instructions) — in both requests and responses — that control caching in browsers and shared caches (e.g. Proxies, CDNs).