it looks like it might be fixed now? its been showing consistently the past few minutes
it looks like it might be fixed now? its been showing consistently the past few minutes
CHey there, thanks for the update on this. Are you still seeing the issue with the KV keys that you were initially reporting?
Ait looks like those keys I mentioned are still sometimes not showing up in the dashboard.
but I have a cron task that collects those keys into a single one, and that appears to have read them correctly this time so its no longer interrupting my application
but I have a cron task that collects those keys into a single one, and that appears to have read them correctly this time so its no longer interrupting my application
Aalso i realized I forgot to mention this before, but those keys also have an expiration set (15 days)
AIf it helps, here is what this kv namespace is for:
each of those keys with colons are "fragments" of an event. they are fragmented because each piece of info is available at different times and I need to record them. that cron then combines them into a single key after the event for easier reading, so those fragments get an expiration ttl of 15 days (enough so that I can diagnose things if something goes wrong with the collapsing, and small enough so that they don't stick around forever wasting storage)
each of those keys with colons are "fragments" of an event. they are fragmented because each piece of info is available at different times and I need to record them. that cron then combines them into a single key after the event for easier reading, so those fragments get an expiration ttl of 15 days (enough so that I can diagnose things if something goes wrong with the collapsing, and small enough so that they don't stick around forever wasting storage)
Athis part of the application was made before D1 existed, otherwise I probably would have used D1. Once D1 read replicas are launched, i might consider migrating this specific namespace to d1 instead, since that would probably better fit this usecase
Aeven though its no longer effecting my application, lmk if theres any other info I can provide if you want to look into preventing this from happening in the future
TWe're looking into how to resolve this, it seems like some routing was changed up which is making it take more time for you to see updates in the dashboard
TI am having issue with KV.
I query a record from D1 Database, it took me 15ms, then I cache that record as json in KV.
I read that record from KV with record id, so I expected it should be faster than D1 but it actually took me 500ms, way slower.
Am I doing it wrong?
I query a record from D1 Database, it took me 15ms, then I cache that record as json in KV.
I read that record from KV with record id, so I expected it should be faster than D1 but it actually took me 500ms, way slower.
Am I doing it wrong?
T
TI finally found a bug that continuously write and invalidate the cache. MY BADDDDDD.
CRead status to show this many Not Found results?Depends entirely on how you're calling KV. If you're using it to serve static assets and people are requesting non-existent paths, not too surprising. If you have a manifest or something in front, making it so every KV get request should always exist, could be a bug in your code.
Your p50 is really high there, how are you using KV? Could you increase cacheTTL/hit the same keys more?
TKV is a global key-value store. If there's not heavy traffic on your keys, then they won't be getting cached in all of the Cloudflare locations. When this is the case, then it needs to fetch the result from the central stores for key-value data. See https://developers.cloudflare.com/kv/concepts/how-kv-works/ for an explanation on this
Cloudflare Docs
KV is a global, low-latency, key-value data store. It stores data in a small number of centralized data centers, then caches that data in Cloudflare's data centers after access.
SCan we store complex JSON in KV as a value? I have a json that’s 4,5 levels deep and need to know if that’s possible with KV.
IAt its core, KV stores bytes so any value (including JSON) that can be serialized/deserialized as bytes can be stored in KV.
INote that in contrast to JSON-oriented databases, KV just stores the bytes. You can't query on nested JSON fields or anything because KV has no idea it's JSON, it's just storing the content.
SThat’s good to know, do you have an example showing json value into KV? How do I translate the object into a Value?
SThat’s not an issue. I’m planning to use the KV as a cache to avoid hitting network excessively
Oif ur JSON is small enough (and since you're working with JSON) you can/should probably also insert it into the metadata https://developers.cloudflare.com/kv/api/write-key-value-pairs/#metadata
https://developers.cloudflare.com/kv/api/list-keys/#optimizing-storage-with-metadata-for-list-operations
https://developers.cloudflare.com/kv/api/list-keys/#optimizing-storage-with-metadata-for-list-operations
IThe
put accepts any string, ReadableStream, or ArrayBuffer so you can just JSON.stringify it: https://developers.cloudflare.com/kv/api/write-key-value-pairs/SIt’s Pirate weather data: https://docs.pirateweather.net/en/latest/API/#example
I hope there’s a way to cache it?
I hope there’s a way to cache it?
SI’ll try this one out tomorrow. Serialising/ Deserialising from ReadableStream should be fine
TYou should probably store it as a string and then read it with json. Check out the new KV examples, this is one where a list of translations are stored https://developers.cloudflare.com/kv/examples/workers-kv-to-serve-assets/#:~:text=Default%20to%20English-,const,-translations%20%3A%20%7B (could have more depth there, as much as you want)
Cloudflare Docs
Example of how to use Workers KV to store static assets
STrue, If there is no performance penalty,
json.Stringify is what I'll go with . Thanks for the example, they are helpful.SIt worked and is pretty fast. I have 2 follow-up questions:
1: I uploaded a json to KV under
1: I uploaded a json to KV under
_preview KV dashboard. How do I get it available locally? I run npx wrangler dev —remote. - Is there a way to clean the stale KV values, using some cron type job?
TThere are going to be 3 'envs' for KV. 1 is prod, 1 is preview, 1 is local. the prod is when your worker is deployed. the preview is in cloudflare when you are running
npx wrangler dev --remote. the local is when your worker is running only locally npx wrangler dev. each is independent and you would need to write data in any of these dev modes to get data in thereTyou could technically make the preview id and the production id the same for development as long as you are aware that you are using your production env with
npx wrangler dev --remote, this is quite dangerous which is why the preview namespace is the defaultTFor stale kv values, you should look at https://developers.cloudflare.com/kv/api/write-key-value-pairs/#expiring-keys or https://developers.cloudflare.com/workers/configuration/cron-triggers/ for cron jobs
Cloudflare Docs
Enable your Worker to be executed on a schedule.
Cloudflare Docs
To create a new key-value pair, or to update the value for a particular key, call the put() method of the KV binding on any KV namespace you have bound to your Worker code:
MIs there a way to purge a complete namespace?
I periodically refresh my D1 database (full import) and in that use case I want to purge the full namespace that I use as cache before D1
I periodically refresh my D1 database (full import) and in that use case I want to purge the full namespace that I use as cache before D1
PI'm new to the Cloudflare environment. I would like to know how can I export all the KV data from an account to a file. We need to do some migration at work.
RHi, I'm having an issue deleting keys from the dashboard. Getting "remove key: 'Endpoint is unknown.'" Is this a known issue?
Mhas anyone tried using KV with better-auth's secondaryStorage? it seems pretty straightforward
just wondering if there are any gotchas or drawbacks I should be wary of?
just wondering if there are any gotchas or drawbacks I should be wary of?
Shello here, I can't find correct information online, is the list() function performant, or should I implement some sort of indexes inside an object instead of relying on list() ?
Susecase can be simplified as, I have a 1 customer with 3 orders, I want to get the 3 orders: should I do
list(customerId) and then 1 getget(customerId-index) and then the 1 get eachSI believe that is important question for performances and there should be some hint in the documentation for that
SGood morning! nobody here? 
IList is performant, yes.
Sthanks @Isaac McFadyen
IAlso probably good to keep in mind that List can see metadata so if your data is small enough it might be worth putting into metadata so you don't need a Get for each one: https://developers.cloudflare.com/kv/api/list-keys/#optimizing-storage-with-metadata-for-list-operations
Syes read that but I read a blog post (old) that was stating GET was around 20-30ms and list 200ms (x10) so I got scared
IHmmm possible it's not cached actually (get is cached). I assumed it was but maybe not?
IDocs don't say... if anyone else can confirm/deny then it might be a good docs addition.
Sthat'd be awesome to have real value here, and answer from CF yes!
LThat blog post is fairly old. Cold read latency for lists is lower than 10x GET latency at P99, but absolute latency values for you depend on your location as cold reads come from central stores, and also depend on how many keys you requested.
IIs List cached or no?