Why not just modify dns to CNAME to the worker of your choice?
Why not just modify dns to CNAME to the worker of your choice?
HAssuming that both are on CF, then you can’t just CNAME them
DAmazing- I was writing my own kv methods with the API as I didn’t find another library out there to use kv outside of a worker
Ayes that’s right, you can’t CNAME to another worker afaik
Ai ended up just building a worker to act as a transparent proxy
TThis seems very similar to a URL shortener. Very valid, KV is a good choice
T
KV update: 2024-11-04: We recently updated our REST API docs to reflect that Workers KV analytics REST APIs (https://developers.cloudflare.com/api/operations/workers-kv-request-analytics-query-request-analytics and https://developers.cloudflare.com/api/operations/workers-kv-stored-data-analytics-query-stored-data-analytics) are deprecated and will be shut down on January 31, 2025.
This follows our previous changelog entry of this deprecation and announcement of the Workers KV GraphQL API https://developers.cloudflare.com/kv/platform/changelog/#2024-08-08. The GraphQL API is the way to get analytics for your Workers KV namespaces going forward, since it provides a lot of the granularity and flexibility that you've been asking for such as per-namespace analytics and is unified with the rest of the developer platform products.
If you are using the Workers KV analytics REST APIs (https://developers.cloudflare.com/api/operations/workers-kv-request-analytics-query-request-analytics and https://developers.cloudflare.com/api/operations/workers-kv-stored-data-analytics-query-stored-data-analytics), migrate to the new GraphQL API for Workers KV analytics. Refer to our docs for more information on the GraphQL API and sample queries https://developers.cloudflare.com/kv/observability/metrics-analytics/
You may have received an email recently if we've detected you're still using these KV analytics APIs reminding you of this deprecation. (CC @Workers KV for those who subscribed)
PDoes anyone have a reference code on how to create a full export of a kv namespace? I am trying to write one myself but I am running into a lot of different issues, ranging from invalid cursors to max subrequest errors and/or inconsistent results. The kv I am testing with shows about 8.5k records in the cloudflare ui where I am already running into issues, the different production kv I will need to export will have around 144k entries so I am a bit worried about being able to export it successfully.
My use-case is that we have a kv setup with redirects (key is the original url, target url is the value of the kv record), with data imported overtime from multiple files and we now sadly lack a "master file" with all our data. So to create this masterfile I want to export the current state of the kv into a csv so that the data can be analysed by business users in excel.
My use-case is that we have a kv setup with redirects (key is the original url, target url is the value of the kv record), with data imported overtime from multiple files and we now sadly lack a "master file" with all our data. So to create this masterfile I want to export the current state of the kv into a csv so that the data can be analysed by business users in excel.
MHi everyone, I'm new to KV and have been reading this guide. It talks about storing and serving static assets. My question is: what are the advantages of using KV for this rather than a public R2 bucket?
HWell, this guide looks to be from back when Workers Sites were still a thing. You can use Assets now for free asset serving
HAnother thing too is that per-key RPS can scale a bit better with KV as compared to R2 + built-in caching
TLike HardlyWorkin' said, there are a lot of better options than a public R2 bucket for serving static assets. Pages (https://developers.cloudflare.com/pages/) and Workers assets (https://developers.cloudflare.com/workers/static-assets/) provide hosting built for serving static sites. Using R2 and KV, while possible, would be sort of rebuilding what Pages & Workers assets do (can be valid in some cases!)
HActually, would be pretty cool to have Assets+, that could fallback to R2 instead of KV for large assets
HWhat is the maximum limit of kv namespace for enterprise edition?
FThese are the official limits -> https://developers.cloudflare.com/kv/platform/limits/
Otherwise talk to your enterprise support contact you should have when you are on an enterprise plan.
Otherwise talk to your enterprise support contact you should have when you are on an enterprise plan.
MThanks, both.
KError: KV PUT failed: 429 Too Many Requests
what is the rate limit on put?
what is the rate limit on put?
WW1 write per key per second
Kanother quick question, will put update remove the expiration if I don’t include it?
kv.put(key, value, {expiration: x}) // Create the KV pairs
kv.put(key,value2) // Update the value only
kv.put(key, value, {expiration: x}) // Create the KV pairs
kv.put(key,value2) // Update the value only
KIn this case, is the expiration updated?
CIs anyone else seeing elevated error rates on KV puts over the last week or so?
TPut won't remove the expiration if you don't include it, but it will overwrite it if you provide a new expiration
TCan you share you namespace id?
TAre you using the REST API? That has an account wide rate limit, and we’d recommend catching your operations using the bulk operations endpoints of the REST API
PContinuing from https://ptb.discord.com/channels/595317990191398933/779390076219686943/1306685718621323308
It appears my reported read latencies are higher than they're supposed to be. Most of the reads are hot, but still my p90 is about 230ms, which is much higher than I would expect.
Namespace ID:
cc @thomasgauvin
It appears my reported read latencies are higher than they're supposed to be. Most of the reads are hot, but still my p90 is about 230ms, which is much higher than I would expect.
Namespace ID:
21fd681cd0074337b08b56b98d1bd07ccc @thomasgauvin
P(I personally don't actively work on this specific project that uses this namespace, but I can grab any info if needed)
TWhat’s happening here is that they are regionally cached but not locally cached, so they get tagged as cache but I would hypothesize that your traffic is still pretty spread out and doesn’t regularly have >1 RPS to the same key within a Cloudflare region
TIf this is the case, you’re probably getting Cloudflare-wide API rate limits. https://developers.cloudflare.com/fundamentals/api/reference/limits/#:~:text=The%20global%20rate%20limit%20for,receiving%20a%20HTTP%20429%20response.
If you have an enterprise account, you could contact your account team to increase these
If you have an enterprise account, you could contact your account team to increase these
Cloudflare Docs
The global rate limit for the Cloudflare API is 1200 requests per five minutes per user, and applies cumulatively regardless of whether the request is made via the dashboard, API key, or API token.
THow are you using KV within your app/can you share more context about the use case? Trying to understand what’s the right solution here
Tevery account has account-wide rate limits of 1200 requests per 5 minutes. That applies to using Workers KV via the REST API (which is what the
We know that's limiting and we have ongoing efforts to revisit how rate limiting is applied, but this is months away (requires per product instance rate limiting).
I strongly recommend using the Workers KV binding that should be using from within your Workers https://developers.cloudflare.com/kv/api/read-key-value-pairs/. Is there a reason why you aren't using the binding currently?
cloudflareWe know that's limiting and we have ongoing efforts to revisit how rate limiting is applied, but this is months away (requires per product instance rate limiting).
I strongly recommend using the Workers KV binding that should be using from within your Workers https://developers.cloudflare.com/kv/api/read-key-value-pairs/. Is there a reason why you aren't using the binding currently?
T KV update: 2024-11-14: We updated our REST API docs to add granular errors to bulk operations endpoints for writes and deletes. https://developers.cloudflare.com/kv/platform/changelog/#2024-11-14
The response body of those endpoints now contain
unsuccessful_keyssuccessful_key_countresultThis gives you a way to know which of the KV pairs have been properly written/deleted in a bulk operation and enables you to retry only the unsuccessful keys in the event of an error.
(CC @Workers KV for those who subscribed)
Pah I see, makes sense. Thanks for the clarification!
C
FPlease do not post your question in multiple channels/post it multiple times per the rules at #
welcome-and-rules. It creates confusion for people trying to help you and doesn't get your issue or question solved any faster.
welcome-and-rules. It creates confusion for people trying to help you and doesn't get your issue or question solved any faster.
THi @xiaoyu2er, that comparison is outdated. Workers KV is the fastest way to retrieve data from a Worker, especially for high read throughput.
Workers KV makes use of caching across the Cloudflare network internally to make sure that highly requested items are very fast. That means that it's ideal for configuration data, non-relational data, A/B tests, user preferences, that requires very high read throughput (high requests per second) and that have infrequent reads. The caching means that your data is effectively global.
You'd use D1 for your general application data. It can handle lower throughput but provides consistency, which you may need if you want to immediately see writes. This is why it's recommended for general application data.
The big part that is missing from the benchmark above, in addition to it being outdated, is regionality. KV is replicated globally, whereas D1 is currently single-region.
Workers KV makes use of caching across the Cloudflare network internally to make sure that highly requested items are very fast. That means that it's ideal for configuration data, non-relational data, A/B tests, user preferences, that requires very high read throughput (high requests per second) and that have infrequent reads. The caching means that your data is effectively global.
You'd use D1 for your general application data. It can handle lower throughput but provides consistency, which you may need if you want to immediately see writes. This is why it's recommended for general application data.
The big part that is missing from the benchmark above, in addition to it being outdated, is regionality. KV is replicated globally, whereas D1 is currently single-region.
JHi - Is there a difference in the below two calls? I'm getting two different results for
const value = await env.NAMESPACE.list({ prefix: "testing" });
vs
a call to
It seems that the default should be 1000 if not provided. My storage has around 200-300 items with the prefix but the first one would return an arbitrary number at times(e.g. 50, 93) while the rest call would always return all of them. I wonder if the first case could be suffering from
const value = await env.NAMESPACE.list({ prefix: "testing" });
vs
a call to
https://api.cloudflare.com/client/v4
/accounts/{account_id}/storage/kv/namespaces/{namespace_id}/keys/prefix=testingIt seems that the default should be 1000 if not provided. My storage has around 200-300 items with the prefix but the first one would return an arbitrary number at times(e.g. 50, 93) while the rest call would always return all of them. I wonder if the first case could be suffering from
It is possible to have an empty array in keys, but still have more keys to fetch, because recently expired or deleted keys ↗ must be iterated through but will not be included in the returned keys. where the deleted ones are counted as part of the limit but the second one is not?THi folks, we added a section to the
Choose a data or storage product
KHi folks, I have a question, I came from traditional Node.js Express REST apis that caches with mysql database data with Redis, so I don't know if I understand KV well, but it's like Redis and is good to use it to cache database list results for performance improvement, or is D1 enough for this purpose? Or the purpose of KV is cache the entire response from a Worker?
1I know it isn't many reads, but man when KV is doing hot only reads it is fast
QHlo
Jhey there - has the behavior with KV changed recently? we are getting KV hits for deleted key-pairs
some context:
some context:
- the key has a cacheTTL of 24h
- we delete the key through the REST API
- the key is not in the KV if i prefix search the key on the dashbaord
- a console.log() after the read in the worker still finds the old row
Khow can I delete a key ? I just saw in the doc the method put and get
Pi have 2 questions, full disclosure: i don't understand back end infrastructure too well
1# is kv as secure as d1? like can i store emails as keys on it? is that a good idea?? or will ANYONE be able to retrieve that info? the docs seem to imply it's fairly secure i think
2# how quick are updates if i write into kv? will that happen under 30 mins and be reflected everywhere?
if info about my use case is needed, i want to store a list of digital item's urls or ids (which i will fetch with ajax on client side)
stored as "email (who purchased) : json (nested list of items purchased)"
originally i thought user sign in sign up would be neat but i don't want the responsibility of storing passwords or anything for something as simple as "hey did the user purchase this product fully?", specially because payments will be handled face to face, not digitally and i will have to input emails and purchase list by hand anyway. i think getting email from google sign in or something then fetching the key : value pair seems reasonable and simple to me
1# is kv as secure as d1? like can i store emails as keys on it? is that a good idea?? or will ANYONE be able to retrieve that info? the docs seem to imply it's fairly secure i think
2# how quick are updates if i write into kv? will that happen under 30 mins and be reflected everywhere?
if info about my use case is needed, i want to store a list of digital item's urls or ids (which i will fetch with ajax on client side)
stored as "email (who purchased) : json (nested list of items purchased)"
originally i thought user sign in sign up would be neat but i don't want the responsibility of storing passwords or anything for something as simple as "hey did the user purchase this product fully?", specially because payments will be handled face to face, not digitally and i will have to input emails and purchase list by hand anyway. i think getting email from google sign in or something then fetching the key : value pair seems reasonable and simple to me
W- No one can ever read another persons resources without explicitly being allowed by that person
- It takes up to 60 seconds for new values to be reflected
Pthanks!
IBy #1 the caveat is that KV values can be viewed on the dashboard, but as Walshy said, they'd need access to your Cloudflare account to do that.
Pas long as it's not publicly available like static websites, that's all good yes thanks
1DNS over Discord: GitHub
View the DNS over Discord source code on GitHub at https://dns-over-discord.v4.wtf/github
