Not to throw my own engineer under the
Not to throw my own engineer under the bus, but he says " Hyperdrive is just a connection pooler with a very light caching layer. It makes connecting to postgres quick yes, but not a true caching layer like KV." -- it sounds like we have some fundamental misunderstanding of the product difference?
7 Replies
It would really depend on what your requirements are. Some folks find success with Hyperdrive's built in caching. Workers KV provides a more typical key-value API that can be used for caching which can provide low latency with high traffic keys
I guess what is unusual about our use case is that we are building an internal app (we are a hedge fund) -- so our traffic will never be huge. What we have is queries to a database that are expensive and want to cache the results
If that's the case, KV is the right solution. I recommend enabling smart placement if you're trying to use KV more as a database to minimize the latency between your Workers and any data services you're interacting with
Hey 👋 I'm that engineer working with Alex, and to close the loop here and to add more context.
We have a postgres server that has it's own connection pooler located in North Virginia (supabase). We then also have hyperdrive bound to our worker to maintain hot connections on the edge.
The user (me) in NY makes an API call to a Workers REST service, which should be spawned in the closest datacenter. The data is cached with a TTL of 1hr in KV.
On a cache miss we get ~1.2s latency for a full roundtrip, while on a cache hit we see 200-400ms.
There are many 10-20 or so requests sent at once. Now the latency isn't an issue for us at all, we save a ton of time already on cache hits. Just wanted to make sure that we're not doing anything wrong here.
It sounds like you're saying you have a very computationally expensive & long query in your Postgres that takes 1.2s in Postgres, and you are able to place them into KV. If that is the case, then KV may be the better fit here as it is a global cache and you have made the right decision.
KV is global, in the sense that results placed into KV will be accessible to all datacenters. KV internally does caching, so it is possible for your to have KV hot reads and KV cold reads. KV hot reads occur when you have high traffic (+10rps to the same key) and these will be cached in the same region as your Worker to provide potentially single digit ms reads. KV cold reads occur when you have keys that have less traffic (<1rpm, depends on CacheTTL), in which case you can expect >100ms for reads to go to the central stores.
In either case, hot reads and cold reads in Workers KV (which I believe you refer to as a 'hit' since they are resolved by KV, I assume you are using Workers KV key expiration as well to determine if your results are stale and you should fetch from your database) are faster that your database operation and you have the right solution here What do you mean by "There are many 10-20 or so requests sent at once."?
KV is global, in the sense that results placed into KV will be accessible to all datacenters. KV internally does caching, so it is possible for your to have KV hot reads and KV cold reads. KV hot reads occur when you have high traffic (+10rps to the same key) and these will be cached in the same region as your Worker to provide potentially single digit ms reads. KV cold reads occur when you have keys that have less traffic (<1rpm, depends on CacheTTL), in which case you can expect >100ms for reads to go to the central stores.
In either case, hot reads and cold reads in Workers KV (which I believe you refer to as a 'hit' since they are resolved by KV, I assume you are using Workers KV key expiration as well to determine if your results are stale and you should fetch from your database) are faster that your database operation and you have the right solution here What do you mean by "There are many 10-20 or so requests sent at once."?
Got it, so considering that our application will very unlikely reach 10rps volumes of traffic for a single key, we will rarely if ever hit KV's hot reads. At the end of the day the latency we're getting from KV cold reads is already a significant improvement over directly hitting our postgres instance so happy with the tool anyway.
Yes just to clarify when I say hit, I mean we're resolving to KV for a particular key before reaching to the postgres instance.
As for 10-20 or so requests sent at once, I mean hitting different keys (paginated results). Which will still only hit cold reads.
There's no way for us to manually specify a namespace as hot reads only right?
Correct, hot reads are a result of the distribution and scale of your traffic. If your traffic is low and highly distributed, then you will have more cold reads.