Is there a relationship between Hyperdrive cache settings and the use of prepared statements? I am

Is there a relationship between Hyperdrive cache settings and the use of prepared statements?

I am exploring Postgres through Rust (using the

tokio-postgres

tokio-postgres

crate) using the

query

query

method. This uses the extended query protocol. I am observing the following behavior. When the Hyperdrive cache is:

Disabled: All (or most) queries fail (the driver reports an
```
UnexpectedMessage
```
```
UnexpectedMessage
```
error). I suspect disabled caching makes Hyperdrive use one connection for preparation and another execution on another. Occasionally, it might use the same connection, leading to a successful query.
Enabled: All queries succeed. I suspect caching makes Hyperdrive choose the same connection it used to prepare the statement.

Am I interpreting this correctly? Is this the expected behavior?

p.s. This seems to have changed recently. Earlier,

query

query

would always fail regardless of the cache setting. So, I started working on a PR for the Postgres driver to make it work with Hyperdrive. But today, during testing that PR, I saw this behavior, thus this question.

Client in tokio_postgres - Rust

An asynchronous PostgreSQL client.

moody 💭•6/4/24, 5:16 AM

is something going on with cloudflare?

moody 💭•6/4/24, 5:17 AM

as of 2 hours ago my workers are performing terribly, same number of requests

Mitya•6/4/24, 10:01 AM

Two questions for the CF Hyperdrive team:

Given HD's caching capabilities, is it redundant to also utilise read replicas on my DB, or would they still be a good idea?
Six months or so ago I and others noticed that HD had a bug whereby, if caching was enabled, sometimes queries returned erroneous results e.g. no rows where there should definitely be rows. I recall vaguely that a CF dev said this was acknowledged and a fix was scheduled. Did this happen, i.e. is it safe for me to re-enable caching? (I disabled it at the time due to this).
Thanks in advance.

moody 💭•6/4/24, 3:11 PM

i seem to be having serious performance issues with hyperdrive connected to a postgres db in hetzner.
why is latency like this? other graphs are relatively flat, but errors and latency are spiking and users are reporting slowness.

Screenshot_2024-06-04_at_11.10.30_AM.png

Screenshot_2024-06-04_at_11.10.39_AM.png

Paul•6/4/24, 3:25 PM

Can we get some verbiage on the Supabase + Hyperdrive setup page in regards to the Supabase pooler? Do we use the pooler connection string or the direct connection string from the Supabase Database settings page?
https://developers.cloudflare.com/hyperdrive/examples/supabase/

Cloudflare Docs

Connect to Supabase · Hyperdrive

This example shows you how to connect Hyperdrive to a Supabase Postgres database.

Mmoody 💭i seem to be having serious performance issues with hyperdrive connected to a po...

moody 💭•6/4/24, 3:36 PM

all users reporting issues seem to be in asia

Paul•6/4/24, 4:27 PM

and so it begins... for some reason i am not able to see the HYPERDRIVE binding from getPlatformProxy within my SvelteKit hooks handler

AJR•6/4/24, 4:58 PM

Taking a look now.

MMitya Two questions for the CF Hyperdrive team: 1. Given HD's caching capabilities, is...

AJR•6/4/24, 5:00 PM

Depends on your use case. There's overlap in the problems they solve, certainly, but there's also reasons you might want both.
Yes, the issue with caching returning empty results was resolved quite a while back

Rramnivas Is there a relationship between Hyperdrive cache settings and the use of prepare...

AJR•6/4/24, 5:01 PM

We recently added support for prepared statements. Caching should not interact with it in this way, no. I'll add a ticket for us to take a look at this.

AAJR We recently added support for prepared statements. Caching should not interact w...

ramnivasOP•6/4/24, 6:26 PM

Thanks. That's good to know. I guess https://developers.cloudflare.com/hyperdrive/configuration/connect-to-postgres/#transaction-and-statement-support needs to change to drop the limitation on PREPARE?

Cloudflare Docs

Connect to PostgreSQL · Hyperdrive

Hyperdrive supports PostgreSQL and PostgreSQL-compatible databases, popular drivers and Object Relational Mapper (ORM) libraries that use those …

Rramnivas Thanks. That's good to know. I guess https://developers.cloudflare.com/hyperdriv...

AJR•6/4/24, 6:34 PM

Not exactly. We support protocol-level prepared statements, like your driver uses. We do not parse out and manage it as a keyword right now. At some point if there's enough need we could but we don't see that a lot honestly

ramnivasOP•6/4/24, 6:41 PM

Ah, I see. Thank you for the clarification.

AAJR Not exactly. We support protocol-level prepared statements, like your driver use...

ramnivasOP•6/4/24, 6:50 PM

BTW, a way to reproduce would be to take this example: https://github.com/cloudflare/workers-rs/blob/main/examples/tokio-postgres/src/lib.rs#L29 and change the line to use

query

query

instead of

simple_query

simple_query

GitHub

workers-rs/examples/tokio-postgres/src/lib.rs at main · cloudflare/...

Write Cloudflare Workers in 100% Rust via WebAssembly - cloudflare/workers-rs

Rramnivas BTW, a way to reproduce would be to take this example: https://github.com/cloudf...

Matt Silverlock•6/4/24, 7:13 PM

See https://github.com/sfackler/rust-postgres/pull/1067

GitHub

support unnamed statements by devsnek · Pull Request #1067 · sfackl...

Refs: #1017
Refs: #1044
This PR implements unnamed statements when querying the database. This is not a performance optimization. The library still uses a "prepared" statement in order to...

Matt Silverlock•6/4/24, 7:13 PM

@moody

(I can't DM you) - https://www.cloudflarestatus.com/incidents/q23jjvwggp78

for clarity, we only have one report so far, but are investigating.

Intermittent Hyperdrive connection issues in APJC

Rramnivas BTW, a way to reproduce would be to take this example: https://github.com/cloudf...

kian•6/4/24, 7:15 PM

Yeah, like Matt linked to, that example uses

simple_query

simple_query

intentionally.

https://github.com/cloudflare/workers-rs/pull/569#issuecomment-2098304851

AAJR We recently added support for prepared statements. Caching should not interact w...

ramnivasOP•6/4/24, 7:23 PM

@kian @elithrar @AJR Thanks for your responses.

Is there a document that describes the pooling mechanism? I am particularly interested in how prepared statements behave if I make two identical queries.

let res1 = client.query("SELECT * FROM FOO", &[]).await?;
let res2 = client.query("SELECT * FROM FOO", &[]).await?;

let res1 = client.query("SELECT * FROM FOO", &[]).await?;
let res2 = client.query("SELECT * FROM FOO", &[]).await?;

Would the statement prepared in the first query be cached? In other words, would Hyperdrive call

prepare

prepare

twice (as in https://github.com/sfackler/rust-postgres/blob/98f5a11bc0a8e451552d8941ffa078c7eb6cd60c/tokio-postgres/src/to_statement.rs#L18, which is invoked from https://github.com/sfackler/rust-postgres/blob/98f5a11bc0a8e451552d8941ffa078c7eb6cd60c/tokio-postgres/src/client.rs#L363.

A bit of context:

I am in the process of supporting Cloudflare Worker for Exograph (https://github.com/exograph/exograph/pull/1081). It is working okay without Hyperdrive (high latency) and great with Hyperdrive (with a fork of tokio-postgres to avoid prepared statements: https://github.com/sfackler/rust-postgres/pull/1147).

I would like to determine the extent of Hyperdrive support to decide if there is value in continuing to use my fork. For example, if Hyperdrive reuses the prepared statement, then going with

query

query

will be better (to be measured, of course). If not, my fork avoids an extra roundtrip.

Rramnivas @kian @elithrar @AJR Thanks for your responses. Is there a document that descri...

AJR•6/4/24, 7:27 PM

I don't think this is well documented. I'm happy to write up some explanation when I have some spare cycles. Hopefully today or tomorrow.

Short answer, it's complicated. We reuse the prepared statement if possible, or we re-send the parse on the new connection if not.

MMatt Silverlock @moody 💭 (I can't DM you) - https://www.cloudflarestatus.com/incidents/q23jjvwg...

moody 💭•6/4/24, 7:27 PM

ty. i think the no dms setting is on your end

AAJR I don't think this is well documented. I'm happy to write up some explanation wh...

ramnivasOP•6/4/24, 7:29 PM

That will be awesome. Thank you!

Kkian Yeah, like Matt linked to, that example uses `simple_query` intentionally. <htt...

ramnivasOP•6/4/24, 10:15 PM

Just a clarification: The use of

simple_query

simple_query

was required earlier (I believe until May 22 https://developers.cloudflare.com/hyperdrive/platform/changelog/#2024-05-22). With the prepared statement support in Hyperdrive, plain

query

query

also works with a caveat I mentioned (must turn on caching in Hyperdrive setting). The caveat is an issue in Hyperdrive (per @AJR), so once that is fixed, it will be all okay!

Cloudflare Docs

Changelog · Hyperdrive

You can now create up to 25 Hyperdrive configurations per account, up from the previous maximum of 10.

Paul•6/5/24, 3:44 AM

Is Hyperdrive smart enough to clear the cache of a SELECT query if a mutate has happened for that row? For instance, the user profile is cached but then they change their bio. If they refresh would they still see the old profile until “max-age”?

PPaul Is Hyperdrive smart enough to clear the cache of a SELECT query if a mutate has ...

Isaac McFadyen•6/5/24, 4:47 AM

max-age

max-age

is related to browser-side caching using HTTP headers. Hyperdrive has no ability to purge the browser-side cache so no.

PPaul Is Hyperdrive smart enough to clear the cache of a SELECT query if a mutate has ...

AJR•6/5/24, 11:58 AM

Without getting into browser caching, no Hyperdrive does not do proactive cache purging after volatile queries. It's something we've discussed but would be a non-trivial feature to add and generally is not something users have asked for overmuch.

If you need to bypass cache on specific queries, we don't check cache on any volatile queries, so you could add any such function. For example including

random()

random()

and ignoring that field in your result would do it.

If not purging cache on mutation is a dealbreaker feature for you, it would be helpful for us to understand a bit about your use case, if you'd be willing to chat about it.

AAJR Without getting into browser caching, no Hyperdrive does not do proactive cache ...

Isaac McFadyen•6/5/24, 1:24 PM

I'd also add that max-age is also browser-side along with server-side.

Isaac McFadyen•6/5/24, 1:24 PM

So even if Hyperdrive could purge the cache somehow (which seems like a bad idea IMO, a DB connection shouldn't be able to modify my CDN config) then it won't affect browser-side caching anyway.

Paul•6/5/24, 2:32 PM

Hmm seems to be a few caching layers here. Let’s say the user who owns the profile has edited it and expects to see the changes immediately. Is there a way to cache bust just for that user? (I’m not understanding how the browser cache plays along with the hyperdrive layer)

PPaul Hmm seems to be a few caching layers here. Let’s say the user who owns the profi...

Isaac McFadyen•6/5/24, 2:43 PM

So these are the cache layers I assume you have (correct me here if I'm wrong):

Browser cache, this is local on the user's web browser and based on
```
max-age
```
```
max-age
```
. This is quite tricky to bust and you usually disable this for APIs in favor of 2.
Server-side cache, this is the Cloudflare CDN and/or your server and/or the Worker that is serving the API requests. To purge this you need control of the Worker, or if it's the CDN cache you need to make an API call to the Cloudflare API. This is also based on
```
maxage
```
```
maxage
```
, but if you want to only cache on the server-side you can use
```
s-maxage
```
```
s-maxage
```
instead.
Hyperdrive query cache. I don't know many details on this, @AJR would know more about how cache busting works here.

Isaac McFadyen•6/5/24, 2:44 PM

Your original question was about whether a query in Hyperdrive would auto-cache-purge 1 or 2. It doesn't do that because it's a separate system and it's unintuitive/broken behavior if a database connection cache purges the other layers. The developer should have full control over whether that happens and if so, when and how.

AJR•6/5/24, 2:59 PM

That is all correct to the best of my understanding. My answer above about including a benign volatile function is the current best solution for cache busting point 3. We expect to add more polish on that front, though it isn't our highest priority at this time.

AAJR That is all correct to the best of my understanding. My answer above about inclu...

Isaac McFadyen•6/5/24, 3:39 PM

Ah completely missed the volatile function part - good to know.

IIsaac McFadyen So these are the cache layers I assume you have (correct me here if I'm wrong): ...

Paul•6/5/24, 4:06 PM

Thanks for the great write up!

IIsaac McFadyen Your original question was about whether a query in Hyperdrive would auto-cache-...

Matt Silverlock•6/5/24, 10:04 PM

I do think some things are being mixed up here:

Hyperdrive’s cache is only specific to Hyperdrive (per config ID). The
```
maxage
```
```
maxage
```
in Hyperdrive dictates that caches maximum TTL per unique query. It’s not a CDN cache and doesn’t cache any results in the browser.
```
maxage
```
```
maxage
```
is just a common name for a TTL-based cache configuration setting.
You can also use Cloudflare’s CDN cache from a Worker or Pages app, but there is no intersection with Hyperdrive (unless you actively choose to insert things into it).

MMatt Silverlock I do think some things are being mixed up here: 1. Hyperdrive’s cache is only s...

Isaac McFadyen•6/5/24, 10:05 PM

Ah, think you're right - apologies, thought this was referring to the

maxage

maxage

in the Cache-ControlCache-Control header.

Muudka•6/5/24, 11:48 PM

Hi, am a bit confused about hyperdrive, at the moment i have a node application that is deployed to aws lamda behind api gateway ,
I am using postgres db with prisma orm,
I have set up hyperdrive and run some test, how can I integrate it into my lambda deployment

MMuudka Hi, am a bit confused about hyperdrive, at the moment i have a node applicati...

Matt Silverlock•6/6/24, 1:57 AM

Where is your database? Hyperdrive is currently focused on making it faster to connect (and query) your database from Workers.

If you’re using Lambda and Aurora in the same region, it won’t make much sense here.

ramnivasOP•6/6/24, 2:01 AM

I have seen mentions in this/workers forum that Hyperdrive's

id

id

is not a secret, but I didn't find that mentioned in the documentation. It will be beneficial to have that mentioned officially.

MMatt Silverlock Where is your database? Hyperdrive is currently focused on making it faster to c...

Muudka•6/6/24, 6:34 AM

I am using postgtres on Neo.tech, does worker only work within the worker and how can it be incorporated into my code.
I am using prisma as ORM

MMuudka I am using postgtres on Neo.tech, does worker only work within the worker and ho...

thomasgauvin•6/6/24, 3:23 PM

@Muudka are you using AWS Lambda or Cloudflare Workers (I see mentions of both in this thread)? Which database vendor are you using/is it AWS RDS/Aurora?

Muudka•6/6/24, 3:26 PM

@thomasgauvin I am using AWS lambda behind api gateway with Postgres on neo.tech serverless db.

katopz•6/8/24, 3:28 AM

Do hyperdrive also work with PostgresML?

Rramnivas That will be awesome. Thank you!

AJR•6/10/24, 4:58 PM

Posting my response a little more publicly, for visibility's sake.

After looking into it, the way tokio-postgres implements prepared statements and the extended query protocol is rather different than how our supported drivers do it. We will work on adding support for extended queries in that driver, but it will not be very soon (lots of other big features we've committed to first). For the meantime, we recommend using simple queries with that driver. I'll check our docs and make sure that they offer similar guidance.

Note that while @ramnivas did see better behavior with caching, that's somewhat incidental and you're likely to still hit bugs using extended queries in tokio-postgres even with caching enabled.

Thank you @ramnivas for the excellent write-up!

ramnivasOP•6/10/24, 5:54 PM

@AJR Thanks for the update.

Meanwhile, for those who want to not use

simple_query

simple_query

, take a look at our fork (the exographexograph branch), which supports query_with_type_paramsquery_with_type_params that works well with Hyperdrive.

https://github.com/exograph/rust-postgres/tree/exograph

GitHub

GitHub - exograph/rust-postgres at exograph

Native PostgreSQL driver for the Rust programming language - GitHub - exograph/rust-postgres at exograph

Is there a relationship between Hyperdrive cache settings and the use of prepared statements? I am

Similar Threads

Similar Threads

Similar Threads