Tiny bug, the minimum postgres.js version mentioned that it should be 3.4.5, should this package sti

Tiny bug, the minimum postgres.js version mentioned that it should be 3.4.5, should this package still be published or should this version number actually be 3.4.4, since 3.4.5 doesn't exist (yet?), see here: https://developers.cloudflare.com/hyperdrive/configuration/connect-to-postgres/#supported-drivers

Cloudflare Docs

Connect to PostgreSQL · Hyperdrive

Hyperdrive supports PostgreSQL and PostgreSQL-compatible databases, popular drivers and Object Relational Mapper (ORM) libraries that use those …

IItsWendell Tiny bug, the minimum postgres.js version mentioned that it should be 3.4.5, sho...

Matt Silverlock•5/24/24, 11:01 AM

Fixing

zensin•5/27/24, 10:28 AM

recommendation for supabase?

zensin•5/27/24, 10:29 AM

I also wonder what (If any) caching supabase is doing on their postgrest API

AJR•5/27/24, 12:59 PM

Can't speak to the caching, but Hyperdrive operates in transaction mode itself. At the supabase level you'd be fine with session mode.

AJR•5/27/24, 1:01 PM

Having both in transaction mode means there's two sets of abstraction to keep in mind and you might run into differences in which things are handled.

AAJR Having both in transaction mode means there's two sets of abstraction to keep in...

zensin•5/28/24, 5:29 PM

Thanks. What about rawdawg direct mode (No connection pooler)? Bad idea?

zensin•5/28/24, 5:31 PM

I feel like it would be a good idea (just intuitively, no evidence)

Zzensin Thanks. What about rawdawg direct mode (No connection pooler)? Bad idea?

AJR•5/28/24, 5:31 PM

If you're putting Hyperdrive on top of that, direct mode is fine too. If you need to do some particularly heavy connection multiplexing, having the supabase knobs to turn might prove useful but in most cases shouldn't be necessary.

zensin•5/28/24, 5:32 PM

zensin•5/28/24, 5:33 PM

whatev I'll just get an ipv4 address

zensin•5/28/24, 5:39 PM

https://github.com/PostgREST/postgrest

GitHub

GitHub - PostgREST/postgrest: REST API for any Postgres database

REST API for any Postgres database. Contribute to PostgREST/postgrest development by creating an account on GitHub.

zensin•5/28/24, 5:39 PM

imagine I could run postgrest in a worker

zensin•5/28/24, 5:40 PM

it's haskell. maybe it could happen

Elektra•5/29/24, 7:11 AM

hello anyone experiencing:
Database error: error: Failed to acquire a connection from the pool.

EElektra hello anyone experiencing: Database error: error: Failed to acquire a connection...

thomasgauvin•5/29/24, 4:09 PM

Any details on this? Using Hyperdrive, what's the database provider, etc.?

Elektra•5/29/24, 4:20 PM

Hi Thomas, I'm hosting my own instance of pg I'm using the recommended library for connecting. I never had any such issues before that's why I was asking. Been fine for the last 4 months since I've used it. I waited a bit then went to pgadmin to investigate (could have been my side, although I tested while waiting with other apis that I have apart from this worker and worked fine) and about 5 min after all was okay again. Went to the status page also of cloudflare but didn't see anything at the time. Maybe some transient error...

Elektra•5/29/24, 4:23 PM

By the the the connections was not exhausted at all. I have the stock 100 and only 7 were active. Although I now don't recall seeing cloudflare mentioned on the connections... So not sure... Unfortunately I do see the errors in the dashboard but... I can't see anything else except from that log message that I was getting. So I don't really have much more information at the moment.

berkinovish•5/31/24, 7:11 AM

While waiting for Cloudflare Tunnel support for Hyperdrive, are there any resources available about connecting to AWS RDS (via RDS Proxy) directly through a Cloudflare Tunnel from a worker?

ink pot monkey ✒🐒•5/31/24, 2:20 PM

Helloo everyone

I facing a puzzling situation. I have a postgres database that I am connecting to via hyperdrive that is run on a server with a firewall which has all of cloudflare's IP's whitelisted. When I trigger the cloudflare worker that uses hyperdrive to do database operations from my home (in Spain) this works. However this worker is a callback handler and whenever the service connects (from Slovakia) the database operations silently fail. I remember working with our sysAdmin to whitelist a different IP so my questions would be, is failure something to do with the firewall? What are the IP's that hyperdrive uses to connect to databases? Would these IP's be different between Spain and Slovakia. And if not does anyone have any idea what could be happening?

ramnivas•6/4/24, 2:18 AM

Is there a relationship between Hyperdrive cache settings and the use of prepared statements?

I am exploring Postgres through Rust (using the tokio-postgrestokio-postgres crate) using the

query

query

method. This uses the extended query protocol. I am observing the following behavior. When the Hyperdrive cache is:

Disabled: All (or most) queries fail (the driver reports an UnexpectedMessageUnexpectedMessage error). I suspect disabled caching makes Hyperdrive use one connection for preparation and another execution on another. Occasionally, it might use the same connection, leading to a successful query.
Enabled: All queries succeed. I suspect caching makes Hyperdrive choose the same connection it used to prepare the statement.

Am I interpreting this correctly? Is this the expected behavior?

p.s. This seems to have changed recently. Earlier,

query

query

would always fail regardless of the cache setting. So, I started working on a PR for the Postgres driver to make it work with Hyperdrive. But today, during testing that PR, I saw this behavior, thus this question.

Client in tokio_postgres - Rust

An asynchronous PostgreSQL client.

moody 💭•6/4/24, 5:16 AM

is something going on with cloudflare?

moody 💭•6/4/24, 5:17 AM

as of 2 hours ago my workers are performing terribly, same number of requests

Mitya•6/4/24, 10:01 AM

Two questions for the CF Hyperdrive team:

Given HD's caching capabilities, is it redundant to also utilise read replicas on my DB, or would they still be a good idea?
Six months or so ago I and others noticed that HD had a bug whereby, if caching was enabled, sometimes queries returned erroneous results e.g. no rows where there should definitely be rows. I recall vaguely that a CF dev said this was acknowledged and a fix was scheduled. Did this happen, i.e. is it safe for me to re-enable caching? (I disabled it at the time due to this).
Thanks in advance.

moody 💭•6/4/24, 3:11 PM

i seem to be having serious performance issues with hyperdrive connected to a postgres db in hetzner.
why is latency like this? other graphs are relatively flat, but errors and latency are spiking and users are reporting slowness.

Screenshot_2024-06-04_at_11.10.30_AM.png

Screenshot_2024-06-04_at_11.10.39_AM.png

Paul•6/4/24, 3:25 PM

Can we get some verbiage on the Supabase + Hyperdrive setup page in regards to the Supabase pooler? Do we use the pooler connection string or the direct connection string from the Supabase Database settings page?
https://developers.cloudflare.com/hyperdrive/examples/supabase/

Cloudflare Docs

Connect to Supabase · Hyperdrive

This example shows you how to connect Hyperdrive to a Supabase Postgres database.

Mmoody 💭i seem to be having serious performance issues with hyperdrive connected to a po...

moody 💭•6/4/24, 3:36 PM

all users reporting issues seem to be in asia

Paul•6/4/24, 4:27 PM

and so it begins... for some reason i am not able to see the HYPERDRIVE binding from getPlatformProxy within my SvelteKit hooks handler

AJR•6/4/24, 4:58 PM

Taking a look now.

MMitya Two questions for the CF Hyperdrive team: 1. Given HD's caching capabilities, is...

AJR•6/4/24, 5:00 PM

Depends on your use case. There's overlap in the problems they solve, certainly, but there's also reasons you might want both.
Yes, the issue with caching returning empty results was resolved quite a while back

Rramnivas Is there a relationship between Hyperdrive cache settings and the use of prepare...

AJR•6/4/24, 5:01 PM

We recently added support for prepared statements. Caching should not interact with it in this way, no. I'll add a ticket for us to take a look at this.

AAJR We recently added support for prepared statements. Caching should not interact w...

ramnivas•6/4/24, 6:26 PM

Thanks. That's good to know. I guess https://developers.cloudflare.com/hyperdrive/configuration/connect-to-postgres/#transaction-and-statement-support needs to change to drop the limitation on PREPARE?

Cloudflare Docs

Connect to PostgreSQL · Hyperdrive

Hyperdrive supports PostgreSQL and PostgreSQL-compatible databases, popular drivers and Object Relational Mapper (ORM) libraries that use those …

Rramnivas Thanks. That's good to know. I guess https://developers.cloudflare.com/hyperdriv...

AJR•6/4/24, 6:34 PM

Not exactly. We support protocol-level prepared statements, like your driver uses. We do not parse out and manage it as a keyword right now. At some point if there's enough need we could but we don't see that a lot honestly

ramnivas•6/4/24, 6:41 PM

Ah, I see. Thank you for the clarification.

AAJR Not exactly. We support protocol-level prepared statements, like your driver use...

ramnivas•6/4/24, 6:50 PM

BTW, a way to reproduce would be to take this example: https://github.com/cloudflare/workers-rs/blob/main/examples/tokio-postgres/src/lib.rs#L29 and change the line to use

query

query

instead of

simple_query

simple_query

GitHub

workers-rs/examples/tokio-postgres/src/lib.rs at main · cloudflare/...

Write Cloudflare Workers in 100% Rust via WebAssembly - cloudflare/workers-rs

Rramnivas BTW, a way to reproduce would be to take this example: https://github.com/cloudf...

Matt Silverlock•6/4/24, 7:13 PM

See https://github.com/sfackler/rust-postgres/pull/1067

GitHub

support unnamed statements by devsnek · Pull Request #1067 · sfackl...

Refs: #1017
Refs: #1044
This PR implements unnamed statements when querying the database. This is not a performance optimization. The library still uses a "prepared" statement in order to...

Matt Silverlock•6/4/24, 7:13 PM

@moody

(I can't DM you) - https://www.cloudflarestatus.com/incidents/q23jjvwggp78

for clarity, we only have one report so far, but are investigating.

Intermittent Hyperdrive connection issues in APJC

Rramnivas BTW, a way to reproduce would be to take this example: https://github.com/cloudf...

kian•6/4/24, 7:15 PM

Yeah, like Matt linked to, that example uses

simple_query

simple_query

intentionally.

https://github.com/cloudflare/workers-rs/pull/569#issuecomment-2098304851

AAJR We recently added support for prepared statements. Caching should not interact w...

ramnivas•6/4/24, 7:23 PM

@kian @elithrar @AJR Thanks for your responses.

Is there a document that describes the pooling mechanism? I am particularly interested in how prepared statements behave if I make two identical queries.

let res1 = client.query("SELECT * FROM FOO", &[]).await?;
let res2 = client.query("SELECT * FROM FOO", &[]).await?;

let res1 = client.query("SELECT * FROM FOO", &[]).await?;
let res2 = client.query("SELECT * FROM FOO", &[]).await?;

Would the statement prepared in the first query be cached? In other words, would Hyperdrive call

prepare

prepare

twice (as in https://github.com/sfackler/rust-postgres/blob/98f5a11bc0a8e451552d8941ffa078c7eb6cd60c/tokio-postgres/src/to_statement.rs#L18, which is invoked from https://github.com/sfackler/rust-postgres/blob/98f5a11bc0a8e451552d8941ffa078c7eb6cd60c/tokio-postgres/src/client.rs#L363.

A bit of context:

I am in the process of supporting Cloudflare Worker for Exograph (https://github.com/exograph/exograph/pull/1081). It is working okay without Hyperdrive (high latency) and great with Hyperdrive (with a fork of tokio-postgres to avoid prepared statements: https://github.com/sfackler/rust-postgres/pull/1147).

I would like to determine the extent of Hyperdrive support to decide if there is value in continuing to use my fork. For example, if Hyperdrive reuses the prepared statement, then going with

query

query

will be better (to be measured, of course). If not, my fork avoids an extra roundtrip.

Rramnivas @kian @elithrar @AJR Thanks for your responses. Is there a document that descri...

AJR•6/4/24, 7:27 PM

I don't think this is well documented. I'm happy to write up some explanation when I have some spare cycles. Hopefully today or tomorrow.

Short answer, it's complicated. We reuse the prepared statement if possible, or we re-send the parse on the new connection if not.

MMatt Silverlock @moody 💭 (I can't DM you) - https://www.cloudflarestatus.com/incidents/q23jjvwg...

moody 💭•6/4/24, 7:27 PM

ty. i think the no dms setting is on your end

AAJR I don't think this is well documented. I'm happy to write up some explanation wh...

ramnivas•6/4/24, 7:29 PM

That will be awesome. Thank you!

Kkian Yeah, like Matt linked to, that example uses `simple_query` intentionally. <htt...

ramnivas•6/4/24, 10:15 PM

Just a clarification: The use of

simple_query

simple_query

was required earlier (I believe until May 22 https://developers.cloudflare.com/hyperdrive/platform/changelog/#2024-05-22). With the prepared statement support in Hyperdrive, plain

query

query

also works with a caveat I mentioned (must turn on caching in Hyperdrive setting). The caveat is an issue in Hyperdrive (per @AJR), so once that is fixed, it will be all okay!

Cloudflare Docs

Changelog · Hyperdrive

You can now create up to 25 Hyperdrive configurations per account, up from the previous maximum of 10.

Paul•6/5/24, 3:44 AM

Is Hyperdrive smart enough to clear the cache of a SELECT query if a mutate has happened for that row? For instance, the user profile is cached but then they change their bio. If they refresh would they still see the old profile until “max-age”?

PPaul Is Hyperdrive smart enough to clear the cache of a SELECT query if a mutate has ...

Isaac McFadyen•6/5/24, 4:47 AM

max-age

max-age

is related to browser-side caching using HTTP headers. Hyperdrive has no ability to purge the browser-side cache so no.

PPaul Is Hyperdrive smart enough to clear the cache of a SELECT query if a mutate has ...

AJR•6/5/24, 11:58 AM

Without getting into browser caching, no Hyperdrive does not do proactive cache purging after volatile queries. It's something we've discussed but would be a non-trivial feature to add and generally is not something users have asked for overmuch.

If you need to bypass cache on specific queries, we don't check cache on any volatile queries, so you could add any such function. For example including

random()

random()

and ignoring that field in your result would do it.

If not purging cache on mutation is a dealbreaker feature for you, it would be helpful for us to understand a bit about your use case, if you'd be willing to chat about it.

Tiny bug, the minimum postgres.js version mentioned that it should be 3.4.5, should this package sti

Similar Threads