Good news: this works Bad news: `tokio-postgres` and connection pooling don't like each other, so y

Good news: this works

Bad news:

tokio-postgres

tokio-postgres

tokio-postgres

tokio-postgres and connection pooling don't like each other, so you have to resort to

query_raw

query_raw

query_raw

query_raw and whatnot which isn't the nicest.

kianOP•10/2/23, 11:39 AM

Good news: https://github.com/sfackler/rust-postgres/pull/1067 looks like it might fix that issue

Kkian Good news: this works Bad news: `tokio-postgres` and connection pooling don't l...

Matt Silverlock•10/2/23, 11:50 AM

Thanks for checking!

kianOP•10/2/23, 11:50 AM

Using that fork, it seems fine - querying my Neon DB is in the realm of 120-140ms

Kkian Good news: https://github.com/sfackler/rust-postgres/pull/1067 looks like it mig...

Matt Silverlock•10/2/23, 11:53 AM

It should. Named prepared statements don’t work with any form of connection pooling as they are connection-scoped in Postgres: https://developers.cloudflare.com/hyperdrive/learning/connect-to-postgres/#transaction-and-statement-support

Connect to PostgreSQL · Hyperdrive

Hyperdrive supports both PostgreSQL and PostgreSQL-compatible databases, as well as popular drivers and the many Object Relational Mapper (ORM) …

kianOP•10/2/23, 11:53 AM

The accompanying wasm_bindgenwasm_bindgen is at https://gist.github.com/KianNH/66c513ac8c37c3ab8e97d36665394094

kianOP•10/2/23, 11:55 AM

I'll write up the rest in the Gist later

MMatt Silverlock It should. Named prepared statements don’t work with any form of connection pool...

kianOP•10/2/23, 11:56 AM

I originally thought I was going mad after using a non-pooled connection string from Neon in Hyperdrive before I realised that's one of the main points of Hyperdrive, connection pooling. meowpuffydizzy

kianOP•10/2/23, 2:55 PM

It'd be nice if Wrangler made a warning/error when you use local development w/ Hyperdrive bindings. You do get a warning that they're experimental, and that callout is made in the docs, but it's an easy issue to hit with local being the default.

kianOP•10/2/23, 2:55 PM

Something similar to Queues, which tells you that they're not available in remote mode.

kianOP•10/2/23, 3:40 PM

Crossposting from #what-i-built

https://kian.org.uk/using-cloudflare-hyperdrive-with-tokio-postgres-in-workers/

Kian's Blog

Using Cloudflare Hyperdrive with tokio-postgres in Workers

Cloudflare recently announced their query caching & connection pooling solution for databases in Workers, Hyperdrive.

Whilst there's plenty of examples of using Hyperdrive with libraries like pg in the documentation, someone in the Cloudflare Developers Discord asked about usability with Rust Workers.

There are examples for using tokio-postgre...

kianOP•10/3/23, 3:57 PM

I think requiring an explicit port number in the connection string is a bit odd and non-standard, though. Could you default it to 5432 like everyone else?

There's an open PR which will do that when merged.

kianOP•10/3/23, 4:05 PM

My suspicion is that the id can only be used from within my own account, and so it isn't?

Yep.

Unsmart•10/3/23, 4:06 PM

All of the things you have to enter into

wrangler.toml

wrangler.toml

for bindings will not be secret, I think it would be an extremely bad look if an Internet security company made you enter in a secret value into a config that should be pushed to git

Jjawj Fab. One question: is my Hyperdrive binding ID secret at all? In other words, is...

Matt Silverlock•10/3/23, 4:09 PM

D1 database IDs, KV namespaces, Hyperdrive IDs, etc - none are sensitive. It’s all locked into your account.

UUnsmart All of the things you have to enter into `wrangler.toml` for bindings will not b...

Matt Silverlock•10/3/23, 4:09 PM

“Never make users put secrets in source”

Picolé•10/4/23, 6:09 PM

During the public beta, your account will need to be explicitly enabled by the Cloudflare team.

this still true? can I have it?

kianOP•10/4/23, 6:12 PM

That sounds like it should be specific to Enterprise accounts? @Matt

PPicolé > During the public beta, your account will need to be explicitly enabled by the...

Matt Silverlock•10/4/23, 6:12 PM

Where is this?

kianOP•10/4/23, 6:12 PM

https://developers.cloudflare.com/hyperdrive/get-started/#prerequisites

Kkian That sounds like it should be specific to Enterprise accounts? @Matt

Matt Silverlock•10/4/23, 6:12 PM

Enterprise accounts can also use it. We enabled them.

Matt Silverlock•10/4/23, 6:12 PM

That’s a typo.

Matt Silverlock•10/4/23, 6:14 PM

Fixing: https://github.com/cloudflare/cloudflare-docs/pull/11091

GitHub

Update get-started.md by elithrar · Pull Request #11091 · cloudflar...

Kkian It'd be nice if Wrangler made a warning/error when you use local development w/ ...

kianOP•10/4/23, 6:24 PM

Who will win, docs aside or https://github.com/cloudflare/workerd/pull/1266/

GitHub

Add Hyperdrive bindings by tmthecoder · Pull Request #1266 · cloudf...

Adda a new binding type Hyperdrive as well as a corresponding Service definition ExternalTcpService that allows TCP connections to the designated host address & port

kianOP•10/4/23, 6:24 PM

Kkian It'd be nice if Wrangler made a warning/error when you use local development w/ ...

Unsmart•10/4/23, 6:45 PM

Yeah ive noticed a lot with newer products people will attempt to use the bindings but get undefined because they have not updated wrangler... I dont think it would be too hard to look at keys not parsed and be like hey this isnt a valid key you might need to update or something... Or the remote/local thing also

UUnsmart Yeah ive noticed a lot with newer products people will attempt to use the bindin...

Matt Silverlock•10/4/23, 6:51 PM

cc @penalosa

UUnsmart Yeah ive noticed a lot with newer products people will attempt to use the bindin...

kianOP•10/4/23, 6:57 PM

I made https://github.com/cloudflare/workers-sdk/issues/2586 ages ago

GitHub

🚀 Feature Request: Expand validateAdditionalProperties to more conf...

Describe the solution The current usage of validateAdditionalProperties is only used for a subset of configuration fields, such as build or the top-level entries. This means that you can use the fo...

Sam•10/5/23, 1:14 PM

Quick questions - in the demo, a new connection is opened for each request (if I'm right). Won't this overload the PostgreSQL server when there're for example thousands of requests each second?

Also, I've created a Hyperdrive Id using wrangler. Should the dashboard in cloudflare change, or will it stay like this?

And last - in local development, I can't access the database (works in prod after wrangler deploy). I'm receiving the following error:

Cannot read properties of undefined (reading 'connectionString')

Cannot read properties of undefined (reading 'connectionString')

Do I need to add any parameters to the dev start up?

Thank you!

MMatt Silverlock Not yet, but very soon - we want to support local dev.

Sam•10/5/23, 1:23 PM

Ah nvm about the local dev.

SSam Quick questions - in the demo, a new connection is opened for each request (if I...

Isaac McFadyen•10/5/23, 1:42 PM

Hyperdrive is a connection pooler and avoids this exact thing. A few connections are opened from Hyperdrive, and then every request opens a new connection to Hyperdrive (but not the DB) so that your DB doesn't get overloaded.

Isaac McFadyen•10/5/23, 1:43 PM

Because connections are held open from Hyperdrive, the connections to Hyperdrive are very very quick and won't overload things.

IIsaac McFadyen Hyperdrive is a connection pooler and avoids this exact thing. A few connections...

Sam•10/5/23, 1:43 PM

Ahh ok, that's what I was thinking but I was not sure. Thanks for clarifying

Sam•10/5/23, 1:58 PM

For local dev, I'm trying this (pasting a string, if HYPERDRIVE is not available in the env), but I'm getting SSL issues. Is this happening for more people?

Or is my (managed by digital ocean) database cluster not set up correctly?

Sam•10/5/23, 2:08 PM

Also wrangler defaults to 3.0.0 after installing, should I update to 3.10.1?

Llithdew Ran into an issue just now trying to create a Hyperdrive over a PostgreSQL datab...

Matt Silverlock•10/5/23, 3:31 PM

Can you DM me your database URL? There’s not enough info in the issue to debug unfortunately.

SSam Also wrangler defaults to 3.0.0 after installing, should I update to 3.10.1?

Matt Silverlock•10/5/23, 3:31 PM

Yes

JJordan (UNCVRD)hey! I was wondering if this statement is accurate about being able to use Postg...

Matt Silverlock•10/5/23, 8:54 PM

You need to use the latest version of Postgres.jsPostgres.js

we are waiting for them to publish 3.4.03.4.0

JJordan (UNCVRD)also...i was wondering if we'd be able to control the max number of connections ...

Matt Silverlock•10/5/23, 8:54 PM

It's not currently configurable: 20 is very, very low though and would impact performance (fewer conns in the pool == more waiting)

mattfreeman•10/6/23, 6:49 AM

Is the connection string returned by env.HYPERDRIVE.connectionString usable outside of workers, is it accessible on from workstation when connecting over Warp Teams into the same account ?

kianOP•10/6/23, 8:54 AM

Nope

Nikil•10/7/23, 3:07 PM

So am I understanding Hyperdrive correctly? It's basically the same idea as Cloudflare DNS w/ CDN-style caching but for DBs? That sounds cool...

Good news: this works Bad news: `tokio-postgres` and connection pooling don't like each other, so y

Similar Threads

Similar Threads

Similar Threads