Where is your database? Hyperdrive is currently focused on making it faster to connect (and query) y

Where is your database? Hyperdrive is currently focused on making it faster to connect (and query) your database from Workers.

If you’re using Lambda and Aurora in the same region, it won’t make much sense here.

ramnivas•6/6/24, 2:01 AM

I have seen mentions in this/workers forum that Hyperdrive's

id

id

is not a secret, but I didn't find that mentioned in the documentation. It will be beneficial to have that mentioned officially.

MMatt Silverlock Where is your database? Hyperdrive is currently focused on making it faster to c...

Muudka•6/6/24, 6:34 AM

I am using postgtres on Neo.tech, does worker only work within the worker and how can it be incorporated into my code.
I am using prisma as ORM

MMuudka I am using postgtres on Neo.tech, does worker only work within the worker and ho...

thomasgauvin•6/6/24, 3:23 PM

@Muudka are you using AWS Lambda or Cloudflare Workers (I see mentions of both in this thread)? Which database vendor are you using/is it AWS RDS/Aurora?

Muudka•6/6/24, 3:26 PM

@thomasgauvin I am using AWS lambda behind api gateway with Postgres on neo.tech serverless db.

katopz•6/8/24, 3:28 AM

Do hyperdrive also work with PostgresML?

Rramnivas That will be awesome. Thank you!

AJR•6/10/24, 4:58 PM

Posting my response a little more publicly, for visibility's sake.

After looking into it, the way tokio-postgres implements prepared statements and the extended query protocol is rather different than how our supported drivers do it. We will work on adding support for extended queries in that driver, but it will not be very soon (lots of other big features we've committed to first). For the meantime, we recommend using simple queries with that driver. I'll check our docs and make sure that they offer similar guidance.

Note that while @ramnivas did see better behavior with caching, that's somewhat incidental and you're likely to still hit bugs using extended queries in tokio-postgres even with caching enabled.

Thank you @ramnivas for the excellent write-up!

ramnivas•6/10/24, 5:54 PM

@AJR Thanks for the update.

Meanwhile, for those who want to not use simple_querysimple_query , take a look at our fork (the exographexograph branch), which supports query_with_type_paramsquery_with_type_params that works well with Hyperdrive.

https://github.com/exograph/rust-postgres/tree/exograph

GitHub

GitHub - exograph/rust-postgres at exograph

Native PostgreSQL driver for the Rust programming language - GitHub - exograph/rust-postgres at exograph

thomasgauvin•6/11/24, 8:59 PM

@Olivier F @Florent funny you should ask! this is one of our main things in progress currently, we should setup a meeting to talk it through, get some of the details from your side and get your early feedback. These are my availabilities https://calendar.app.google/ynRoYiJudGqyGPRN9

Appointments

ramnivas•6/12/24, 5:31 PM

Check out our latest blog that makes use of Hyperdrive to improve latency:

https://exograph.dev/blog/cloudflare-workers

Exograph at the Edge with Cloudflare Workers | Exograph

We are excited to announce that Exograph can now run as a Cloudflare Worker! This new capability allows deploying Exograph servers at the edge, closer to your users, and with lower latency.

DanielZero•6/13/24, 5:05 PM

I use postgres.js (latest version 3.4.4), hyperdrive and supabase. Every now and then a query fails due to CONNECTION_CLOSEDCONNECTION_CLOSED Disabling the hyperdrive cache didn't help and I can't find any open issue on that topic in the postgres.js repo. Am I the only one?

Isak•6/13/24, 10:19 PM

Why does query latency fluctuate so much? It's normally around 60 ms but sometimes down to 10 ms for short periods.

IvanLisz•6/16/24, 4:44 PM

(pages + hyperdrive) question: Can i separate prod and development or do i need two accounts?

IvanLisz•6/16/24, 9:27 PM

"This repository is being used for a Cloudflare Pages project on a different Cloudflare account." Nop, not even that.

IvanLisz•6/16/24, 9:45 PM

forked and works

IIvanLisz (pages + hyperdrive) question: Can i separate prod and development or do i need ...

Walshy•6/16/24, 11:26 PM

You can use the production and preview environments, e.g.

[env.production]
[[env.production.hyperdrive]]
binding = "HYPERDRIVE"
id = "id-1"

[env.preview]
[[env.preview.hyperdrive]]
binding = "HYPERDRIVE"
id = "id-2"

[env.production]
[[env.production.hyperdrive]]
binding = "HYPERDRIVE"
id = "id-1"

[env.preview]
[[env.preview.hyperdrive]]
binding = "HYPERDRIVE"
id = "id-2"

Docs: https://developers.cloudflare.com/pages/functions/wrangler-configuration/#configure-environments

IvanLisz•6/18/24, 7:27 PM

but this means that I being a developer can switch my preview branch hyperdrive binding and access the production database, right?

kindaraffy•6/20/24, 10:04 PM

Hi, I believe you could run drizzle with both postgres-js and node-postgres drivers, or does this piece of the documentation relate to something else entirely? https://developers.cloudflare.com/hyperdrive/configuration/connect-to-postgres/#supported-drivers

Cloudflare Docs

Connect to PostgreSQL · Hyperdrive

Hyperdrive supports PostgreSQL and PostgreSQL-compatible databases, popular drivers and Object Relational Mapper (ORM) libraries that use those …

kindaraffy•6/20/24, 10:04 PM

https://orm.drizzle.team/docs/get-started-postgresql#supabase Step 3 shows how you could use the Postgres js driver with drizzle.

Drizzle ORM - PostgreSQL

Drizzle ORM is a lightweight and performant TypeScript ORM with developer experience in mind.

Kkindaraffy Hi, I believe you could run drizzle with both postgres-js and node-postgres driv...

Sam•6/21/24, 12:48 PM

I'm using drizzle myself with node pg (https://www.npmjs.com/package/pg). Everything is working correctly, or is that not your question?

kindaraffy•6/21/24, 3:12 PM

No I was just pointing out that the docs may potentially be misleading when it mentions drizzle only works with node postgres when you can run it using the recommended postgres-js client.

Kkindaraffy No I was just pointing out that the docs may potentially be misleading when it m...

AJR•6/21/24, 4:04 PM

Ah good catch, thank you!

Elektra•6/24/24, 4:10 PM

Is hyperdrive down?

Elektra•6/24/24, 4:11 PM

For the past 40 to 50 minutes. Out of nowhere I get cannot get it...failed to aquire a connection from the pool.

Elektra•6/24/24, 4:11 PM

Nothing changed. I haven't touched code for the last 6h ...

Elektra•6/24/24, 4:12 PM

All was working...and I can connect directly to the db.

AJR•6/24/24, 4:12 PM

Taking a look

Elektra•6/24/24, 4:12 PM

Thank you AJR

AJR•6/24/24, 4:13 PM

What's your hyperdrive ID?

AJR•6/24/24, 4:13 PM

I'm still seeing normal global traffic levels, no elevated error rates.

Elektra•6/24/24, 4:14 PM

sent DM

Divyendu•6/26/24, 12:15 PM

Does hyperdrive work with Cloudflare pages or just workers for now?

Walshy•6/26/24, 12:18 PM

it's supported on Pages

Divyendu•6/26/24, 12:22 PM

Thanks, I get this error in the build logs, I have the binding defined in my wrangler.toml. Investigating

. I asked the question becuase I didn't see the binding in pages UI, but good to know that it works!

Divyendu•6/26/24, 4:11 PM

Aha, got it working, for anyone who runs into this, in the end it was simpler than I thought, just need compatibility_flags = [ "nodejs_compat" ]compatibility_flags = [ "nodejs_compat" ] in

wrangler.toml

wrangler.toml

and then normal functions directory just works™. It took me a while to figure it out because most of the community resources, gh issues etc are slightly outdated and took me a while to understand that node_compat = truenode_compat = true is a build time thing not supported by pages and likely won't be (as it is legacy) and compatibility_flagscompatibility_flags is the way to go!

Divyendu•6/27/24, 10:18 AM

I am not sure if this fix should happen in Cloudflare or Remix but I need to provide a "fake" env var "WRANGLER_HYPERDRIVE_LOCAL_CONNECTION_STRING_HYPERDRIVE" for remix vite:buildremix vite:build to succeed in CI.

I think this should be fixed in Cloudflare (it should detect that it is running in the CI and not fail with the "when running locally" message)

Matt SilverlockOP•6/28/24, 2:43 PM

If you're interested in how Hyperdrive works under the hood, @AJR just published a blog: https://blog.cloudflare.com/postgres-named-prepared-statements-supported-hyperdrive

The Cloudflare Blog

Supporting Postgres Named Prepared Statements in Hyperdrive

Hyperdrive (Cloudflare’s globally distributed SQL connection pooler and cache) recently added support for Postgres protocol-level named prepared statements across pooled connections. We dive deep on what it took to add this feature.

kindaraffy•6/29/24, 11:54 AM

Is there a way to specify read replicas for a hyperdrive configuration?

thomasgauvin•6/30/24, 11:53 PM

Not currently possible, how many replicas do you have? Which db provider are you using?

Tthomasgauvin Not currently possible, how many replicas do you have? Which db provider are you...

Sam•7/1/24, 1:09 AM

Will read replicas be supported in the near future, like when using SELECT queries Hyperdrive automatically uses the nearest Postgres servers? Ideally with the same set-up as HTTP load balancing

anirudh•7/1/24, 4:31 AM

We are using Hyperdrive, but it seems to be very slow. A query that should take 20ms-30ms is actually taking 800-900ms. The dashboard reports a query latency of 150ms, but it should still be less than 800ms. The database is located in Bangalore (BLR) and the worker is in Chennai (MAA), so they are quite close to each other.

anirudh•7/1/24, 4:48 AM

Is there a way to check in which location the hyperdrive is?

AJR•7/1/24, 2:19 PM

@anirudh What is your hyperdrive ID? Send it in a DM?

Is this consistently the latency you're seeing, or is this a spike?

AJR•7/2/24, 12:28 PM

We have made some changes since June 15th, yes. Please DM me with some details of your setup, if you wouldn't mind. Where's your origin DB, where are most of your users, what driver/version are you using, and any other info you're able to share.

Where is your database? Hyperdrive is currently focused on making it faster to connect (and query) y

Similar Threads

Similar Threads

Similar Threads