Yes, the exograph folks are around here occasionally, and that's a good write-up. While we don't of

Yes, the exograph folks are around here occasionally, and that's a good write-up. While we don't officially support extended protocol messages in tokio-postgres right now, their fork seems to work quite well for people.

AAJR Yes, the exograph folks are around here occasionally, and that's a good write-up...

Matt Silverlock•10/7/24, 1:02 PM

IIRC they merged this into mainline a little while ago: https://github.com/sfackler/rust-postgres/pull/1147

GitHub

Work with pools that don't support prepared statements by ramnivas ...

Introduce a new query_with_param_types method that allows to specify Postgres type parameters. This obviated the need to use prepared statements just to obtain parameter types for a query. It then ...

MMatt Silverlock IIRC they merged this into mainline a little while ago: https://github.com/sfack...

AJROP•10/7/24, 1:05 PM

Oh cool, I missed that one entirely. Nice!

Aviation•10/7/24, 11:29 PM

I noticed that Hyperdrive now supports prepared statements, but that the docs have not been updated to reflect this change.

I'm wondering if it now supports

SET

SET

statements, too, but the docs just isn't up to date

Cloudflare Docs

How Hyperdrive works | Hyperdrive

Connecting to traditional centralized databases from Cloudflare’s global network which consists of over 300 data center locations

presents a few challenges as queries can originate from any of these locations.

AJROP•10/8/24, 12:01 AM

No, the PREPARE keyword is still not supported I'm afraid. Named prepared statements at the protocol level are supported now. This was to support the default behavior of some key drivers in the ecosystem.

thomasgauvin•10/10/24, 10:32 AM

We documented how you can use Hyperdrive bindings from Pages here!

https://developers.cloudflare.com/pages/functions/bindings/#hyperdrive-configs

Note that you will need to use Postgres.js, and set Node.js compatibility using the following configurations in your wrangler.toml

compatibility_flags = [ "nodejs_compat" ]
compatibility_date = "2024-09-23"

compatibility_flags = [ "nodejs_compat" ]
compatibility_date = "2024-09-23"

(This was mentioned in a separate thread but raising this in the main channel so all can see!)

Nick•10/10/24, 4:05 PM

Does Hyperdrive produce error logs that are visible somewhere? All I'm seeing returned to my worker is "TypeError: Cannot redefine property: query", and nothing in the Tunnel logs.

AJROP•10/10/24, 5:00 PM

That's a JS error, Hyperdrive isn't throwing that one.

AJROP•10/10/24, 5:01 PM

Example: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Errors/Cant_redefine_property

MDN Web Docs

TypeError: can't redefine non-configurable property "x" - JavaScrip...

The JavaScript exception "can't redefine non-configurable property" occurs when it was
attempted to redefine a property, but that property is non-configurable.

AJROP•10/10/24, 5:02 PM

To answer the actual question, though, we return all errors from the origin server, plus selected Hyperdrive-specific errors, as Postgres errors to your PG client. The recommended path here is to handle errors from your driver, and either log or condition on them as needed. Logging them will have them populated with all the other Worker logs

bennycondon•10/11/24, 10:48 AM

Sorry if this is a naive question. I saw on How Hyperdrive works | Hyperdrive that

SET

SET

statements aren't supported. In that case, how would I perform an

UPDATE

UPDATE

statement without using a

SET

SET

statement?

Cloudflare Docs

How Hyperdrive works | Hyperdrive

Connecting to traditional centralized databases from Cloudflare’s global network which consists of over 300 data center locations

presents a few challenges as queries can originate from any of these locations.

Bbennycondon Sorry if this is a naive question. I saw on [How Hyperdrive works | Hyperdrive](...

kian•10/11/24, 11:13 AM

You might be confusing the

SET

SET

keyword within an

UPDATE

UPDATE

statement with the unrelated

SET

SET

statement

kian•10/11/24, 11:13 AM

The

SET

SET

statement is for changing run-time configuration: https://www.postgresql.org/docs/current/sql-set.html

bennycondon•10/11/24, 11:15 AM

I was confusing the two. Thanks so much for the help kian!

yevgen•10/14/24, 10:38 AM

New how-to configuration for Hyperdrive https://developers.cloudflare.com/hyperdrive/configuration/rotate-credentials/

Cloudflare Docs

Rotating database credentials | Hyperdrive

You can change the connection information and credentials of your Hyperdrive configuration in one of two ways:

Aalgads I assume the limitation on SETs is only if they are issued as separate commands....

AJROP•10/18/24, 3:54 PM

This is a scenario where the Postgres syntax is a bit unfortunate. SET in the context of UPDATE commands (keyword) and SET in the context of session state management (statement) are entirely unrelated, from the message protocol all the way down to the internals and back. They just happen to share the same name.

SET ROLE and such that are for session state management are going to be a problem, almost regardless of how you wrap them. We've done work to make the system more resilient, so you might get lucky, but generally I'd avoid those.

The SET keyword in the context of UPDATE statements is fine, go nuts with it.

AJROP•10/18/24, 3:58 PM

I will say that the root cause of problems here is polluting state on shared/pooled connections, so if you scope that state pollution such that by the time your query resolves everything is back how it started, it seems to me that you're less likely to have problems. If you can make it work for your use case, we certainly aren't going to stop you. But it's not something we support so, y'know, we won't be able to help you much. And we do reserve the right to change things in ways that might break that workflow if we need to. Kind of "undefined behavior, in the programming languange sense" territory.

thomasgauvin•10/18/24, 4:39 PM

Hey folks! It's great seeing all the feedback for Hyperdrive with private databases using tunnels, if you've been trying out and using it for the past few weeks, we want to learn more about your experience.

Likewise, if you've been using Hyperdrive, would love to chat 20-30 mins to better understand how you're using Hyperdrive, what are some of your asks around feature updates, how your performance has been, etc.

We have a bunch of improvements under way to improve compatibility (custom certs

) and performance and we want to make sure we're taking in your feedback! Here's my calendar -> https://calendar.app.google/b3w5dNYQj9YANw449

Appointments

dvddvd300•10/19/24, 5:07 AM

Hi, its there an updated timeline for MySQL support? (like early/beta/unstable access)
(MariaDB 10, 12GB database, ~ 200 Read Query/day)
tks

Ddvddvd300 Hi, its there an updated timeline for MySQL support? (like early/beta/unstable a...

AJROP•10/19/24, 11:14 AM

Not a timeline we can share publicly yet, no.

I'll say that nobody has beta access yet, but when we get that ready we'll be asking/DMing folks. If you're interested I'll keep a note to see about reaching out to you.

We have to land custom cert support and some pretty important perf work first before we add on new features of that scale.

Also, please do hit up @thomasgauvin so he can get some details about your use case. We want to make sure that when we do ship MySQL support we don't miss features of it that a bunch of people need.

marcbejar•10/19/24, 10:13 PM

Hi! I'm trying to set up a VPS with Postgres via docker compose and then bind it in a worker via Hyperdrive.

I have followed the instructions line by line except when setting up the public hostname: instead of localhost:5432 -> postgres:5432.

When the API calls the Hyperdrive returns 504 (Gateway Timeout) to the client. To make sure the DB is working, I have tried to access the database from the VPS console through postgres:5432 and it works fine.

Any suggestions?

ZZak @ insertchat.com Hey @Matt, is it possible to use HyperDrive without using workers? My project is...

mr nooli•10/20/24, 10:40 PM

Hey! Seems like this has already been asked, but I wanted to ask if there's been any updates - I want to test hyperdrive with my vercel (nextjs) project. I have plans to switch to cloudflare, but don't have the bandwidth (

) to fix all the bugs that'll come up in the migration - yet. Either way I want to test if hyperdrive can speed up my platform by any means.

Mmr nooli Hey! Seems like this has already been asked, but I wanted to ask if there's been...

AJROP•10/21/24, 12:43 AM

Short answer is that this isn't something that's likely to happen any time soon. While I wouldn't close the door on it ever happening, it's not on our radar for the time being.

mr nooli•10/21/24, 12:44 AM

Okay cool! Not an issue - is it something that would work well once I’ve deployed my next app on cloudflare?

mr nooli•10/21/24, 12:44 AM

As in, can I use the binding with my next app?

AJROP•10/21/24, 12:44 AM

Sorry, which binding is that? Not sure I'm following.

Mmr nooli As in, can I use the binding with my next app?

marcbejar•10/21/24, 7:30 AM

I think you are looking for this:
https://developers.cloudflare.com/workers/frameworks/framework-guides/nextjs/

Cloudflare Docs

Next.js | Cloudflare Workers docs

Create an Next.js application and deploy it to Cloudflare Workers with Workers Assets.

Mmarcbejar I think you are looking for this: https://developers.cloudflare.com/workers/fram...

mr nooli•10/21/24, 8:23 AM

Yep, exactly! The hyperdrive plugin with next on pages or open next

mr nooli•10/21/24, 9:26 AM

That’s really interesting actually, I tried to use the neon driver with open next @marcbejar but ran into this issue, https://github.com/cloudflare/next-on-pages/issues/605#issuecomment-2419842548

is this something that hyperdrive would fix?

GitHub

[🐛 Bug]: `pg` does not work · Issue #605 · cloudflare/next-on-pages

next-on-pages environment related information System: Platform: linux Arch: x64 Version: #202312122204 SMP PREEMPT_DYNAMIC Tue Dec 12 22:43:56 UTC 2023 CPU: (16) x64 AMD EPYC 7B13 Memory: 63 GB She...

marcbejar•10/21/24, 9:48 AM

@mr nooli If you are using Hyperdrive with a NEON connected to it, cloudflare recomends using postgres instead of neon driver or pg. See:

https://developers.cloudflare.com/hyperdrive/examples/neon/

Cloudflare Docs

Connect to Neon | Hyperdrive

Connect Hyperdrive to a Neon Postgres database.

mr nooli•10/21/24, 10:00 AM

Ah that's really interesting, I'm unfortunately using supabase and as far as I can see prisma doesn't support the postgres driver! They do however support neon and pg, and pg has been buggy from day 1! I've not been able to get it to work at all.

AJROP•10/21/24, 12:50 PM

We support pg and postgres.js drivers. Both work very well as standalones, but depending on what Prisma is doing with them it may not cooperate.

I don't know if we've evaluated the neon driver. We all use and love neon as a product, might be worth giving it a look. I'll mention to our product folks and see what they think.

mr nooli•10/21/24, 1:00 PM

That would be really useful! I want to migrate the opennext but I'm just running into a bunch of bugs, that I'm clueless on how to fix!

AJROP•10/21/24, 1:02 PM

Unfortunately I'm far from the JavaScript ecosystem expert in our department. If you are finding bugs, please do file issues. I know that our library folks are very invested in making this a good experience

mr nooli•10/21/24, 1:06 PM

Issue is, I've done it before and for a small time I was on next-on-pages but whenever I encountered an issue, I would tell prisma and then prisma would tell cf and then it'd end up going nowhere. Most of the issues that come up are bc of prisma not playing nice with cloudflare/adapters etc.

AJROP•10/21/24, 1:07 PM

Ahhh. Yeah, I can appreciate that.

Mmr nooli Ah that's really interesting, I'm unfortunately using supabase and as far as I c...

thomasgauvin•10/21/24, 1:32 PM

Surprised to see Prisma doesn't natively support a driver for postgres.js https://www.prisma.io/docs/orm/overview/databases/database-drivers

Might be something worth talking with them about

Database drivers | Prisma Documentation

Learn how Prisma connects to your database using the built-in drivers and how you can use Prisma along with other JavaScript database drivers using driver adapters (Preview)

Tthomasgauvin Surprised to see Prisma doesn't natively support a driver for postgres.js https:...

mr nooli•10/21/24, 1:33 PM

I agree - but I'm also not an expert in the field and most of this goes straight over my head

I just want my app faster! It's very fast right now with Vercel's top level of function compute power, but I feel like it could be faster and ik that the pricing won't scale.

Mmr nooli I agree - but I'm also not an expert in the field and most of this goes straight...

thomasgauvin•10/21/24, 1:37 PM

@mr nooli actually, you should look into the new support for assets on workers https://developers.cloudflare.com/workers/frameworks/framework-guides/nextjs/

you could then use the 'pg' driver as recommended ('pg' is not supported on pages, only 'postgres')

Cloudflare Docs

Next.js | Cloudflare Workers docs

Create an Next.js application and deploy it to Cloudflare Workers with Workers Assets.

mr nooli•10/21/24, 1:39 PM

Ah, so this would mean deploying on workers? Does that include the CI pipeline from pages?

mr nooli•10/21/24, 1:39 PM

(Open next right?)

Mmr nooli Ah, so this would mean deploying on workers? Does that include the CI pipeline f...

thomasgauvin•10/21/24, 1:43 PM

Yes, I would recommend following the guide I linked to get a sample app up and connecting to your database to see if it's possible for you to migrate

mr nooli•10/21/24, 1:44 PM

I’ll give it a go tonight!

mr nooli•10/21/24, 1:44 PM

But also worth me questioning if it’s worth it as I had managed to get a very small part of my app running on next on pages and it matched the page speed (the web dev page speed insights) as my Vercel app.

mr nooli•10/21/24, 1:44 PM

It should be faster in theory right?

Mmr nooli It should be faster in theory right?

thomasgauvin•10/21/24, 8:20 PM

I don't know the specifics of your stack so I can't confirm if it would be faster or not. Hyperdrive makes it faster for you to connect to a regional database from global workers

mr nooli•10/21/24, 9:22 PM

Let's give it a go and find out ig!

mr nooli•10/21/24, 10:09 PM

no dice! Just so many errors popping up!

I'm gonna give next-on-pages a go, and if there's no luck there I'm gonna just have to drop the idea of using CF for the mo

thomasgauvin•10/23/24, 3:37 PM

@mr nooli I think Pages should also have enhanced compatibility, you should use https://developers.cloudflare.com/workers/runtime-apis/nodejs/#get-started

Cloudflare Docs

Node.js compatibility | Cloudflare Workers docs

Node.js APIs available in Cloudflare Workers

AJROP•10/25/24, 1:30 PM

New blog about the Hyperdrive Tunnel integration is out. It goes into some depth, if anyone has been curious about how it works.

https://blog.cloudflare.com/elephants-in-tunnels-how-hyperdrive-connects-to-databases-inside-your-vpc-networks/

The Cloudflare Blog

Elephants in tunnels: how Hyperdrive connects to databases inside y...

Hyperdrive (Cloudflare’s globally distributed SQL connection pooler and cache) recently added support for directing database traffic from Workers across Cloudflare Tunnels. We dive deep on what it took to add this feature.

Yes, the exograph folks are around here occasionally, and that's a good write-up. While we don't of

Similar Threads