I don't personally. Only case would be data import but there's an inbuilt 4 second budget for that a

I don't personally. Only case would be data import but there's an inbuilt 4 second budget for that and if that budget is surpassed there's no real consequences

PPing for toast I don't personally. Only case would be data import but there's an inbuilt 4 seco...

AJR•9/26/24, 1:30 PM

I meant if you do try it and do see slowdowns. We know some folks do larger inserts, but we don't think it should move the needle much for those. It's just an area of interest.

irajsavadlo•9/27/24, 10:20 AM

PrinceKelsey•9/27/24, 9:39 PM

How can I move hyperdrive to active ?
Is there a way to get hyperdrive connection string from the dashboard
Must hyperdrive be used with a worker ? I need the connection string to add on my vercel deployment DATABASE_URL

PPrinceKelsey How can I move hyperdrive to active ? Is there a way to get hyperdrive connectio...

DaniFoldi•9/27/24, 11:26 PM

Hyperdrive is only for Workers. It will say active if it’s received queries recently

DDaniFoldi Hyperdrive is only for Workers. It will say active if it’s received queries rece...

PrinceKelsey•9/28/24, 1:47 PM

Thank you. It won't work for my use case then.

zwily•9/29/24, 8:50 PM

if i want to use hyperdrive+tunnel to access a private db in google cloud sql, do i need to run an instance in gcp to run the tunnel daemon?

AJR•9/29/24, 11:14 PM

Yes, you'll need somewhere running cloudflared that has access to your database, for the traffic to egress from

Adar•10/2/24, 1:02 AM

Does Hyperdrive have any mechanism for rotating the connection string or are you supposed to change the ID and re-deploy workers? Tried looking around, but couldn't find anything on this.

AAdar Does Hyperdrive have any mechanism for rotating the connection string or are you...

Matt Silverlock•10/2/24, 10:22 AM

The connection string is unique to the Worker you call it in and can’t be used outside Hyperdrive or your account, so it can’t be leaked. cc @thomasgauvin for docs here.

AJR•10/2/24, 10:27 AM

What Matt said, the Hyperdrive ID and your connection string are not secrets. Also, if the underlying secrets (password/client secret) were leaked, those can be updated either via Wrangler or API, so you don't need to make a whole new Hyperdrive if it isn't convenient.

AAJR What Matt said, the Hyperdrive ID and your connection string are not secrets. A...

Hello, I’m Allie!•10/2/24, 10:29 AM

What would be the process if you need to rotate because you are switching providers(or for another reason)?

Hello, I’m Allie!•10/2/24, 10:29 AM

Just swap out the IDs?

HHello, I’m Allie!What would be the process if you need to rotate because you are switching provid...

AJR•10/2/24, 10:30 AM

At some point it's probably easiest to make a new one for the new provider, and then just swap out the ID in wrangler.toml, yeah.

Michael H•10/3/24, 6:27 PM

Howdy! Is there any type of logging viewable for hyperdrive private database tunnels? I have a postgresql server that is connectable directly via ssh port forwarding, via the tunnel established for hyperdrive

cloudflared access tcp

cloudflared access tcp

, and of course on the machine itself. However, running the recommended test code (

const sql = postgres(env.XXXX.connectionString);

const sql = postgres(env.XXXX.connectionString);

) results in a connection timeout with no further info provided. Thanks!

cosbgn•10/4/24, 10:34 AM

Did anyone managed to connect a digital ocean manage postgres to a worker?

cosbgn•10/4/24, 10:35 AM

tried with both hyperdrive and none and it just times out, on my local it works perfectly

Ccosbgn tried with both hyperdrive and none and it just times out, on my local it works ...

Michael H•10/4/24, 3:44 PM

Yeah, that's the same issue to which I was referring in my thread yesterday (hyperdrive) - I was building a full-stack in Workers Pages and the local hyperdrive dev connection worked OK, but the hyperdrive private connection seemed to be unable to talk to the tunnel, timing out in the process. I had a very productive chat with @AJR, who is looking into the issue

AJR•10/4/24, 4:03 PM

May or may not be the same issue. Is the DO connection behind a tunnel? Or publicly accessible?

Nick•10/4/24, 6:01 PM

Is there a better channel for asking questions specific to Tunnels? I'm having a connectivity issue setting up my database, but believe I've isolated it to the Tunnel.

Nick•10/4/24, 6:09 PM

I followed the instructions at https://developers.cloudflare.com/hyperdrive/configuration/connect-to-private-database/ and wanted to test database connectivity, so I allowed myself through the Access Application, and am receiving connection timeout errors while trying to connect directly from pgAdmin.

Cloudflare Docs

Connect to a private database using Tunnel | Hyperdrive

Hyperdrive can securely connect to your private databases using Cloudflare Tunnel and Cloudflare Access.

AJR•10/4/24, 11:36 PM

If the questions are specifically about Hyperdrive through a tunnel this is the channel. Have you had a chance to look through some of the debugging points in this thread?

hyperdriveHowdy! Is there any type of logging

Nick•10/4/24, 11:45 PM

I read about the logging there, and found that no logs were being generated by my connection attempts.

AJR•10/5/24, 12:27 AM

Okay. If you DM me your hyperdrive ID, I'll dig a bit when I'm back in office Monday and see what I can turn up

Nick•10/5/24, 12:37 AM

Thank you! I'll try to do a bit more troubleshooting this weekend, and will send that over if it doesn't work.

AAJR May or may not be the same issue. Is the DO connection behind a tunnel? Or publ...

cosbgn•10/5/24, 11:00 AM

public accessible: here you have a reproduction if it helps: pages-helpPostgres timesout on cloudflare:

I disabled hyperdrive for the reproduction, the the issue is the same (with/without hyperdrive)

Ccosbgn public accessible: here you have a reproduction if it helps: https://discord.com...

AJR•10/5/24, 11:37 PM

If hyperdrive is out of the equation it gets harder for me to help with that, I'm afraid. I'll ask internally on Monday and see if anyone on some of the other dev platform teams knows something about it

James Eastham•10/6/24, 8:54 PM

Hey everyone, cross post here from the Rust channel and wondering if anyone can help.

Is anyone else using Rust alongside a Postgres database with Hyperdrive? I'm getting some weird intermittent connection issues, that is like either the worker can't connect or is timing out. I'm wondering if it's something to do with this background work happening and the connection not quite being ready.

James Eastham•10/6/24, 8:54 PM

James Eastham•10/6/24, 8:54 PM

Additional context, when I run locally with a local database it works absolutely fine

James Eastham•10/6/24, 9:29 PM

Ok, so I figured this out and just following up here.

It's actually to do with using SQL prepared statements. There's a great write up here. If I switch all the queries to use

query_typed

query_typed

it works perfectly. If the folks building Exograph are around then thankyou

https://exograph.dev/blog/wasm-pg-explorations-2#hyperdrive-with-the-extended-query-protocol

AJR•10/6/24, 11:47 PM

Yes, the exograph folks are around here occasionally, and that's a good write-up. While we don't officially support extended protocol messages in tokio-postgres right now, their fork seems to work quite well for people.

AAJR Yes, the exograph folks are around here occasionally, and that's a good write-up...

Matt Silverlock•10/7/24, 1:02 PM

IIRC they merged this into mainline a little while ago: https://github.com/sfackler/rust-postgres/pull/1147

GitHub

Work with pools that don't support prepared statements by ramnivas ...

Introduce a new query_with_param_types method that allows to specify Postgres type parameters. This obviated the need to use prepared statements just to obtain parameter types for a query. It then ...

MMatt Silverlock IIRC they merged this into mainline a little while ago: https://github.com/sfack...

AJR•10/7/24, 1:05 PM

Oh cool, I missed that one entirely. Nice!

Aviation•10/7/24, 11:29 PM

I noticed that Hyperdrive now supports prepared statements, but that the docs have not been updated to reflect this change.

I'm wondering if it now supports

SET

SET

statements, too, but the docs just isn't up to date

Cloudflare Docs

How Hyperdrive works | Hyperdrive

Connecting to traditional centralized databases from Cloudflare’s global network which consists of over 300 data center locations

presents a few challenges as queries can originate from any of these locations.

AJR•10/8/24, 12:01 AM

No, the PREPARE keyword is still not supported I'm afraid. Named prepared statements at the protocol level are supported now. This was to support the default behavior of some key drivers in the ecosystem.

thomasgauvin•10/10/24, 10:32 AM

We documented how you can use Hyperdrive bindings from Pages here!

https://developers.cloudflare.com/pages/functions/bindings/#hyperdrive-configs

Note that you will need to use Postgres.js, and set Node.js compatibility using the following configurations in your wrangler.toml

compatibility_flags = [ "nodejs_compat" ]
compatibility_date = "2024-09-23"

compatibility_flags = [ "nodejs_compat" ]
compatibility_date = "2024-09-23"

(This was mentioned in a separate thread but raising this in the main channel so all can see!)

Nick•10/10/24, 4:05 PM

Does Hyperdrive produce error logs that are visible somewhere? All I'm seeing returned to my worker is "TypeError: Cannot redefine property: query", and nothing in the Tunnel logs.

AJR•10/10/24, 5:00 PM

That's a JS error, Hyperdrive isn't throwing that one.

AJR•10/10/24, 5:01 PM

Example: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Errors/Cant_redefine_property

MDN Web Docs

TypeError: can't redefine non-configurable property "x" - JavaScrip...

The JavaScript exception "can't redefine non-configurable property" occurs when it was
attempted to redefine a property, but that property is non-configurable.

AJR•10/10/24, 5:02 PM

To answer the actual question, though, we return all errors from the origin server, plus selected Hyperdrive-specific errors, as Postgres errors to your PG client. The recommended path here is to handle errors from your driver, and either log or condition on them as needed. Logging them will have them populated with all the other Worker logs

bennycondon•10/11/24, 10:48 AM

Sorry if this is a naive question. I saw on How Hyperdrive works | Hyperdrive that

SET

SET

statements aren't supported. In that case, how would I perform an

UPDATE

UPDATE

statement without using a

SET

SET

statement?

Cloudflare Docs

How Hyperdrive works | Hyperdrive

Connecting to traditional centralized databases from Cloudflare’s global network which consists of over 300 data center locations

presents a few challenges as queries can originate from any of these locations.

Bbennycondon Sorry if this is a naive question. I saw on [How Hyperdrive works | Hyperdrive](...

kian•10/11/24, 11:13 AM

You might be confusing the

SET

SET

keyword within an

UPDATE

UPDATE

statement with the unrelated

SET

SET

statement

kian•10/11/24, 11:13 AM

The

SET

SET

statement is for changing run-time configuration: https://www.postgresql.org/docs/current/sql-set.html

bennycondon•10/11/24, 11:15 AM

I was confusing the two. Thanks so much for the help kian!

yevgen•10/14/24, 10:38 AM

New how-to configuration for Hyperdrive https://developers.cloudflare.com/hyperdrive/configuration/rotate-credentials/

Cloudflare Docs

Rotating database credentials | Hyperdrive

You can change the connection information and credentials of your Hyperdrive configuration in one of two ways:

Aalgads I assume the limitation on SETs is only if they are issued as separate commands....

AJR•10/18/24, 3:54 PM

This is a scenario where the Postgres syntax is a bit unfortunate. SET in the context of UPDATE commands (keyword) and SET in the context of session state management (statement) are entirely unrelated, from the message protocol all the way down to the internals and back. They just happen to share the same name.

SET ROLE and such that are for session state management are going to be a problem, almost regardless of how you wrap them. We've done work to make the system more resilient, so you might get lucky, but generally I'd avoid those.

The SET keyword in the context of UPDATE statements is fine, go nuts with it.

AJR•10/18/24, 3:58 PM

I will say that the root cause of problems here is polluting state on shared/pooled connections, so if you scope that state pollution such that by the time your query resolves everything is back how it started, it seems to me that you're less likely to have problems. If you can make it work for your use case, we certainly aren't going to stop you. But it's not something we support so, y'know, we won't be able to help you much. And we do reserve the right to change things in ways that might break that workflow if we need to. Kind of "undefined behavior, in the programming languange sense" territory.

thomasgauvin•10/18/24, 4:39 PM

Hey folks! It's great seeing all the feedback for Hyperdrive with private databases using tunnels, if you've been trying out and using it for the past few weeks, we want to learn more about your experience.

Likewise, if you've been using Hyperdrive, would love to chat 20-30 mins to better understand how you're using Hyperdrive, what are some of your asks around feature updates, how your performance has been, etc.

We have a bunch of improvements under way to improve compatibility (custom certs

) and performance and we want to make sure we're taking in your feedback! Here's my calendar -> https://calendar.app.google/b3w5dNYQj9YANw449

Appointments

dvddvd300•10/19/24, 5:07 AM

Hi, its there an updated timeline for MySQL support? (like early/beta/unstable access)
(MariaDB 10, 12GB database, ~ 200 Read Query/day)
tks

I don't personally. Only case would be data import but there's an inbuilt 4 second budget for that a

Similar Threads

Similar Threads

Similar Threads