Cloudflare Developers•16mo ago

@Hyperdrive new feature just dropped!

complicated•9/26/24, 1:14 PM

MMatt Silverlock @Hyperdrive new feature just dropped!

AJR•9/26/24, 1:16 PM

Thanks Matt. Pings are hard, apparently....

AAJR @hyperdrive For all of you who've asked about connecting to your database witho...

Hard@Work•9/26/24, 1:17 PM

So if I am reading this correctly, Hyperdrive needs a public domain for the tunnel, correct? You can't point Hyperdrive directly at the ArgoTunnel domain(I don't remember the exact domain name off the top of my head)

HHard@Work So if I am reading this correctly, Hyperdrive needs a public domain for the tunn...

AJR•9/26/24, 1:17 PM

Correct.

Hard@Work•9/26/24, 1:18 PM

Would it be possible to make Tunnels/Hyperdrives on the same account be able to communicate with each other directly?

HHard@Work Would it be possible to make Tunnels/Hyperdrives on the same account be able to ...

AJR•9/26/24, 1:19 PM

In what sense? The traffic flow right now is Worker<->Hyperdrive<->DatabaseWorker<->Hyperdrive<->Database. What flow would them talking to each other look like?

AAJR In what sense? The traffic flow right now is `Worker<->Hyperdrive<->Database`. ...

Hard@Work•9/26/24, 1:21 PM

Like, where I give my Hyperdrive a Tunnel ID, then it connects directly, so that there is no outside evidence that I have a connection to my DB exposed

AJR•9/26/24, 1:21 PM

Ah. We're poking at that, it's a possible future option. CC @thomasgauvin

Ping for toast•9/26/24, 1:25 PM

How does this impact performance?

AJR•9/26/24, 1:26 PM

In almost all scenarios it's going to be very comparable to current performance. I wouldn't reach for this to solve perf problems, but I wouldn't expect it to be slower than your current setup.

AJR•9/26/24, 1:27 PM

With (really) big bulk inserts you might see some slowdowns. Please let us know if you do, as that's something we would want to address.

Ping for toast•9/26/24, 1:29 PM

I don't personally. Only case would be data import but there's an inbuilt 4 second budget for that and if that budget is surpassed there's no real consequences

PPing for toast I don't personally. Only case would be data import but there's an inbuilt 4 seco...

AJR•9/26/24, 1:30 PM

I meant if you do try it and do see slowdowns. We know some folks do larger inserts, but we don't think it should move the needle much for those. It's just an area of interest.

irajsavadlo•9/27/24, 10:20 AM

PrinceKelsey•9/27/24, 9:39 PM

How can I move hyperdrive to active ?
Is there a way to get hyperdrive connection string from the dashboard
Must hyperdrive be used with a worker ? I need the connection string to add on my vercel deployment DATABASE_URL

PPrinceKelsey How can I move hyperdrive to active ? Is there a way to get hyperdrive connectio...

DaniFoldi•9/27/24, 11:26 PM

Hyperdrive is only for Workers. It will say active if it’s received queries recently

DDaniFoldi Hyperdrive is only for Workers. It will say active if it’s received queries rece...

PrinceKelsey•9/28/24, 1:47 PM

Thank you. It won't work for my use case then.

zwily•9/29/24, 8:50 PM

if i want to use hyperdrive+tunnel to access a private db in google cloud sql, do i need to run an instance in gcp to run the tunnel daemon?

AJR•9/29/24, 11:14 PM

Yes, you'll need somewhere running cloudflared that has access to your database, for the traffic to egress from

Adar•10/2/24, 1:02 AM

Does Hyperdrive have any mechanism for rotating the connection string or are you supposed to change the ID and re-deploy workers? Tried looking around, but couldn't find anything on this.

AAdar Does Hyperdrive have any mechanism for rotating the connection string or are you...

Matt SilverlockOP•10/2/24, 10:22 AM

The connection string is unique to the Worker you call it in and can’t be used outside Hyperdrive or your account, so it can’t be leaked. cc @thomasgauvin for docs here.

AJR•10/2/24, 10:27 AM

What Matt said, the Hyperdrive ID and your connection string are not secrets. Also, if the underlying secrets (password/client secret) were leaked, those can be updated either via Wrangler or API, so you don't need to make a whole new Hyperdrive if it isn't convenient.

AAJR What Matt said, the Hyperdrive ID and your connection string are not secrets. A...

Hello, I’m Allie!•10/2/24, 10:29 AM

What would be the process if you need to rotate because you are switching providers(or for another reason)?

Hello, I’m Allie!•10/2/24, 10:29 AM

Just swap out the IDs?

HHello, I’m Allie!What would be the process if you need to rotate because you are switching provid...

AJR•10/2/24, 10:30 AM

At some point it's probably easiest to make a new one for the new provider, and then just swap out the ID in wrangler.toml, yeah.

Michael H•10/3/24, 6:27 PM

Howdy! Is there any type of logging viewable for hyperdrive private database tunnels? I have a postgresql server that is connectable directly via ssh port forwarding, via the tunnel established for hyperdrive cloudflared access tcpcloudflared access tcp, and of course on the machine itself. However, running the recommended test code (const sql = postgres(env.XXXX.connectionString);const sql = postgres(env.XXXX.connectionString);) results in a connection timeout with no further info provided. Thanks!

cosbgn•10/4/24, 10:34 AM

Did anyone managed to connect a digital ocean manage postgres to a worker?

cosbgn•10/4/24, 10:35 AM

tried with both hyperdrive and none and it just times out, on my local it works perfectly

Ccosbgn tried with both hyperdrive and none and it just times out, on my local it works ...

Michael H•10/4/24, 3:44 PM

Yeah, that's the same issue to which I was referring in my thread yesterday (hyperdrive) - I was building a full-stack in Workers Pages and the local hyperdrive dev connection worked OK, but the hyperdrive private connection seemed to be unable to talk to the tunnel, timing out in the process. I had a very productive chat with @AJR, who is looking into the issue

AJR•10/4/24, 4:03 PM

May or may not be the same issue. Is the DO connection behind a tunnel? Or publicly accessible?

Nick•10/4/24, 6:01 PM

Is there a better channel for asking questions specific to Tunnels? I'm having a connectivity issue setting up my database, but believe I've isolated it to the Tunnel.

Nick•10/4/24, 6:09 PM

I followed the instructions at https://developers.cloudflare.com/hyperdrive/configuration/connect-to-private-database/ and wanted to test database connectivity, so I allowed myself through the Access Application, and am receiving connection timeout errors while trying to connect directly from pgAdmin.

Cloudflare Docs

Connect to a private database using Tunnel | Hyperdrive

Hyperdrive can securely connect to your private databases using Cloudflare Tunnel and Cloudflare Access.

AJR•10/4/24, 11:36 PM

If the questions are specifically about Hyperdrive through a tunnel this is the channel. Have you had a chance to look through some of the debugging points in this thread?

hyperdriveHowdy! Is there any type of logging

Nick•10/4/24, 11:45 PM

I read about the logging there, and found that no logs were being generated by my connection attempts.

AJR•10/5/24, 12:27 AM

Okay. If you DM me your hyperdrive ID, I'll dig a bit when I'm back in office Monday and see what I can turn up

Nick•10/5/24, 12:37 AM

Thank you! I'll try to do a bit more troubleshooting this weekend, and will send that over if it doesn't work.

AAJR May or may not be the same issue. Is the DO connection behind a tunnel? Or publ...

cosbgn•10/5/24, 11:00 AM

public accessible: here you have a reproduction if it helps: pages-helpPostgres timesout on cloudflare:

I disabled hyperdrive for the reproduction, the the issue is the same (with/without hyperdrive)

Ccosbgn public accessible: here you have a reproduction if it helps: https://discord.com...

AJR•10/5/24, 11:37 PM

If hyperdrive is out of the equation it gets harder for me to help with that, I'm afraid. I'll ask internally on Monday and see if anyone on some of the other dev platform teams knows something about it

James Eastham•10/6/24, 8:54 PM

Hey everyone, cross post here from the Rust channel and wondering if anyone can help.

Is anyone else using Rust alongside a Postgres database with Hyperdrive? I'm getting some weird intermittent connection issues, that is like either the worker can't connect or is timing out. I'm wondering if it's something to do with this background work happening and the connection not quite being ready.

James Eastham•10/6/24, 8:54 PM

James Eastham•10/6/24, 8:54 PM

Additional context, when I run locally with a local database it works absolutely fine

James Eastham•10/6/24, 9:29 PM

Ok, so I figured this out and just following up here.

It's actually to do with using SQL prepared statements. There's a great write up here. If I switch all the queries to use query_typedquery_typed it works perfectly. If the folks building Exograph are around then thankyou

https://exograph.dev/blog/wasm-pg-explorations-2#hyperdrive-with-the-extended-query-protocol

AJR•10/6/24, 11:47 PM

Yes, the exograph folks are around here occasionally, and that's a good write-up. While we don't officially support extended protocol messages in tokio-postgres right now, their fork seems to work quite well for people.

AAJR Yes, the exograph folks are around here occasionally, and that's a good write-up...

Matt SilverlockOP•10/7/24, 1:02 PM

IIRC they merged this into mainline a little while ago: https://github.com/sfackler/rust-postgres/pull/1147

GitHub

Work with pools that don't support prepared statements by ramnivas ...

Introduce a new query_with_param_types method that allows to specify Postgres type parameters. This obviated the need to use prepared statements just to obtain parameter types for a query. It then ...

MMatt Silverlock IIRC they merged this into mainline a little while ago: https://github.com/sfack...

AJR•10/7/24, 1:05 PM

Oh cool, I missed that one entirely. Nice!

Aviation•10/7/24, 11:29 PM

I noticed that Hyperdrive now supports prepared statements, but that the docs have not been updated to reflect this change.

I'm wondering if it now supports SETSET statements, too, but the docs just isn't up to date

Cloudflare Docs

How Hyperdrive works | Hyperdrive

Connecting to traditional centralized databases from Cloudflare’s global network which consists of over 300 data center locations

presents a few challenges as queries can originate from any of these locations.

AJR•10/8/24, 12:01 AM

No, the PREPARE keyword is still not supported I'm afraid. Named prepared statements at the protocol level are supported now. This was to support the default behavior of some key drivers in the ecosystem.

thomasgauvin•10/10/24, 10:32 AM

We documented how you can use Hyperdrive bindings from Pages here!

https://developers.cloudflare.com/pages/functions/bindings/#hyperdrive-configs

Note that you will need to use Postgres.js, and set Node.js compatibility using the following configurations in your wrangler.toml

compatibility_flags = [ "nodejs_compat" ]
compatibility_date = "2024-09-23"

compatibility_flags = [ "nodejs_compat" ]
compatibility_date = "2024-09-23"

(This was mentioned in a separate thread but raising this in the main channel so all can see!)

Nick•10/10/24, 4:05 PM

Does Hyperdrive produce error logs that are visible somewhere? All I'm seeing returned to my worker is "TypeError: Cannot redefine property: query", and nothing in the Tunnel logs.

AJR•10/10/24, 5:00 PM

That's a JS error, Hyperdrive isn't throwing that one.

@Hyperdrive new feature just dropped!

Similar Threads

Similar Threads

Similar Threads