Cloudflare Developers•12mo ago

During local dev, is Hyperdrive supposed to do connection pooling?

Ddave During local dev, is Hyperdrive supposed to do connection pooling?

Hello, I’m Allie!•1/14/25, 2:40 AM

I forget, isn’t Hyperdrive completely disabled in local?

Hello, I’m Allie!•1/14/25, 2:41 AM

I thought it just passed in the URL directly

HHello, I’m Allie!I forget, isn’t Hyperdrive completely disabled in local?

daveOP•1/14/25, 3:09 AM

It gives a string like

4870f521c22c2295fdfaceef9b4aeca5.hyperdrive.local

4870f521c22c2295fdfaceef9b4aeca5.hyperdrive.local

for the host.

daveOP•1/14/25, 3:59 PM

Is there a way to see if a connection to Hyperdrive is working without binding it to a Worker?

AJR•1/14/25, 4:16 PM

We validate connections at create time. If the credentials/hostname provided don't connect successfully you'll get an error and it'll fail to create.

AAJR We validate connections at create time. If the credentials/hostname provided do...

daveOP•1/14/25, 4:37 PM

Can I validate a connection after it's created?

AJR•1/14/25, 4:42 PM

We re run it on updates. You could increase your cache ttl by 1 second or something.

Alternatively, you can bind the same Hyperdrive to multiple workers. Spin up a hello world worker and test it out that way with some small queries?

AAJR We re run it on updates. You could increase your cache ttl by 1 second or someth...

daveOP•1/14/25, 5:45 PM

Oh nice, so if an update works, that means for sure that the connection is good?

Ddave Oh nice, so if an update works, that means for sure that the connection is good?

AJR•1/14/25, 5:47 PM

Yep, that's correct. The point is to avoid having folks change a password or hostname or something and have a typo that they don't notice til their alerts blow up, but it can be used for this too

AJR•1/14/25, 5:48 PM

You can confirm this on the DB side by looking at logins, if you have that monitoring set up.

AJR•1/14/25, 5:54 PM

Hey everyone. I'm looking for some beta testers. We have a fairly large change going out soon, that will pretty dramatically improve performance for Hyperdrives with wide geographic distributions of their users.

If you're interested, please respond in the thread I'm about to create indicating your interest, and which Hyperdrive IDs you'd like me to enroll in this beta test. I expect that we'll get this rolling early next week.

Walshy•1/14/25, 6:00 PM

@Hyperdrive

if you want access to beta, see above!

oritwoen•1/14/25, 6:03 PM

I see that this is beta access to active hyperdrive instances. I am eagerly awaiting the possibility of connecting mysql. I have a very large service made on Nuxt 3 with Headless for WordPress and deploys on Cloudflare and hyperdrive would probably reduce costs and improve the performance of the site.

Ooritwoen I see that this is beta access to active hyperdrive instances. I am eagerly awai...

AJR•1/14/25, 6:10 PM

We hear you.

WWalshy @Hyperdrive 👀 if you want access to beta, see above!

Peppy•1/14/25, 6:17 PM

chapel1337•1/14/25, 8:42 PM

Cchimame Has Hyperdrive been updated recently? The number of connections to our database...

Isaac McFadyen•1/16/25, 12:25 AM

There is a beta going on at the moment but I believe it was opt-in meaning if you haven't given your account IDs you shouldn't be in the beta

Isaac McFadyen•1/16/25, 12:26 AM

One of the Hyperdrive engineers could tell you more whenever they see this, probably. Can you grab your account ID and hyperdrive ID (It's safe to share publicly)?

AJR•1/16/25, 12:28 AM

No, the beta code has been sitting out for weeks and is gated, it's not that.

Let me check our monitoring when I can get back to my desk. No outages or anything like that but I can see if we're seeing wider variation in the region right now.

AAJR No, the beta code has been sitting out for weeks and is gated, it's not that. L...

Isaac McFadyen•1/16/25, 12:28 AM

Aha I was going to ping you but saw you were offline, thanks

Maraciuca•1/16/25, 12:31 AM

can anyone please take a look at https://github.com/ARC-Solutions/IntelliQ-V2/tree/main/apps/dashboard whenever they have time? ive been trying to solve my issue for a week now and ive went down the route of:
Tried Deployment on CF Pages: didnt know I can get the context inside my api routes
Tried Deployment on CF Workers: my env vars weren't working
Went Back to CF Pages Again: found out about bindings, getting the context for the Hyperdrive Environment VAR, but now i've been trying to connect somehow through postgres.js but that doesn't work even though i have the nodejs_compat flag

my techstack is:
Supabase
NextJS 14
Node
Drizzle
CF Pages

GitHub

IntelliQ-V2/apps/dashboard at main · ARC-Solutions/IntelliQ-V2

Contribute to ARC-Solutions/IntelliQ-V2 development by creating an account on GitHub.

Cchimame Has Hyperdrive been updated recently? The number of connections to our database...

AJR•1/16/25, 12:40 AM

Taking a look now. No wider impacts, checking on traffic isolated to your region

Cchimame Has Hyperdrive been updated recently? The number of connections to our database...

AJR•1/16/25, 12:57 AM

We are registering very large variations in your traffic over this time period. It ranges from about 3k client connections per hour at the low end, with a steady state of about 25k client connections per hour. At the time shown, we register an approximately 10x increase in traffic for you, spiking up to about 400k client connections per hour. The connection exhaustion and performance variation you're seeing seems to be a result of this, and the connection exhaustion seems to be expected.

We have some interventions that are available for helping this, please discuss with @thomasgauvin about what we can do for you here.

AJR•1/16/25, 1:22 AM

Yes it does mean that, though somewhat indirectly. I'm limited in how much I can discuss that architecture at the moment, looking forward to a rather longer blog post about it when I can. Please message me and we'll get you sorted out during next week's release.

In the mean time you can create another hyperdrive, split your load across it, and DM me the ID so I can pin it for you tonight

IIsaac McFadyen If so then likely, but if you're using some sort of global/persistent connection...

Mitya•1/16/25, 12:59 PM

This caught my eye. What would a global persistent connection between distinct worker requests look like, @Isaac | AS213339? I thought everything shut down, including a DB connection, when a Worker had sent its response. My app connects fresh for each request - in truth I didn't know there was another way.

MMitya This caught my eye. What would a global persistent connection between distinct w...

Isaac McFadyen•1/16/25, 1:15 PM

In general it's not a good idea when using Hyperdrive. If not using Hyperdrive it might be an OK idea... but anything in the global scope of a Worker is recommended against generally.

Isaac McFadyen•1/16/25, 1:17 PM

It would be something like:

let db: DbConnection | null = null;
let dbUrl: string | null = null:

// URL would come from `env.URL` or similar secret stored with the Worker
const getDb(url: string) => {
    if(dbUrl !== url || db = null) {
        db = new DbConnection(url);
        dbUrl = url;
    }
    return db;
};

let db: DbConnection | null = null;
let dbUrl: string | null = null:

// URL would come from `env.URL` or similar secret stored with the Worker
const getDb(url: string) => {
    if(dbUrl !== url || db = null) {
        db = new DbConnection(url);
        dbUrl = url;
    }
    return db;
};

Isaac McFadyen•1/16/25, 1:17 PM

Theoretically the Worker shouldn't stay alive if the DB URL is changed but that's not guaranteed by the Workers platform so that's why the extra URL check.

Isaac McFadyen•1/16/25, 1:18 PM

But your current "reconnect on every request" is a good approach and recommended with Hyperdrive.

Ddave Oh nice, so if an update works, that means for sure that the connection is good?

AJR•1/17/25, 8:55 PM

Normally I wouldn't mention a change like this since it's kinda just internals, but given that it was just discussed publicly/recently and it might be useful to have it here:

If you're intentionally trying to confirm your connection will still work through Hyperdrive, it won't trigger on updates done via the dash anymore (can't change anything via dash that would affect our ability to connect, so it was mostly wasted traffic). Use the PUT API. https://developers.cloudflare.com/api-next/resources/hyperdrive/subresources/configs/

Cloudflare API | Hyperdrive › Configs

Interact with Cloudflare's products and services via the Cloudflare API

Shemesh•1/19/25, 8:38 AM

Hey there

I'm using Hyperdrive with a TimescaleDB instance located in eu-west region.
I have a queue worker that gets a batch of 10 messages and inserts them to the DB.
For some reason, the insert logic(creating Postgres.js instance + query) takes ~1.5s.
However:

In the Hyperdrive dashboard, i can see that the query latency is ~150ms(connection latency negligible).
In Timescale dashboard, the query time is also pretty fast.

Any idea?
Thanks!

SShemesh Hey there 🙂 I'm using Hyperdrive with a TimescaleDB instance located in eu-west...

AJR•1/19/25, 12:59 PM

This feels like a network routing issue. You're in the APAC region?

I see you signed up for the beta above, if so that will address this issue. If this is time sensitive please DM me and we'll get you sorted out.

AAJR This feels like a network routing issue. You're in the APAC region? I see you ...

Shemesh•1/19/25, 1:00 PM

Nope, eu-west region

AJR•1/19/25, 1:09 PM

Hmmm. Yeah, please DM me the ID. I can take a look when I'm back at my desk.

Uriel•1/21/25, 4:49 PM

Hi, can I check what might cause a PostgresError: Internal error. I'm trying to add hyperdrive to neon db instead of connecting directly.

daveOP•1/22/25, 12:42 PM

With AWS RDS, how do I give Hyperdrive the CA needed so I can use SSL?

AAJR We currently cannot support SSL handshake from local dev. It's on our roadmap, ...

daveOP•1/22/25, 1:06 PM

Is this a limitation of Hyperdrive, or local Workers in general? For context, I tried to use postgres.js without hyperdrive locally, and SSL is failing.

AJR•1/22/25, 1:14 PM

That's a fair question and I'm honestly not sure yet, part of the work to support it may help other services too. Local dev goes through miniflare same as the rest of the Workers services, which is open source.

Ddave With AWS RDS, how do I give Hyperdrive the CA needed so I can use SSL?

AJR•1/22/25, 1:15 PM

Work to support custom CAs is in flight but not publicly available yet.

AAJR Work to support custom CAs is in flight but not publicly available yet.

daveOP•1/22/25, 1:16 PM

Is Hyperdrive in production connecting to RDS without SSL right now then? (Sorry, in a government building right now and can’t pull out my laptop to test.

)

Ddave Is Hyperdrive in production connecting to RDS without SSL right now then? (Sorry...

AJR•1/22/25, 1:16 PM

It's connecting with SSL, exclusively, using the standard suite of Mozilla Web PKI CAs

Mitya•1/23/25, 2:04 PM

Struggling to set up a Tunnel to my HD. I get as far as step 7:

Once the command has finished running, your connector will appear in Zero Trust.

This doesn't happen for me - it stays empty ("No connectors installed"), even though the command ran successfully in my terminal. Any idea what I could be doing wrong?

MMitya Struggling to set up a Tunnel to my HD. I get as far as [step 7](https://develop...

AJR•1/23/25, 6:14 PM

That command runs cloudflared as a service. Generally for long-lived things that's what you want, but it may need to be started by your service manager (e.g. systemdsystemd or similar). For initial setup it might be easier to do it as cloudflared tunnel run <NAME>cloudflared tunnel run <NAME>

ZT team has pretty good docs on this, that seems like it'll be most helpful for you here.
https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/get-started/create-local-tunnel/#6-run-the-tunnel

Cloudflare Docs

Create a locally-managed tunnel (CLI) · Cloudflare Zero Trust docs

Follow this step-by-step guide to get your first tunnel up and running using the CLI.

Mitya•1/23/25, 7:51 PM

Thanks, I'll have a read of that. So it seems I was on the wrong lines. I was just following the guide in the docs (though I did wonder why I needed to download and install an MSI to set up a tunnel to a remote DB.) I'm trying to switch from IP-based connection to my Azure-hosted DB to use tunnels instead. Is that possible?

MMitya Thanks, I'll have a read of that. So it seems I was on the wrong lines. I was ju...

Isaac McFadyen•1/23/25, 8:03 PM

You'll need the Tunnel within the same VPC/network/whatever Azure calls it as the DB.

Isaac McFadyen•1/23/25, 8:04 PM

I believe with Azure you can either change the DB to be within the same VPC as a VM instance or use Private Connect (I think that's what they call it?)

Isaac McFadyen•1/23/25, 8:04 PM

Regardless, the Tunnel needs to be able to contact the Azure DB.

let db: DbConnection | null = null; let dbUrl: string | null = null: // URL would come from `env.URL` or similar secret stored with the Worker const getDb(url: string) => { if(dbUrl !== url || db = null) { db = new DbConnection(url); dbUrl = url; } return db; };

During local dev, is Hyperdrive supposed to do connection pooling?

Similar Threads

Similar Threads

Similar Threads