This caught my eye. What would a global persistent connection between distinct worker requests look

This caught my eye. What would a global persistent connection between distinct worker requests look like, @Isaac | AS213339? I thought everything shut down, including a DB connection, when a Worker had sent its response. My app connects fresh for each request - in truth I didn't know there was another way.

MMitya This caught my eye. What would a global persistent connection between distinct w...

Isaac McFadyen•1/16/25, 1:15 PM

In general it's not a good idea when using Hyperdrive. If not using Hyperdrive it might be an OK idea... but anything in the global scope of a Worker is recommended against generally.

Isaac McFadyen•1/16/25, 1:17 PM

It would be something like:

let db: DbConnection | null = null;
let dbUrl: string | null = null:

// URL would come from `env.URL` or similar secret stored with the Worker
const getDb(url: string) => {
    if(dbUrl !== url || db = null) {
        db = new DbConnection(url);
        dbUrl = url;
    }
    return db;
};

let db: DbConnection | null = null;
let dbUrl: string | null = null:

// URL would come from `env.URL` or similar secret stored with the Worker
const getDb(url: string) => {
    if(dbUrl !== url || db = null) {
        db = new DbConnection(url);
        dbUrl = url;
    }
    return db;
};

Isaac McFadyen•1/16/25, 1:17 PM

Theoretically the Worker shouldn't stay alive if the DB URL is changed but that's not guaranteed by the Workers platform so that's why the extra URL check.

Isaac McFadyen•1/16/25, 1:18 PM

But your current "reconnect on every request" is a good approach and recommended with Hyperdrive.

Ddave Oh nice, so if an update works, that means for sure that the connection is good?

AJR•1/17/25, 8:55 PM

Normally I wouldn't mention a change like this since it's kinda just internals, but given that it was just discussed publicly/recently and it might be useful to have it here:

If you're intentionally trying to confirm your connection will still work through Hyperdrive, it won't trigger on updates done via the dash anymore (can't change anything via dash that would affect our ability to connect, so it was mostly wasted traffic). Use the PUT API. https://developers.cloudflare.com/api-next/resources/hyperdrive/subresources/configs/

Cloudflare API | Hyperdrive › Configs

Interact with Cloudflare's products and services via the Cloudflare API

Shemesh•1/19/25, 8:38 AM

Hey there

I'm using Hyperdrive with a TimescaleDB instance located in eu-west region.
I have a queue worker that gets a batch of 10 messages and inserts them to the DB.
For some reason, the insert logic(creating Postgres.js instance + query) takes ~1.5s.
However:

In the Hyperdrive dashboard, i can see that the query latency is ~150ms(connection latency negligible).
In Timescale dashboard, the query time is also pretty fast.

Any idea?
Thanks!

SShemesh Hey there 🙂 I'm using Hyperdrive with a TimescaleDB instance located in eu-west...

AJR•1/19/25, 12:59 PM

This feels like a network routing issue. You're in the APAC region?

I see you signed up for the beta above, if so that will address this issue. If this is time sensitive please DM me and we'll get you sorted out.

AAJR This feels like a network routing issue. You're in the APAC region? I see you ...

Shemesh•1/19/25, 1:00 PM

Nope, eu-west region

AJR•1/19/25, 1:09 PM

Hmmm. Yeah, please DM me the ID. I can take a look when I'm back at my desk.

Uriel•1/21/25, 4:49 PM

Hi, can I check what might cause a PostgresError: Internal error. I'm trying to add hyperdrive to neon db instead of connecting directly.

dave•1/22/25, 12:42 PM

With AWS RDS, how do I give Hyperdrive the CA needed so I can use SSL?

AAJR We currently cannot support SSL handshake from local dev. It's on our roadmap, ...

dave•1/22/25, 1:06 PM

Is this a limitation of Hyperdrive, or local Workers in general? For context, I tried to use postgres.js without hyperdrive locally, and SSL is failing.

AJR•1/22/25, 1:14 PM

That's a fair question and I'm honestly not sure yet, part of the work to support it may help other services too. Local dev goes through miniflare same as the rest of the Workers services, which is open source.

Ddave With AWS RDS, how do I give Hyperdrive the CA needed so I can use SSL?

AJR•1/22/25, 1:15 PM

Work to support custom CAs is in flight but not publicly available yet.

AAJR Work to support custom CAs is in flight but not publicly available yet.

dave•1/22/25, 1:16 PM

Is Hyperdrive in production connecting to RDS without SSL right now then? (Sorry, in a government building right now and can’t pull out my laptop to test.

)

Ddave Is Hyperdrive in production connecting to RDS without SSL right now then? (Sorry...

AJR•1/22/25, 1:16 PM

It's connecting with SSL, exclusively, using the standard suite of Mozilla Web PKI CAs

MityaOP•1/23/25, 2:04 PM

Struggling to set up a Tunnel to my HD. I get as far as step 7:

Once the command has finished running, your connector will appear in Zero Trust.

This doesn't happen for me - it stays empty ("No connectors installed"), even though the command ran successfully in my terminal. Any idea what I could be doing wrong?

MMitya Struggling to set up a Tunnel to my HD. I get as far as [step 7](https://develop...

AJR•1/23/25, 6:14 PM

That command runs cloudflared as a service. Generally for long-lived things that's what you want, but it may need to be started by your service manager (e.g.

systemd

systemd

or similar). For initial setup it might be easier to do it as

cloudflared tunnel run <NAME>

cloudflared tunnel run <NAME>

ZT team has pretty good docs on this, that seems like it'll be most helpful for you here.
https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/get-started/create-local-tunnel/#6-run-the-tunnel

Cloudflare Docs

Create a locally-managed tunnel (CLI) · Cloudflare Zero Trust docs

Follow this step-by-step guide to get your first tunnel up and running using the CLI.

MityaOP•1/23/25, 7:51 PM

Thanks, I'll have a read of that. So it seems I was on the wrong lines. I was just following the guide in the docs (though I did wonder why I needed to download and install an MSI to set up a tunnel to a remote DB.) I'm trying to switch from IP-based connection to my Azure-hosted DB to use tunnels instead. Is that possible?

MMitya Thanks, I'll have a read of that. So it seems I was on the wrong lines. I was ju...

Isaac McFadyen•1/23/25, 8:03 PM

You'll need the Tunnel within the same VPC/network/whatever Azure calls it as the DB.

Isaac McFadyen•1/23/25, 8:04 PM

I believe with Azure you can either change the DB to be within the same VPC as a VM instance or use Private Connect (I think that's what they call it?)

Isaac McFadyen•1/23/25, 8:04 PM

Regardless, the Tunnel needs to be able to contact the Azure DB.

MityaOP•1/23/25, 8:04 PM

Eeek, sounds complex. I'm no network administrator. Appreciate the info, though, I'll do some reading.

MMitya Eeek, sounds complex. I'm no network administrator. Appreciate the info, though,...

Isaac McFadyen•1/23/25, 8:05 PM

Hyperdrive can either connect over the public internet, or via a Tunnel. If you use the public internet your DB security will be weakened a bit because you'll need give it a public IP or similar. If you use a Tunnel then you can use the private IP of the DB, but whatever is running the Tunnel needs to be able to contact that private IP (that's why I mentioned VPC).

MityaOP•1/23/25, 8:07 PM

Yeah at the moment my HD is connecting over public IP, which isn't ideal for the reason you mention. When you say "whatever is running the tunnel," what does that mean? Isn't that Cloudflare? I realise I'm quite ignorant here.

AJR•1/23/25, 8:07 PM

(we are working on getting over to the cloudflare IPs, to get a door #3 where you allowlist just those and could skip setting up the tunnel. it's taking a while to land, but is still coming)

MMitya Yeah at the moment my HD is connecting over public IP, which isn't ideal for the...

AJR•1/23/25, 8:08 PM

The egress end of the tunnel, where traffic leaves it and goes to your DB, has to run somewhere. That's what

cloudflared

cloudflared

does, and that's what needs to be reachable by your DB.

Isaac McFadyen•1/23/25, 8:09 PM

^ Usually that's a VM instance hosted by you, probably on Azure as otherwise there's no way for it to talk to the DB.

Isaac McFadyen•1/23/25, 8:09 PM

That's what the MSI you were using was for. If you installed that on Windows Server (or the Linux one on a Linux machine) then Hyperdrive would route through that server to contact your DB.

MMitya Thanks, I'll have a read of that. So it seems I was on the wrong lines. I was ju...

AJR•1/23/25, 8:09 PM

Just checked, @thomasgauvin did specifically test that this works with Azure, btw.

MityaOP•1/23/25, 8:10 PM

Haha - I thought I was supposed to literally install that on my laptop. OK, it's becoming clearer.

thomasgauvin•1/23/25, 8:11 PM

@Mitya I have some time now if I can help answer some questions https://calendar.app.google/9C8a9s8j37WCW1e58

Appointments

Jeff•1/23/25, 8:54 PM

@thomasgauvin I'm actually implementing as well, are you available for another calendar slot?

thomasgauvin•1/23/25, 8:58 PM

Also, if anyone else wants to book a call w/ me looking for clarity on any Hyperdrive feature, any other Hyperdrive questions, things you think we should know about, or just want to share with us what you're building (we want to know!), book a chat here: https://calendar.app.google/9C8a9s8j37WCW1e58

Appointments

森

森@VerseLink (RW)•1/23/25, 9:12 PM

~~only for hyperdrive?~~

AAJR (we are working on getting over to the cloudflare IPs, to get a door #3 where yo...

dave•1/23/25, 10:18 PM

Please ping me when this happens

Jeff•1/23/25, 10:24 PM

learned something new today: you can skip a lot of the cloudflare one configuration if you just click hyperdrive -> private database in the CF dashboard. The only part you need to do is get cloudflared running in your network

Zn4rK•1/24/25, 11:12 AM

Are cached Hyperdrive queries invalidated if a write occurs?

ZZn4rK Are cached Hyperdrive queries invalidated if a write occurs?

AJR•1/24/25, 11:40 AM

Discussed recently:
hyperdrive

AAJR Discussed recently: https://discord.com/channels/595317990191398933/115055798623...

Zn4rK•1/24/25, 11:41 AM

Ty!

dave•1/25/25, 1:59 PM

Is there any plans to allow using Hyperdrive as a pooler for non-Workers connections?

Ddave Is there any plans to allow using Hyperdrive as a pooler for non-Workers connect...

AJR•1/25/25, 8:03 PM

Plans is a strong word. It's something we've discussed internally, but no firm commitments yet. If it's something you'd be interested in, I'll make sure to filter that back to our product folks.

AAJR No. You can cache-bust individual queries by including any volatile function in...

sid•1/25/25, 8:08 PM

Im using hyperdrive to cache queries to my supabase db, is there a way to cache bust a particular document. E.g. if I run a volatile query like SELECT random() FROM table WHERE id = 1234SELECT random() FROM table WHERE id = 1234 will the cache only be invalidated for that record or the whole table?

Ssid Im using hyperdrive to cache queries to my supabase db, is there a way to cache ...

AJR•1/25/25, 8:10 PM

We don't do any purging. If you do SELECT random(), * FROM table WHERE id = 1234SELECT random(), * FROM table WHERE id = 1234 you'll get uncached results for that query. What's in the cache won't be touched either way.

sid•1/25/25, 8:12 PM

ah okay understood

AAJR Plans is a strong word. It's something we've discussed internally, but no firm ...

dave•1/25/25, 8:45 PM

Would be quite interested.

AAJR Hey everyone. I'm looking for some beta testers. We have a fairly large change...

AJR•1/27/25, 2:39 PM

Hey everyone. Beta test has started, last call for folks who want to join.

Please reply in thread with your Hyperdrive ID if so. Thanks!

AAJR We don't do any purging. If you do `SELECT random(), * FROM table WHERE id = 123...

dave•1/27/25, 2:53 PM

Do you optimize this type of query by skipping even checking the cache? Or is there still a small delay due to a cache lookup miss?

Ddave Do you optimize this type of query by skipping even checking the cache? Or is th...

AJR•1/27/25, 2:54 PM

We skip checking the cache. The delay for doing so is incredibly small, but no reason to waste the traffic.

let db: DbConnection | null = null; let dbUrl: string | null = null: // URL would come from `env.URL` or similar secret stored with the Worker const getDb(url: string) => { if(dbUrl !== url || db = null) { db = new DbConnection(url); dbUrl = url; } return db; };

This caught my eye. What would a global persistent connection between distinct worker requests look

Similar Threads

Similar Threads

Similar Threads