version of cloudflared is `2024.9.1` is that an issue? im using the cloudflare-operator for k8s and

version of cloudflared is

2024.9.1

2024.9.1

2024.9.1

2024.9.1 is that an issue? im using the cloudflare-operator for k8s and unsure how to upgrade the binary

mattOP•3/2/25, 11:06 AM

code=2015
Failed to connect to the provided database: Connecting to database via Cloudflare Tunnel failed: 403 Forbidden

code=2015
Failed to connect to the provided database: Connecting to database via Cloudflare Tunnel failed: 403 Forbidden

mattOP•3/2/25, 11:23 AM

what the heck is going on. this is the tunnel configuration

- hostname: postgres.unweb.io
      service: tcp://postgresql-f4b1ec99.cloud-prod.svc:5432
      originRequest:
        noTLSVerify: true

- hostname: postgres.unweb.io
      service: tcp://postgresql-f4b1ec99.cloud-prod.svc:5432
      originRequest:
        noTLSVerify: true

ive confirmed multiple boxes in the cluster can access via psql, and im still not seeing anything in the cloudflared logs

mattOP•3/2/25, 12:10 PM

none of this has worked:

enable SSL on postgres
disable TLS verification for tunnel backend due to self-signed cert
upgrade version of cloudflared to 2025.2.12025.2.1

meanwhile the tunnel diagnostics are showing absolutely zero information. i cannot see hyperdrive attempting to make the connection. i've torn down the tunnel and recreated from scratch, nothing seems to work and the inability to see any diagnostics or logs is extremely frustrating

Mmatt none of this has worked: * enable SSL on postgres * disable TLS verification for...

AJR•3/2/25, 12:47 PM

Run the tunnel with loglevel=debug to see individual connection attempts and responses.

AJR•3/2/25, 12:48 PM

403 is usually an Access refusal. Check the logs on your access policy too? For service tokens you'll need to query them via API, the tunnel dash doesn't show service access login attempts

Mmatt does hyperdrive over cloudflared tunnel require dns records?

AJR•3/2/25, 12:48 PM

Yes it does

Mmatt version of cloudflared is `2024.9.1` is that an issue? im using the cloudflare-o...

AJR•3/2/25, 12:48 PM

Should be fine with this version

mattOP•3/2/25, 12:50 PM

I had to abandon hyperdrive, I noticed you're limited to 25 db connections which bundles auth and my per-user auth reaches to the db so i'd quickly hit the 25 limit

mattOP•3/2/25, 12:52 PM

one thought i had while i was prototyping is that it shouldn't require public DNS records

load balancer works without them and provides much more security by not exposing your db endpoint publically

mattOP•3/2/25, 12:55 PM

there probably needs to a bigger push to allow any cloudflare service to connect to tunnels without having them exposed publically

Mmatt one thought i had while i was prototyping is that it shouldn't require public DN...

AJR•3/2/25, 2:28 PM

This is something we're tinkering with. The trick is to still take advantage of Access for security without a hostname.

Mmatt I had to abandon hyperdrive, I noticed you're limited to 25 db connections which...

AJR•3/2/25, 2:29 PM

Hyperdrive is a transaction mode pooler, so those 25 go a lot further than you might imagine. Also the default limit is 60 after last month, with plans to make it adjustable down the road a bit.

Not saying you should reconsider, but just stating for posterity

kchro3•3/4/25, 8:11 PM

Hi, I am seeing a ton of logs like this:
Error: write CONNECTION_CLOSED <...>.hyperdrive.local:5432

can someone explain what this does

kchro3•3/4/25, 8:16 PM

we had as high as 20 errors per second and a couple hundred failures over a 10 min period

kchro3•3/4/25, 8:21 PM

brandonpsmith•3/5/25, 5:46 AM

i am using neon. i use neon branches for local development. when i set my neon branch connection string in .dev.vars (WRANGLER_HYPERDRIVE_LOCAL_CONNECTIONSTRING<BINDING>=neon branch connection string) and run wrangler dev. it recognizes my environment variable

Found a non-empty WRANGLER_HYPERDRIVE_LOCAL_CONNECTION_STRING variable for binding. Hyperdrive will connect to this database during local development.

Found a non-empty WRANGLER_HYPERDRIVE_LOCAL_CONNECTION_STRING variable for binding. Hyperdrive will connect to this database during local development.

now when i get the connection string from the binding (BINDING.connectionString), it returns a hyperdrive.localhyperdrive.local host instead of my neon connection string.

what am i doing wrong?

kchro3•3/5/25, 2:15 PM

has there been an uptick in the number of these Error: write CONNECTION_CLOSED <...>.hyperdrive.local:5432Error: write CONNECTION_CLOSED <...>.hyperdrive.local:5432 errors?

I've been getting tons of these starting yesterday, and even though I've added retry logic, it's still concerning & a waste of CPU time

AJR•3/5/25, 2:18 PM

There was an accelerated release yesterday that may have severed some connections. We'll need to investigate that, there shouldn't have been the level of disruption you're describing.

Are you still observing elevated error rates? I'm not seeing any such in our broader monitoring.

kchro3•3/5/25, 2:23 PM

yup, here's our 24hr view

kchro3•3/5/25, 2:24 PM

we haven't made any significant code changes before we started seeing errors, except to add retries

AJR•3/5/25, 2:24 PM

I think you were working with Adrian? Let me touch base with him

kchro3•3/5/25, 2:24 PM

yes, thank you

kchro3•3/5/25, 8:56 PM

still seeing this error... please let me know if there are things that I can try as mitigations or if there are any updates

for now, i'm trying to see what'll happen if i disable hyperdrive caching since i have a separate cache in front of my postgres client

AJR•3/5/25, 9:27 PM

We're investigating internally. We will share updates as we have them.

iano•3/5/25, 10:04 PM

We are also seeing increased CONNECTION CLOSED errors, although also are dealing with increased traffic so hard to tell if its the same % or not

AJR•3/5/25, 10:05 PM

same ID as in our thread?

Iiano We are also seeing increased CONNECTION CLOSED errors, although also are dealing...

AJR•3/5/25, 10:36 PM

Went looking. I do see a bump in disconnects for you, but it's not related to what kchro3 is dealing with.

Franco Romano Losada•3/6/25, 3:55 AM

Hey everyone, I'm testing hyperdrive. These are my results:
/api/test/ ---> 200ms first hit, 3-7ms subsequent calls. Great!
/api/test/:id ---> 200-600ms always.

I see in the Hyperdrive analytics that I have a lot of "Uncacheable" requests. I guess this is the dynamic endpoint.

I'm trying to understand why or how it works? Is there a workaround? Does hyperdrive always relay on caching? I'm wondering for the stale data too.

Thanks!

FFranco Romano Losada Hey everyone, I'm testing hyperdrive. These are my results: /api/test/ ---> 200m...

Sam•3/6/25, 9:30 AM

Only static SELECT queries qualify for caching.

SSam Only static SELECT queries qualify for caching.

Franco Romano Losada•3/6/25, 9:57 AM

Thanks. Do you think the 200ms latency is due to me being in Brasil and the DB in Germany?

FFranco Romano Losada Thanks. Do you think the 200ms latency is due to me being in Brasil and the DB i...

Sam•3/6/25, 10:18 AM

Yes for sure.

SSam Yes for sure.

Franco Romano Losada•3/6/25, 12:48 PM

Thanks. Just tested it with a VPN and yes

Sam•3/6/25, 12:49 PM

I think smart placement would be something usefull to look at. Saves time in roundtrips in your code

Franco Romano Losada•3/6/25, 12:50 PM

I was going to ask you that. I have it in smart placement, but it seems like the worker is processed close to the end-user (in this case, me)

Sam•3/6/25, 12:51 PM

You'll need more requests to have it kick in then

Franco Romano Losada•3/6/25, 12:52 PM

Interesting. So when you have a lot of requests it will process the request close to the db?

Sam•3/6/25, 12:55 PM

Yes it needs to analyse your requests: https://developers.cloudflare.com/workers/configuration/smart-placement/#understand-how-smart-placement-works

Sam•3/6/25, 12:56 PM

Ah wait nvm, it does not apply to Hyperdrive. Sorry

Franco Romano Losada•3/6/25, 12:57 PM

Damn. Would be nice if we could set a base region and maybe some replicas if needed

AJR•3/6/25, 1:41 PM

It actually does apply to Hyperdrive

AJR•3/6/25, 1:41 PM

Part of the benefit of the recent pool placement work is that it'll play way nicer with smart placement

SSam Only static SELECT queries qualify for caching.

AJR•3/6/25, 1:42 PM

Actually we can cache extended protocol queries too, but each different ID in this case will be a cache miss the first time you see it

AJR•3/6/25, 1:43 PM

If you re run with the same ID and are seeing misses I'd be curious to poke a bit more. Probably there's something odd with your connection settings that is causing issues for caching. First guess would be a prepare: false setting

Franco Romano Losada•3/6/25, 2:25 PM

Okay, cool! I will check it later if this is an issue with my settings or code

AJR•3/6/25, 2:26 PM

Feel free to DM me with specific queries and connections settings and such. Happy to dig a bit, this is a complicated part and it's always useful to me to see how exactly folks are using it

Franco Romano Losada•3/6/25, 3:11 PM

Thanks, I appreciate it

Jeff•3/6/25, 4:34 PM

Hello, seeing a lot of Connection terminated unexpectedlyConnection terminated unexpectedly errors suddenly, how can I debug?

JJeff Hello, seeing a lot of `Connection terminated unexpectedly` errors suddenly, how...

AJR•3/6/25, 4:37 PM

please post or DM your hyperdrive ID.

thomasgauvin•3/7/25, 4:34 PM

We made it official with an announcement

: https://developers.cloudflare.com/changelog/2025-03-04-hyperdrive-pooling-near-database-and-ip-range-egress/

TLDR:

Query latency is reduced by 90% (!) by pooling databases connections closer to your database (as mentioned by @AJR earlier this week)
Hyperdrive uses Cloudflare IP addresses to connect to your database so you can configure access control lists on your database firewall (as mentioned by @xortive earlier this week)
Workers Smart Placement is more effective with Hyperdrive to ensure minimal network latency for db queries

@Hyperdrive ^

Cloudflare Docs

Hyperdrive reduces query latency by up to 90% and now supports IP a...

Tthomasgauvin We made it official with an announcement 📢: https://developers.cloudflare.com/...

Rapid•3/7/25, 4:43 PM

The link in the changelog is https://developers.cloudflare.com/hyperdrive/platform/changelog/(https://www.cloudflare.com/ips/https://developers.cloudflare.com/hyperdrive/platform/changelog/(https://www.cloudflare.com/ips/

version of cloudflared is `2024.9.1` is that an issue? im using the cloudflare-operator for k8s and

Similar Threads

Similar Threads

Similar Threads