understood, but was hopeful you'd just enable TCP in a place I could write my own 😉 any other worka

understood, but was hopeful you'd just enable TCP in a place I could write my own

any other workarounds you are aware of? do you know anyone successfully using an external connection pooler like prisma accelerate? i came across that last night

Bbryan understood, but was hopeful you'd just enable TCP in a place I could write my ow...

Matt Silverlock•1/10/24, 2:32 PM

You can write your own. Hyperdrive uses the socket API in Workers - the Postgres (and soon) MySQL drivers we support adopted that. MongoDB’s driver a) doesn’t yet run in any edge serverless runtime and b) holds a lot of client state. Both of these are big blockers to us.

https://blog.cloudflare.com/hyperdrive-making-regional-databases-feel-distributed

The Cloudflare Blog

Hyperdrive: making databases feel like they’re global

Hyperdrive makes accessing your existing databases from Cloudflare Workers, wherever they are running, hyper fast.

Bbryan understood, but was hopeful you'd just enable TCP in a place I could write my ow...

Matt Silverlock•1/10/24, 2:33 PM

IIRC Prisma Accelerate also doesn’t support Mongo, likely for many of the same architectural reasons (running a highly stateful Mongo proxy is non-trivial)

bryanOP•1/10/24, 2:37 PM

Good to know re: socket API, am I able to create connections that survive worker requests and selectively pool? Not seeing anything in the API docs for that specifically

bryanOP•1/10/24, 2:43 PM

I'll stop bugging, but are you aware of any active mongodb projects (from mong) to make the driver edge-compatible? Happy to go bark up their tree

Mitya•1/10/24, 5:56 PM

Is there an issue with HD right now? Suddenly all my requests are giving this

Mitya•1/10/24, 5:58 PM

Tried restarting the server, same deal

Mitya•1/10/24, 5:59 PM

It's just come back up (with no code changes) but for ~5-10 mins that was all I was getting for every DB request.

MMitya It's just come back up (with no code changes) but for ~5-10 mins that was all I ...

Chaika•1/10/24, 6:13 PM

Intermittent API Availability

Mitya•1/10/24, 6:22 PM

Thanks @Chaika

Bbryan Good to know re: socket API, am I able to create connections that survive worker...

Matt Silverlock•1/10/24, 7:38 PM

No, because pooling needs to be application-aware - there's not really a concept of a "generic TCP connection pooler" because any pooling needs to understand the application protocol layered on top - e.g. when a connection is "done" and can be re-used.

bryanOP•1/10/24, 9:01 PM

agreed, was hoping "mark for reuse" to be a part of the API! Otherwise, it should just close all connections at termination

ernest•1/12/24, 7:28 AM

Hey I am new to using Hyperdrive, but how do I get the IP range to whitelist for creating a hyperdrive? I have a public IP on Cloud SQL that i want to connect to but it's currently limiting access to a white listed IP range.

ernest•1/12/24, 7:49 AM

Ah okay read that we will need to wait for Tunnel to work with Hyperdrive, until then only fully public IP access is allowed

Eernest Hey I am new to using Hyperdrive, but how do I get the IP range to whitelist for...

Matt Silverlock•1/13/24, 12:14 AM

Correct, right now we require the server is publicly reachable. You can (if you want to) still do that with a Google Cloud SQL server - 0.0.0.0/00.0.0.0/0

Mitya•1/24/24, 2:35 PM

Minor bug. If you enter an invalidly-formed connection string when creating a HD via the dashboard, you get a 400 but no UI feedback.

Mitya•1/24/24, 2:50 PM

Are there plans to enable toggling of caching via the dashboard? As far as I can see this is achievable via CLI only (the dashboard reports the caching status, but you can't change it.)

Mitya•1/24/24, 2:51 PM

Finally (sorry for all the questions at once), am I right in thinking that any HD-powered DB would need to be globally accessible i.e. no IP restrictions? Normally we restrict IPs only to the users/servers that connect to it, but I assume this isn't possible with Workers since the IPs would be unpredictable?

kian•1/24/24, 2:52 PM

am I right in thinking that any HD-powered DB would need to be globally accessible i.e. no IP restrictions?

Yes, but Cloudflare Tunnels/Magic WAN integrations are on the way.

Mitya•1/25/24, 11:50 AM

If I want to specify a different HD binding for different environments, is the following the correct format?

[env.stage]
[[hyperdrive]]
binding = ...
id = ...

[env.prod]
[[hyperdrive]]
binding = ...
id = ...

[env.stage]
[[hyperdrive]]
binding = ...
id = ...

[env.prod]
[[hyperdrive]]
binding = ...
id = ...

(I'm a little shakey on TOML syntax.) Thank you.

kian•1/25/24, 11:50 AM

[[env.stage.hyperdrive]][[env.stage.hyperdrive]]

kian•1/25/24, 11:50 AM

kian•1/25/24, 11:51 AM

[env.stage]
hyperdrive = [
  { binding = "...", id = "..." }
]

[env.stage]
hyperdrive = [
  { binding = "...", id = "..." }
]

kian•1/25/24, 11:51 AM

the [[ or [[[[ syntax indicates a table, which are not inherited by other tables

Mitya•1/25/24, 11:53 AM

Right, thank you - clearly I need to brush up on TOML. Probably off-topic for here, and just out of curiosity, but why did CF opt for TOML - are there obvious advantages over JSON or YAML?

kian•1/25/24, 11:54 AM

Wrangler 1 was Rust where TOML is pretty much standard, 2 and 3 just kept using TOML

kian•1/25/24, 11:54 AM

There's a JSON flag if you want to use

wrangler.json

wrangler.json

instead.

Mitya•1/25/24, 11:54 AM

Oh! Good to know.

WcaleNieWolny•1/27/24, 6:04 AM

Hello, I tried hyperdrive but I have a problem with caching
here is the dashboard and here are the SQL statements:

"select \"id\", \"user_id\" from \"app_versions\" where (\"app_versions\".\"name\" = $1 or \"app_versions\".\"app_id\" = $2) limit $3"

"select \"id\", \"user_id\" from \"app_versions\" where (\"app_versions\".\"name\" = $1 or \"app_versions\".\"app_id\" = $2) limit $3"

"select \"app_id\" from \"apps\" where \"apps\".\"app_id\" = $1 limit $2""select \"app_id\" from \"apps\" where \"apps\".\"app_id\" = $1 limit $2"

Is there any way to cache these?

Sam•1/29/24, 8:53 AM

Is Hyperdrive able to resolve ipv6 only servers?

MMatt Silverlock IIRC Prisma Accelerate also doesn’t support Mongo, likely for many of the same a...

jonfanz•1/30/24, 12:29 AM

(cc: @divided421) Jon from Prisma here. Accelerate does support Mongo. Agree that we could make it more obvious though, it's only briefly mentioned here

xatxat•1/30/24, 9:51 AM

hi, i'm occasionally getting "could not get connection from the pool""could not get connection from the pool"-errors with one of my hyperdrive connections. it seems to happen quite randomly, maybe about one in ten times. any advice how to debug this?

Xxatxat hi, i'm occasionally getting `"could not get connection from the pool"`-errors w...

morpig•1/30/24, 10:24 AM

are you experiencing it only recently? or from day 0 since deploy?

Mmorpig are you experiencing it only recently? or from day 0 since deploy?

xatxat•1/30/24, 10:29 AM

i experienced it on a worker that was last deployed about two weeks ago. however, i just re-deployed that worker and so far the error didn't happen anymore.

xatxat•1/30/24, 10:30 AM

in my code i'm setting up a pool of with 10 connections max:

const dialect = new PostgresDialect({
    pool: new Pool({
      connectionString,
      max: 10,
    }),
  });

const dialect = new PostgresDialect({
    pool: new Pool({
      connectionString,
      max: 10,
    }),
  });

xatxat•1/30/24, 10:31 AM

however, i'm not sure if i should even be setting up a pool? since hyperdrive itself acts as a connection pool?

xatxat•1/30/24, 10:32 AM

maybe i should reduce max connections to 1?

kian•1/30/24, 10:38 AM

you'd be better off just using a single client

Kkian you'd be better off just using a single client

xatxat•1/30/24, 10:47 AM

thx, then i'll configure the pool with max: 1max: 1. afaik kysely needs to be initialized with a pool instance.

Colin•1/30/24, 3:24 PM

hey y'all - wanted to talk about the localConnectionStringlocalConnectionString in

wrangler.toml

wrangler.toml

with Hyperdrive

I've seen a couple other messages in here saying people didn't think it should be stored in plain text like this/might contain secrets (password/endpoint)/etc., and at least for my use case, I agree.

I believe the intention on CF's side is that this shouldn't ever be a string that has secrets + you should only point to a local Postgres instance (IE so it'd be postgres://localhostpostgres://localhost, or something that's fine to commit to VCS), but there's definitely been times where I need to connect to a remote dev database. For example, AWS Aurora-exclusive features that don't have a local equivalent/emulator.

The current model doesn't seem to support this very well, as you're forced to either put the sensitive string in

wrangler.toml

wrangler.toml

or make something akin to a OVERRIDE_DB_CONNECTION_STRINGOVERRIDE_DB_CONNECTION_STRING env var, and handle choosing between hyperdrive.connectionStringhyperdrive.connectionString or the override in your app. Even if you do make an env var/some other way to sidestep the builtin solution,

wrangler

wrangler

still requires a localConnectionStringlocalConnectionString to be set in

wrangler.toml

wrangler.toml

.

Is there any chance of a better way to do handle this / am I missing something to make local dev easier?

SSam Is Hyperdrive able to resolve ipv6 only servers?

Matt Silverlock•1/31/24, 8:16 AM

This should work.

carlos-alberto•1/31/24, 9:25 AM

Hey! Are there plans to add hyperdrive support to the cloudflare terraform provider?

kian•1/31/24, 9:30 AM

It's publicly documented (https://developers.cloudflare.com/api/operations/create-hyperdrive) so it just needs to be added into cloudflare-go and cloudflare-terraform-provider.

kian•1/31/24, 9:31 AM

You can open an issue at https://github.com/cloudflare/terraform-provider-cloudflare/issues to request it

MMatt Silverlock This should work.

Sam•1/31/24, 3:21 PM

Can confirm

CColin hey y'all - wanted to talk about the `localConnectionString` in `wrangler.toml` ...

Matt Silverlock•1/31/24, 4:04 PM

We're going to introduce an env var that takes predecence over

wrangler.toml

wrangler.toml

localConnectionStringlocalConnectionString

carlos-alberto•1/31/24, 4:14 PM

I opened a PR at https://github.com/cloudflare/cloudflare-go/pull/1492 to add the support for Hyperdrive to cloudflare-go, hope it's okay, first gogo attempt

GitHub

Add Hyperdrive CRUD support by carlos-alberto · Pull Request #1492 ...

Add support for managing Hyperdrive configs
Description
Resolves #1491
Mainly wrote by copying the existing queue.go
Has your change been tested?

Tested ListHyperdriveConfigs manually
Added tests
...

Paul•2/2/24, 8:48 PM

if i'm using hyperdrive to connect to a database that handles my user auth sessions, and i have caching enabled with 60 second max_age, and someone logs out... does that mean that for the next 60 seconds they might be pulling their old cached session?

JJames Unfortunately Pages Functions doesn’t support node_compat bundling, nor the hype...

Paul•2/3/24, 12:49 AM

This is tragic for me -- is this still the case?

PPaul This is tragic for me -- is this still the case?

James•2/3/24, 1:17 AM

Unfortunately so, yeah. There's no real way to get Hyperdrive on Pages Functions today - you will have to use Workers as far as I know.