Where there any known issues with Hyperdrive not closing off connections? Our memory and connections

Where there any known issues with Hyperdrive not closing off connections? Our memory and connections peaked at an all time high for some reason, with no extremely high traffic coming in. This happened between 6 and 7AM UTC. Could be at our end of course, but just want to make sure it's not coming from Cloudflare. Thanks.

SSam Where there any known issues with Hyperdrive not closing off connections? Our me...

Matt Silverlock•11/9/23, 11:47 AM

You can identify connections from Hyperdrive: https://developers.cloudflare.com/hyperdrive/learning/connect-to-postgres/#identify-connections-from-hyperdrive

Connect to PostgreSQL · Hyperdrive

Hyperdrive supports both PostgreSQL and PostgreSQL-compatible databases, as well as popular drivers and the many Object Relational Mapper (ORM) …

James•11/9/23, 12:34 PM

Are there any plans to support MongoDB with Hyperdrive? Would be awesome!

JJames Are there any plans to support MongoDB with Hyperdrive? Would be awesome!

Matt Silverlock•11/9/23, 1:07 PM

Not in the near term, but we're talking with MongoDB. Their driver architecture makes it extremely complex, and their drivers themselves don't run in any edge serverless (Workers/Deno/Bun/etc) runtime today.

Kasumi•11/10/23, 12:45 PM

Is it now free to use or just available for people who have a paid worker plan?

Kasumi•11/10/23, 1:36 PM

Uhh someone know how to fix that?

KKasumi Is it now free to use or just available for people who have a paid worker plan?

Cyb3r-Jak3•11/10/23, 1:36 PM

Free to anyone on a paid Workers plan

Kasumi•11/10/23, 1:36 PM

That means you need a Worker paid plan

Cyb3r-Jak3•11/10/23, 1:37 PM

Correct. But there is no extra cost for hyperdrive

Kasumi•11/10/23, 1:37 PM

Kasumi•11/10/23, 1:38 PM

is there a trial?

Kasumi•11/10/23, 1:38 PM

no right?

Cyb3r-Jak3•11/10/23, 1:38 PM

Correct. There is not a trial

Kasumi•11/10/23, 1:38 PM

sadge

Kasumi•11/10/23, 1:38 PM

want to test it and look how to implement

Matt Silverlock•11/10/23, 3:03 PM

https://x.com/TimescaleDB/status/1722722066443669576?s=20

MMatt Silverlock https://x.com/TimescaleDB/status/1722722066443669576?s=20

Matt Silverlock•11/10/23, 3:03 PM

New tutorial: use Timescale with Hyperdrive + Workers! https://developers.cloudflare.com/hyperdrive/tutorials/serverless-timeseries-api-with-timescale/

Create a serverless, globally distributed time-series API with Time...

In this tutorial you will learn to build an API on Workers which will ingest and query time-series data stored in Timescale (they make PostgreSQL …

KKasumi want to test it and look how to implement

SamOP•11/10/23, 5:44 PM

There’s no difference with a regular Postgres connection in development. You’ll just not benefit from the features Hyperdrive has to offer.

So if you really want to use it now or in the future, the $5 per month for the Paid Workers is really worth it

.Zero•11/10/23, 10:01 PM

Is there any idea on how much hyperdrive will cost once out of beta?

.Zero•11/10/23, 10:01 PM

I dont want to implement this now just to see that it gets suprt expensive

..Zero Is there any idea on how much hyperdrive will cost once out of beta?

Matt Silverlock•11/10/23, 10:21 PM

We’re still working on pricing but our goal is to make it extremely affordable.

.Zero•11/10/23, 10:41 PM

Alright

MMatt Silverlock We’re still working on pricing but our goal is to make it extremely affordable.

.Zero•11/10/23, 10:42 PM

Just because i got your attention i wanna let you know that i think cloudflare is one of the coolest and most awesome companies out there as a tech person.

MMatt Silverlock We’re still working on pricing but our goal is to make it extremely affordable.

Colin•11/11/23, 2:29 AM

Just to confirm there's still no intent to charge for pooling-only usage?

Sameer Ali•11/14/23, 7:55 AM

Has anyone experienced a situation where, with caching turned on, the same query randomly returns an empty array for a while and then starts to return correct results? I never noticed this issue when caching is disabled.

DanielZero•11/14/23, 12:03 PM

There's no example that shows how to setup Hyperdrive for different environments in the

wrangler.toml

wrangler.toml

I'm new to CF Workers and don't know what the

[[hyperdrive]]

[[hyperdrive]]

notation would translate to. Would it be something like

[[env.staging.hyperdrive]]

[[env.staging.hyperdrive]]

? Thanks for any help!

DDanielZero There's no example that shows how to setup Hyperdrive for different environments...

Matt Silverlock•11/14/23, 12:08 PM

Yes.

MMatt Silverlock Yes.

DanielZero•11/14/23, 12:09 PM

Thanks a lot for that quick answer!

amorfati•11/15/23, 8:06 PM

is the hyperdrive id I get when i create a new one with wrangler ok to commit to public repo or secret?

zegevlier•11/15/23, 8:07 PM

Anything in wrangler.toml is safe to commit, except if you put anything secret in the [env] catagory

Aamorfati is the hyperdrive id I get when i create a new one with wrangler ok to commit to...

Emilio•11/15/23, 10:40 PM

The ID is scoped to your account, nobody else can use that ID to glean any information -- except me and the other Cloudflare employees

reimertz•11/18/23, 1:47 PM

Hi all, been trying to explore if it would be possible to use hyperdrive in combination with supabase.js. The idea would be to setup hyperdrive with supabase and then point the supabase web client to use hyperdrive rather then directly querying their global postgres instance. Just haven't been able to find any documentation/ examples how to specify a custom postgres url.

Rreimertz Hi all, been trying to explore if it would be possible to use hyperdrive in comb...

Matt Silverlock•11/18/23, 3:03 PM

Hyperdrive doesn’t accept arbitrary client connections itself today. It’s designed to help folks use Workers to connect to their databases (including Supabase or Neon!).

Supabase.js makes HTTP requests directly to Supabase’s custom HTTP proxy (likely using pgrest in some way) as well - it’s not something we can really support given it’s very Supabase specific.

Mas Inis•11/19/23, 5:29 AM

Hi folks, what would be good managed postgre candidate to use with hyperdrive? (Or workers in general)

Mas Inis•11/19/23, 5:32 AM

I see Neon and Supabase mentioned quite often

MMas Inis Hi folks, what would be good managed postgre candidate to use with hyperdrive? (...

Colin•11/19/23, 3:37 PM

Just throwing in my 2c, we evaluated all of the popular “serverless” Postgres DBs and none of them ticked all the boxes/gave us as much confidence as AWS Aurora

Colin•11/19/23, 3:38 PM

I feel like Neon will be a great choice in a year or two, but rn it seemed to lack some basic features that AWS has

Isaac McFadyen•11/20/23, 2:31 PM

Google Cloud SQL is also quite good if you're already integrated into their ecosystem.

arbfay•11/21/23, 11:10 AM

I know more databases are coming later, but may I still ask if ClickHouse is on your list?

Aarbfay I know more databases are coming later, but may I still ask if ClickHouse is on ...

Matt Silverlock•11/21/23, 12:46 PM

Not yet, no. The ClickHouse protocol doesn’t have a specification: https://clickhouse.com/docs/en/interfaces/tcp

Native Interface (TCP) | ClickHouse Docs

The native protocol is used in the command-line client, for inter-server communication during distributed query processing, and also in other C++ programs. Unfortunately, native ClickHouse protocol does not have formal specification yet, but it can be reverse-engineered from ClickHouse source code (starting around here) and/or by intercepting an...

Aarbfay I know more databases are coming later, but may I still ask if ClickHouse is on ...

Matt Silverlock•11/21/23, 12:48 PM

Also: are you querying ClickHouse on every request? I can’t imagine that will be fast regardless.

MMatt Silverlock Also: are you querying ClickHouse on every request? I can’t imagine that will be...

DaniFoldi•11/21/23, 12:53 PM

With hyperdrive’s caching that could become possible meowwink

DDaniFoldi With hyperdrive’s caching that could become possible <:meowwink:3804724056778342...

Matt Silverlock•11/21/23, 1:20 PM

The caching relies on the protocol being specified (and a stable parser being available, plus a JS driver). But I’m asking as if you’re instead just querying it periodically/in a Cron/etc, the HTTP interface for ClickHouse is likely a more straightforward approach right now.

MMatt Silverlock Not yet, no. The ClickHouse protocol doesn’t have a specification: https://click...

arbfay•11/21/23, 2:26 PM

Ah I didn't know. I guess an experiment with their PostgreSQL protocol is necessary

MMatt Silverlock Also: are you querying ClickHouse on every request? I can’t imagine that will be...

arbfay•11/21/23, 2:28 PM

Sry I'm not sure I see why you think this is a problem.
ClickHouse for OLAP is very performant. I use it for about 600 dashboards (for different users), and yes queries every time (but cached with a worker). It's also what powers Cloudflare's Analytics Engine IIRC.

arbfay•11/21/23, 2:29 PM

It's got an HTTP endpoint to run queries, and honestly I might just stick to the Worker for caching + HTTP. I guess the most interesting part for me was the connection pooling.

Aarbfay Sry I'm not sure I see why you think this is a problem. ClickHouse for OLAP is v...

Matt Silverlock•11/21/23, 2:43 PM

I am very aware of how ClickHouse performs. What I am trying to point out is that if you rely on native ClickHouse support you may be waiting a long time, and that CH’s baseline performance may be “fast” but isn’t commonly sub-second for anything but the most simple queries.

Instead, using a HTTP interface will likely get you moving in the meantime, or trying their Postgres interface (untested).

MMatt Silverlock I am very aware of how ClickHouse performs. What I am trying to point out is tha...

arbfay•11/21/23, 3:06 PM

Yeah, I use their HTTP interface since Workers didn't have TCP support yet when we decided to use it and I couldn't be bothered with creating and maintaining a HTTP proxy for any other OLAP/time-series DB. Performance-wise, the queries seem to run fast enough (20-200ms, we really aren't the same size as CF), helped with caching, but the latency is spiky (I suspect its due to making new the connections but haven't investigated yet).

arbfay•11/21/23, 3:06 PM

Anyway, it's all good to know. Might try the Postgres interface some time later

Aarbfay Yeah, I use their HTTP interface since Workers didn't have TCP support yet when ...

Cyb3r-Jak3•11/21/23, 3:28 PM

Workers have outbound TCP connections.