Cloudflare Developers•12mo ago

Are cached Hyperdrive queries invalidated if a write occurs?

ZZn4rK Are cached Hyperdrive queries invalidated if a write occurs?

AJR•1/24/25, 11:40 AM

Discussed recently:
hyperdrive

AAJR Discussed recently: https://discord.com/channels/595317990191398933/115055798623...

Zn4rKOP•1/24/25, 11:41 AM

Ty!

dave•1/25/25, 1:59 PM

Is there any plans to allow using Hyperdrive as a pooler for non-Workers connections?

Ddave Is there any plans to allow using Hyperdrive as a pooler for non-Workers connect...

AJR•1/25/25, 8:03 PM

Plans is a strong word. It's something we've discussed internally, but no firm commitments yet. If it's something you'd be interested in, I'll make sure to filter that back to our product folks.

AAJR No. You can cache-bust individual queries by including any volatile function in...

sid•1/25/25, 8:08 PM

Im using hyperdrive to cache queries to my supabase db, is there a way to cache bust a particular document. E.g. if I run a volatile query like

SELECT random() FROM table WHERE id =  1234

SELECT random() FROM table WHERE id =  1234

will the cache only be invalidated for that record or the whole table?

Ssid Im using hyperdrive to cache queries to my supabase db, is there a way to cache ...

AJR•1/25/25, 8:10 PM

We don't do any purging. If you do

SELECT random(), * FROM table WHERE id = 1234

SELECT random(), * FROM table WHERE id = 1234

you'll get uncached results for that query. What's in the cache won't be touched either way.

sid•1/25/25, 8:12 PM

ah okay understood

AAJR Plans is a strong word. It's something we've discussed internally, but no firm ...

dave•1/25/25, 8:45 PM

Would be quite interested.

AAJR Hey everyone. I'm looking for some beta testers. We have a fairly large change...

AJR•1/27/25, 2:39 PM

Hey everyone. Beta test has started, last call for folks who want to join.

Please reply in thread with your Hyperdrive ID if so. Thanks!

AAJR We don't do any purging. If you do `SELECT random(), * FROM table WHERE id = 123...

dave•1/27/25, 2:53 PM

Do you optimize this type of query by skipping even checking the cache? Or is there still a small delay due to a cache lookup miss?

Ddave Do you optimize this type of query by skipping even checking the cache? Or is th...

AJR•1/27/25, 2:54 PM

We skip checking the cache. The delay for doing so is incredibly small, but no reason to waste the traffic.

AJR•1/27/25, 2:58 PM

For the record, we are aware this is not the best UX for bypassing cache on a single query, and are hoping to find/build something better that still plays nicely with the variety of drivers/libs we want to support.

letincho5•1/27/25, 8:15 PM

Hi, I'm using Hyperdrive + Neon + Drizzle, and I've an issue: I've to connect a new connection (const db = drizzle()...) for each query, in the same "request/worker" execution. When I not use Hyperdrive, I can have a singleton of drizzle and do multiples queries.

I'm use thats imports:
import { drizzle, NodePgDatabase } from "drizzle-orm/node-postgres";

Any ideas?

Hello, I’m Allie!•1/28/25, 5:25 AM

How does Hyperdrive handle replicas now?

HHello, I’m Allie!How does Hyperdrive handle replicas now?

AJR•1/28/25, 11:21 AM

For now, the same as it always has. Folks generally make separate Hyperdrives for read replicas and direct queries there at the application level.

Jeroen•1/29/25, 4:14 AM

I get a ton of errors of this form:

[ERROR]   Error: write CONNECTION_ENDED 4386caf43864cfa4ca634cafca3a4cf2.hyperdrive.local:5432

[ERROR]   Error: write CONNECTION_ENDED 4386caf43864cfa4ca634cafca3a4cf2.hyperdrive.local:5432

What causes this? I think it is on Cloudflare's end, since I have no errors in my actual database.

Jeroen•1/29/25, 4:15 AM

Happens in about 50% of cases.

Jeroen•1/29/25, 4:29 AM

ah it was unexpected behaviour of waitUntil

Jeroen•1/29/25, 4:29 AM

I thought it'd chain but it doesn't

AAJR For now, the same as it always has. Folks generally make separate Hyperdrives fo...

dave•1/29/25, 1:38 PM

It would be pretty cool if Hyperdrive could automatically redirect read-only queries to the read replica server.

Ddave It would be pretty cool if Hyperdrive could automatically redirect read-only que...

AJR•1/29/25, 1:39 PM

It's something we're kicking around, definitely not a Q1 project but enough folks have mentioned it that I'll be surprised if we don't end up shipping it this year.

thomasgauvin•1/29/25, 5:10 PM

Hey folks! We announced the new automatic configuration for Hyperdrive's support for private databases over tunnels. You may have caught this previously, but now you only need to setup your tunnel in your virtual network/VPC where you can access your database, and then Hyperdrive will automatically configure the tunnel to be restricted to Hyperdrive

https://developers.cloudflare.com/changelog-next/2025-01-28-hyperdrive-automated-private-database-configuration/

hyperdrive-private-database-automatic-configuration.png

Vero•1/29/25, 5:13 PM

@Hyperdrive

Tthomasgauvin Hey folks! We announced the new automatic configuration for Hyperdrive's support...

M.A.K•1/29/25, 5:20 PM

nice

darkpool•1/29/25, 9:20 PM

is it possible to use google spanner with hyperdrive?

Ddarkpool is it possible to use google spanner with hyperdrive?

AJR•1/29/25, 9:45 PM

I don't believe anyone is doing that, no. We haven't tested it, though some brief googling does show spanner has a "postgres-compatible" interface, which I hadn't realized. Might be worth us taking a look, at some point.

https://developers.cloudflare.com/hyperdrive/reference/supported-databases/

Cloudflare Docs

Supported databases · Hyperdrive

Details on which database engines and/or specific database providers are supported are detailed in the following table.

AAJR I don't believe anyone is doing that, no. We haven't tested it, though some bri...

darkpool•1/29/25, 9:57 PM

Thanks for the reply. Spanner Graph is in preview right now and looks pretty interesting plus recent support for proto data types. Although it seems new graph capabilities are only available in GoogleSQL interface not PG atm.

Ddarkpool Thanks for the reply. Spanner Graph is in preview right now and looks pretty int...

dave•1/30/25, 5:05 AM

There's Amazon Aurora DSQL too

dave•1/30/25, 5:06 AM

That said, I found the limitations too hard to work with.. Takes away a lot of the things I love about using PostgreSQL (making sure I don't shoot myself in the foot).

dave•1/30/25, 5:07 AM

AWS RDS Aurora Serverless v2 is my goto. Scales to zero as well, which is very nice for dev instances

Agung Vivekananda•1/30/25, 8:00 AM

Hello,
Currently i'm using hyperdrive with direct connection to supabase.

But sometimes i got Database error: Error: write CONNECTION_CLOSED 86ec870f264501983a0.hyperdrive.local:5432

it its any fix for this? Or the problem its on supabase

Ddave There's Amazon Aurora DSQL too

AJR•1/30/25, 12:09 PM

They do actually work over Hyperdrive, that one we checked. The needing to refresh an IAM token every 7 days isn't great though, we'd need to build some AWS IAM integrations to automate that to really call them "supported", imo.

AAJR They do actually work over Hyperdrive, that one we checked. The needing to refre...

dave•1/30/25, 5:50 PM

That's pretty easy to do. I already wrote a function for refreshing the IAM token, so I'd just need to use one of these endpoints to update it every 24 hours (to be safe to avoid long lasting connections from expiring).

https://developers.cloudflare.com/api/resources/hyperdrive/subresources/configs/methods/edit/

https://developers.cloudflare.com/api/resources/hyperdrive/subresources/configs/methods/update/

What's the difference between edit and update?

Cloudflare API | Hyperdrive › Configs › Patch Hyperdrive

Interact with Cloudflare's products and services via the Cloudflare API

Cloudflare API | Hyperdrive › Configs › Update Hyperdrive

Interact with Cloudflare's products and services via the Cloudflare API

Ddave That's pretty easy to do. I already wrote a function for refreshing the IAM toke...

AJR•1/30/25, 5:51 PM

PATCH vs PUT. PATCH will only update the things that were changed. PUT will update/overwrite the whole config, and trigger revalidation.

More practically, it's because PATCH in terraform is a pain so we felt it would be most efficient to just have both.

AAJR PATCH vs PUT. PATCH will only update the things that were changed. PUT will upd...

dave•1/30/25, 5:52 PM

ah, so I'd want to use PATCH here?

AJR•1/30/25, 5:54 PM

IIRC refreshing the token gives you a new DSQL connection string, so there's not much difference between them in this scenario. New conn string will trigger revalidation anyways, since that's the stuff that affects whether hyperdrive can still connect.

AJR•1/30/25, 5:54 PM

The dash uses PATCH, can't change anything there that would change our ability to connect so it can skip the revalidation traffic.

AAJR IIRC refreshing the token gives you a new DSQL connection string, so there's not...

dave•1/30/25, 5:55 PM

I only need to update the the password. Everything else stays the same.

AJR•1/30/25, 5:55 PM

Ah, neat. I misunderstood what DSQL was doing there.

Same answer though. Typos when entering a password is like half the reason that we do the config-time validation in the first place.

AAJR IIRC refreshing the token gives you a new DSQL connection string, so there's not...

dave•1/30/25, 5:56 PM

To confirm, does changing the password with either PATCH or PUT cause a trigger revalidation?

AJR•1/30/25, 5:58 PM

Yes it does.

dave•1/30/25, 5:59 PM

cool, I'll do a blog post on it.

AJR•1/30/25, 5:59 PM

Nice! Share it here and we'll repost it around, if you like.

dave•1/30/25, 6:00 PM

ya I'm doing it for the SEO, so please do share and ideally link.

AJR•1/30/25, 6:00 PM

Don't hate the player, hate the game.

dave•1/30/25, 6:48 PM

@AJR Is there a way I can create a Hyperdrive without needing a valid password?

AJR•1/30/25, 6:49 PM

No, there is not.

dave•1/30/25, 6:55 PM

btw, so happy that the permissions required now shows up in the API docs!!

dave•1/30/25, 6:55 PM

I think this is new?

AJR•1/30/25, 6:56 PM

There was a big docs revamp recently, if I recall. All generated from our internal schemas. Very slick, and much less toil on our end. I really appreciate that team shouldering the project.

Are cached Hyperdrive queries invalidated if a write occurs?

Similar Threads

Similar Threads

Similar Threads