I have some Cloudflare Pages apps that needs to access a Postgres DB. I also have traditional worklo

I have some Cloudflare Pages apps that needs to access a Postgres DB. I also have traditional workloads that connects to Postgres, which lives right next to the Postgres box.

Is it better to connect everything through Hyperdrive or should the traditional workload also be routed through Hyperdrive? And if I should keep the traditional workloads connecting directly to Postgres, how should I set up pgBouncer so that I don't run out of connections - should Hyperdrive connect to pgBouncer, or to Postgres directly?

Tt4 I have some Cloudflare Pages apps that needs to access a Postgres DB. I also hav...

AJR•7/11/24, 4:43 PM

In that architecture, traditional workloads should probably not go through Hyperdrive.

Our general recommendation is that Hyperdrive should connect directly, as stacking multiple poolers on top of each other usually has no benefit and can cause issues in some cases. That said, I know many users do connect Hyperdrive to another pooler and it works fine for them. Your use case is unique enough that having everything centrally run through pgbouncer might be warranted. You'd need to do some testing to see how well either arrangement works.

SSurjith S M Hey all 👋 New to Hyperdrive. I have three questions about cache invalidation: ...

AJR•7/11/24, 4:46 PM

It's a simple TTL cache. Once a request is cached, Hyperdrive will attempt to serve from cache until the cached result expires. It does not do any proactive purging on mutation.
Same as 1. If you really need to bypass cache on individual queries, current best option is include any volatile function in your query (e.g. including
```
random()
```
```
random()
```
in your SELECT statement). Offering a better option for this is on our radar but not for very soon.
Latency will still be much better through Hyperdrive than a direct connection in most cases (with the caveat that networking is a dynamic thing and sometimes routes can get weird). However cache hits will always be much faster than anything else, so it's up to you what tradeoffs are right for your use case.

SSurjith S M thanks, for my use case, i can put longer cache period. like 1 hour or 1 day. bu...

AJR•7/11/24, 4:49 PM

Planned, but not for very soon. Got a few big features we've promised to deliver on first.

Ping for toast•7/15/24, 9:14 PM

somehow we've started hitting cache a wee bit!

Ping for toast•7/15/24, 9:15 PM

I assume you did something on the backend, thank you!

jared•7/16/24, 10:34 AM

I'm not able to create new Hyperdrive configs or update the existing ones via the CF Dashboard and wrangler on the terminal on an account with a paid plan. I was able to create before. I'm getting this error:

[ERROR] A request to the Cloudflare API (/accounts/[account-id]/hyperdrive/configs/[config-id]) failed.
Internal Server Error [code: 10000]

Token Permissions: If scopes are missing, you may need to logout and re-login.
Scope (Access)
- account (read)
- user (read)
- workers (write)
- workers_kv (write)
- workers_routes (write)
- workers_scripts (write)
- workers_tail (read)
- d1 (write)
- pages (write)
- zone (read)
- ssl_certs (write)
- constellation (write)
- ai (write)
- queues (write)
- offline_access

[ERROR] A request to the Cloudflare API (/accounts/[account-id]/hyperdrive/configs/[config-id]) failed.
Internal Server Error [code: 10000]

Token Permissions: If scopes are missing, you may need to logout and re-login.
Scope (Access)
- account (read)
- user (read)
- workers (write)
- workers_kv (write)
- workers_routes (write)
- workers_scripts (write)
- workers_tail (read)
- d1 (write)
- pages (write)
- zone (read)
- ssl_certs (write)
- constellation (write)
- ai (write)
- queues (write)
- offline_access

I did try logging out then re-logging in.

AJR•7/16/24, 1:36 PM

Let me check in on this.

AJR•7/16/24, 3:05 PM

@jared give it a shot now.

AAJR @jared give it a shot now.

jared•7/16/24, 3:28 PM

I was already able to update the existing configs. Thank you.

Xxetic Any updated timeline regarding support for Cloudflare Tunnel connections to prev...

AJR•7/16/24, 11:54 PM

Currently our top priority and will be our next big feature released. See hyperdriveWhile waiting for Cloudflare Tunnel for a bit more discussion.

AAJR Currently our top priority and will be our next big feature released. See https...

anirudh•7/17/24, 4:43 AM

Will this adress the issue of Hyperdrive placement in APAC region?

Aanirudh Will this adress the issue of Hyperdrive placement in APAC region?

AJR•7/17/24, 11:13 AM

That's a separate track of work and is coming along soon. That's not a feature it's a fix, sorry for the confusion there.

AJR•7/17/24, 11:17 AM

Though, that raises a good point. If your setup is in APAC and you're having latency issues, please message me directly.

Ping for toast•7/17/24, 7:59 PM

@AJR have your team done performance work with Hyperdrive under the hood in the past two weeks?

Ping for toast•7/17/24, 7:59 PM

Ping for toast•7/17/24, 7:59 PM

I'm in the US east again and this is my latency now

PPing for toast No hyperdrive (US), No hyperdrive (VPN to EU)

Ping for toast•7/17/24, 8:00 PM

2x improvement

Ping for toast•7/17/24, 8:00 PM

same location

PPing for toast I'm sorry to have so many things go wrong, but I'm also seeing latency issues wi...

Ping for toast•7/17/24, 8:00 PM

oop, sorry, here are the correct numbers

Ping for toast•7/17/24, 8:01 PM

so from 620-770ms to 360-520

Ping for toast•7/17/24, 8:02 PM

(still not as fast as a VPN to europe but incredible improvement in such a short time!)

Xxortive this is for a DB located in europe?

Ping for toast•7/17/24, 8:02 PM

Yeah

Ping for toast•7/17/24, 8:02 PM

US East → hetzner germany

Ping for toast•7/17/24, 8:04 PM

ah, I was so excited... well i'm still excited but

Ping for toast•7/17/24, 8:04 PM

I was about to be like "wow wtf did you change to save 200ms"

Ping for toast•7/17/24, 8:05 PM

lmk if I can beta test it in the future, sounds very exciting to me

João Castro•7/21/24, 8:08 AM

Hi! In which regions/colos can Hyperdrive spawn? And is there any way to know in which location i'm currently connected?

thomasgauvin•7/23/24, 5:28 PM

Hi João! Hyperdrive can spawn in various regions, usually one that's closest to your Worker making the request, and no, there's no way to know which region it's spawned in

Isaac McFadyen•7/24/24, 2:23 AM

^ to add to this, Hyperdrive is always spawned in a region with a DO but may not spawn in every DO region (I'm uncertain, only someone on the Hyperdrive team would know). But you can certainly use the non-DO regions as an indicator of where it does not spawn. Here's a community-sourced tool for exploring DO regions: https://where.durableobjects.live/

Tracking where Durable Objects are created, wherever you are in the world.

João Castro•7/29/24, 8:07 PM

It seems like sometimes the EU Hyperdrive pool for our DB is going unavailable for a few minutes and then coming back online. Could someone take a look at this? Thanks!

Account ID: 978f7d8fe0d7c314743586a46e505f14

João Castro•7/29/24, 8:08 PM

While testing we noticed that having a worker close to London try to connect would result in an error, but in other regions it would run properly

Elektra•7/31/24, 10:22 AM

hello and good day!
im trying to understand. Every other day HyperDrive fails for me for about 1 hour...
and then comes back.
like right now is failing : error: Failed to acquire a connection from the pool.
and there is nothing i can do because i know its not my side, because i can connect directly to the DB. (Yes, this could be a solution, trying to monitor these error and switch to direct connection)
but i think i should be able to work with hyperdrive.

Elektra•7/31/24, 10:23 AM

is anyone else experiencing this?

Elektra•7/31/24, 10:28 AM

i dont even do that many things.... but was hoping to do in the future...

Elektra•7/31/24, 10:28 AM

there is the timeouts....

Elektra•7/31/24, 10:33 AM

and connections not overwhelmed or anything...

Elektra•7/31/24, 10:45 AM

Just wondering if anyone is experiencing this and if there is anyway to solve this.
I think what ill do for the time being is to also put direct connections to the DB and figure out how to failover during these hours.
on the pg side of things:
2024-07-31 10:38:16.201 UTC [569089] [unknown]@[unknown] LOG: invalid length of startup packet
2024-07-31 10:20:26.462 UTC [568369] postgres@db LOG: could not receive data from client: Connection timed out
2024-07-31 10:20:26.462 UTC [568369] postgres@db LOG: unexpected EOF on client connection with an open transaction

Ccopper @Ping for toast saw that you use/play around with Hyperdrive; how much latency d...

Ping for toast•7/31/24, 3:56 PM

this is EU client → US host, I'm going US client → EU host. Maybe I should run formal query benchmarks but anecdotally yes, my latency is anywhere between 320-590 or so on a warm conn

Ping for toast•7/31/24, 3:57 PM

I've decided to delay my db performance testing on AWS until we're closer to launch

Ping for toast•7/31/24, 3:57 PM

no point in paying 3x more for the same resources until then :)

Ping for toast•7/31/24, 4:02 PM

the plus side is our read queries are now quite optimized, and a replication story could be good for that as well

[ERROR] A request to the Cloudflare API (/accounts/[account-id]/hyperdrive/configs/[config-id]) failed. Internal Server Error [code: 10000] Token Permissions: If scopes are missing, you may need to logout and re-login. Scope (Access) - account (read) - user (read) - workers (write) - workers_kv (write) - workers_routes (write) - workers_scripts (write) - workers_tail (read) - d1 (write) - pages (write) - zone (read) - ssl_certs (write) - constellation (write) - ai (write) - queues (write) - offline_access

I have some Cloudflare Pages apps that needs to access a Postgres DB. I also have traditional worklo

Similar Threads

Similar Threads

Similar Threads