Thanks. What about rawdawg direct mode (No connection pooler)? Bad idea?
Thanks. What about rawdawg direct mode (No connection pooler)? Bad idea?
ZI feel like it would be a good idea (just intuitively, no evidence)
AIf you're putting Hyperdrive on top of that, direct mode is fine too. If you need to do some particularly heavy connection multiplexing, having the supabase knobs to turn might prove useful but in most cases shouldn't be necessary.
Z
Zwhatev I'll just get an ipv4 address
ZGitHub
REST API for any Postgres database. Contribute to PostgREST/postgrest development by creating an account on GitHub.
Zimagine I could run postgrest in a worker
Zit's haskell. maybe it could happen
Ehello anyone experiencing:
Database error: error: Failed to acquire a connection from the pool.
Database error: error: Failed to acquire a connection from the pool.
TAny details on this? Using Hyperdrive, what's the database provider, etc.?
EHi Thomas, I'm hosting my own instance of pg I'm using the recommended library for connecting. I never had any such issues before that's why I was asking. Been fine for the last 4 months since I've used it. I waited a bit then went to pgadmin to investigate (could have been my side, although I tested while waiting with other apis that I have apart from this worker and worked fine) and about 5 min after all was okay again. Went to the status page also of cloudflare but didn't see anything at the time. Maybe some transient error...
EBy the the the connections was not exhausted at all. I have the stock 100 and only 7 were active. Although I now don't recall seeing cloudflare mentioned on the connections... So not sure... Unfortunately I do see the errors in the dashboard but... I can't see anything else except from that log message that I was getting. So I don't really have much more information at the moment.
BWhile waiting for Cloudflare Tunnel support for Hyperdrive, are there any resources available about connecting to AWS RDS (via RDS Proxy) directly through a Cloudflare Tunnel from a worker?
IHelloo everyone 
I facing a puzzling situation. I have a postgres database that I am connecting to via hyperdrive that is run on a server with a firewall which has all of cloudflare's IP's whitelisted. When I trigger the cloudflare worker that uses hyperdrive to do database operations from my home (in Spain) this works. However this worker is a callback handler and whenever the service connects (from Slovakia) the database operations silently fail. I remember working with our sysAdmin to whitelist a different IP so my questions would be, is failure something to do with the firewall? What are the IP's that hyperdrive uses to connect to databases? Would these IP's be different between Spain and Slovakia. And if not does anyone have any idea what could be happening?
I facing a puzzling situation. I have a postgres database that I am connecting to via hyperdrive that is run on a server with a firewall which has all of cloudflare's IP's whitelisted. When I trigger the cloudflare worker that uses hyperdrive to do database operations from my home (in Spain) this works. However this worker is a callback handler and whenever the service connects (from Slovakia) the database operations silently fail. I remember working with our sysAdmin to whitelist a different IP so my questions would be, is failure something to do with the firewall? What are the IP's that hyperdrive uses to connect to databases? Would these IP's be different between Spain and Slovakia. And if not does anyone have any idea what could be happening?
RIs there a relationship between Hyperdrive cache settings and the use of prepared statements?
I am exploring Postgres through Rust (using the
p.s. This seems to have changed recently. Earlier,
I am exploring Postgres through Rust (using the
tokio-postgres crate) using the query method. This uses the extended query protocol. I am observing the following behavior. When the Hyperdrive cache is:- Disabled: All (or most) queries fail (the driver reports an
UnexpectedMessageerror). I suspect disabled caching makes Hyperdrive use one connection for preparation and another execution on another. Occasionally, it might use the same connection, leading to a successful query. - Enabled: All queries succeed. I suspect caching makes Hyperdrive choose the same connection it used to prepare the statement.
p.s. This seems to have changed recently. Earlier,
query would always fail regardless of the cache setting. So, I started working on a PR for the Postgres driver to make it work with Hyperdrive. But today, during testing that PR, I saw this behavior, thus this question.An asynchronous PostgreSQL client.
Mis something going on with cloudflare?
Mas of 2 hours ago my workers are performing terribly, same number of requests
MTwo questions for the CF Hyperdrive team:
- Given HD's caching capabilities, is it redundant to also utilise read replicas on my DB, or would they still be a good idea?
- Six months or so ago I and others noticed that HD had a bug whereby, if caching was enabled, sometimes queries returned erroneous results e.g. no rows where there should definitely be rows. I recall vaguely that a CF dev said this was acknowledged and a fix was scheduled. Did this happen, i.e. is it safe for me to re-enable caching? (I disabled it at the time due to this).
Thanks in advance.
Mi seem to be having serious performance issues with hyperdrive connected to a postgres db in hetzner.
why is latency like this? other graphs are relatively flat, but errors and latency are spiking and users are reporting slowness.
why is latency like this? other graphs are relatively flat, but errors and latency are spiking and users are reporting slowness.
PCan we get some verbiage on the Supabase + Hyperdrive setup page in regards to the Supabase pooler? Do we use the pooler connection string or the direct connection string from the Supabase Database settings page?
https://developers.cloudflare.com/hyperdrive/examples/supabase/
https://developers.cloudflare.com/hyperdrive/examples/supabase/
Cloudflare Docs
This example shows you how to connect Hyperdrive to a Supabase Postgres database.
Mall users reporting issues seem to be in asia
Pand so it begins... for some reason i am not able to see the HYPERDRIVE binding from getPlatformProxy within my SvelteKit hooks handler
ATaking a look now.
A- Depends on your use case. There's overlap in the problems they solve, certainly, but there's also reasons you might want both.
- Yes, the issue with caching returning empty results was resolved quite a while back
AWe recently added support for prepared statements. Caching should not interact with it in this way, no. I'll add a ticket for us to take a look at this.
RThanks. That's good to know. I guess https://developers.cloudflare.com/hyperdrive/configuration/connect-to-postgres/#transaction-and-statement-support needs to change to drop the limitation on PREPARE?
Cloudflare Docs
Hyperdrive supports PostgreSQL and PostgreSQL-compatible databases, popular drivers and Object Relational Mapper (ORM) libraries that use those …
ANot exactly. We support protocol-level prepared statements, like your driver uses. We do not parse out and manage it as a keyword right now. At some point if there's enough need we could but we don't see that a lot honestly
RAh, I see. Thank you for the clarification.
RBTW, a way to reproduce would be to take this example: https://github.com/cloudflare/workers-rs/blob/main/examples/tokio-postgres/src/lib.rs#L29 and change the line to use
query instead of simple_query.GitHub
Write Cloudflare Workers in 100% Rust via WebAssembly - cloudflare/workers-rs
MGitHub
Refs: #1017
Refs: #1044
This PR implements unnamed statements when querying the database. This is not a performance optimization. The library still uses a "prepared" statement in order to...
Refs: #1044
This PR implements unnamed statements when querying the database. This is not a performance optimization. The library still uses a "prepared" statement in order to...
M@moody 
(I can't DM you) - https://www.cloudflarestatus.com/incidents/q23jjvwggp78
(I can't DM you) - https://www.cloudflarestatus.com/incidents/q23jjvwggp78 - for clarity, we only have one report so far, but are investigating.
KYeah, like Matt linked to, that example uses
https://github.com/cloudflare/workers-rs/pull/569#issuecomment-2098304851
simple_query intentionally.https://github.com/cloudflare/workers-rs/pull/569#issuecomment-2098304851
R@kian @elithrar @AJR Thanks for your responses.
Is there a document that describes the pooling mechanism? I am particularly interested in how prepared statements behave if I make two identical queries.
Would the statement prepared in the first query be cached? In other words, would Hyperdrive call
A bit of context:
I am in the process of supporting Cloudflare Worker for Exograph (https://github.com/exograph/exograph/pull/1081). It is working okay without Hyperdrive (high latency) and great with Hyperdrive (with a fork of tokio-postgres to avoid prepared statements: https://github.com/sfackler/rust-postgres/pull/1147).
I would like to determine the extent of Hyperdrive support to decide if there is value in continuing to use my fork. For example, if Hyperdrive reuses the prepared statement, then going with
Is there a document that describes the pooling mechanism? I am particularly interested in how prepared statements behave if I make two identical queries.
Would the statement prepared in the first query be cached? In other words, would Hyperdrive call
prepare twice (as in https://github.com/sfackler/rust-postgres/blob/98f5a11bc0a8e451552d8941ffa078c7eb6cd60c/tokio-postgres/src/to_statement.rs#L18, which is invoked from https://github.com/sfackler/rust-postgres/blob/98f5a11bc0a8e451552d8941ffa078c7eb6cd60c/tokio-postgres/src/client.rs#L363.A bit of context:
I am in the process of supporting Cloudflare Worker for Exograph (https://github.com/exograph/exograph/pull/1081). It is working okay without Hyperdrive (high latency) and great with Hyperdrive (with a fork of tokio-postgres to avoid prepared statements: https://github.com/sfackler/rust-postgres/pull/1147).
I would like to determine the extent of Hyperdrive support to decide if there is value in continuing to use my fork. For example, if Hyperdrive reuses the prepared statement, then going with
query will be better (to be measured, of course). If not, my fork avoids an extra roundtrip.AI don't think this is well documented. I'm happy to write up some explanation when I have some spare cycles. Hopefully today or tomorrow.
Short answer, it's complicated. We reuse the prepared statement if possible, or we re-send the parse on the new connection if not.
Short answer, it's complicated. We reuse the prepared statement if possible, or we re-send the parse on the new connection if not.
Mty. i think the no dms setting is on your end
RThat will be awesome. Thank you!
RJust a clarification: The use of
simple_query was required earlier (I believe until May 22 https://developers.cloudflare.com/hyperdrive/platform/changelog/#2024-05-22). With the prepared statement support in Hyperdrive, plain query also works with a caveat I mentioned (must turn on caching in Hyperdrive setting). The caveat is an issue in Hyperdrive (per @AJR), so once that is fixed, it will be all okay!Cloudflare Docs
You can now create up to 25 Hyperdrive configurations per account, up from the previous maximum of 10.
PIs Hyperdrive smart enough to clear the cache of a SELECT query if a mutate has happened for that row? For instance, the user profile is cached but then they change their bio. If they refresh would they still see the old profile until “max-age”?
Imax-age is related to browser-side caching using HTTP headers. Hyperdrive has no ability to purge the browser-side cache so no.AWithout getting into browser caching, no Hyperdrive does not do proactive cache purging after volatile queries. It's something we've discussed but would be a non-trivial feature to add and generally is not something users have asked for overmuch.
If you need to bypass cache on specific queries, we don't check cache on any volatile queries, so you could add any such function. For example including
If not purging cache on mutation is a dealbreaker feature for you, it would be helpful for us to understand a bit about your use case, if you'd be willing to chat about it.
If you need to bypass cache on specific queries, we don't check cache on any volatile queries, so you could add any such function. For example including
random() and ignoring that field in your result would do it.If not purging cache on mutation is a dealbreaker feature for you, it would be helpful for us to understand a bit about your use case, if you'd be willing to chat about it.
II'd also add that max-age is also browser-side along with server-side.
ISo even if Hyperdrive could purge the cache somehow (which seems like a bad idea IMO, a DB connection shouldn't be able to modify my CDN config) then it won't affect browser-side caching anyway.
PHmm seems to be a few caching layers here. Let’s say the user who owns the profile has edited it and expects to see the changes immediately. Is there a way to cache bust just for that user? (I’m not understanding how the browser cache plays along with the hyperdrive layer)
ISo these are the cache layers I assume you have (correct me here if I'm wrong):
- Browser cache, this is local on the user's web browser and based on
max-age. This is quite tricky to bust and you usually disable this for APIs in favor of 2. - Server-side cache, this is the Cloudflare CDN and/or your server and/or the Worker that is serving the API requests. To purge this you need control of the Worker, or if it's the CDN cache you need to make an API call to the Cloudflare API. This is also based on
maxage, but if you want to only cache on the server-side you can uses-maxageinstead. - Hyperdrive query cache. I don't know many details on this, @AJR would know more about how cache busting works here.
IYour original question was about whether a query in Hyperdrive would auto-cache-purge 1 or 2. It doesn't do that because it's a separate system and it's unintuitive/broken behavior if a database connection cache purges the other layers. The developer should have full control over whether that happens and if so, when and how.
AThat is all correct to the best of my understanding. My answer above about including a benign volatile function is the current best solution for cache busting point 3. We expect to add more polish on that front, though it isn't our highest priority at this time.
IAh completely missed the volatile function part - good to know.
