Hi folks, what would be good managed postgre candidate to use with hyperdrive? (Or workers in genera
Hi folks, what would be good managed postgre candidate to use with hyperdrive? (Or workers in general)
MI see Neon and Supabase mentioned quite often
CJust throwing in my 2c, we evaluated all of the popular “serverless” Postgres DBs and none of them ticked all the boxes/gave us as much confidence as AWS Aurora
CI feel like Neon will be a great choice in a year or two, but rn it seemed to lack some basic features that AWS has
IGoogle Cloud SQL is also quite good if you're already integrated into their ecosystem.
AI know more databases are coming later, but may I still ask if ClickHouse is on your list?
MNot yet, no. The ClickHouse protocol doesn’t have a specification: https://clickhouse.com/docs/en/interfaces/tcp
The native protocol is used in the command-line client, for inter-server communication during distributed query processing, and also in other C++ programs. Unfortunately, native ClickHouse protocol does not have formal specification yet, but it can be reverse-engineered from ClickHouse source code (starting around here) and/or by intercepting an...
MAlso: are you querying ClickHouse on every request? I can’t imagine that will be fast regardless.
DWith hyperdrive’s caching that could become possible 
MThe caching relies on the protocol being specified (and a stable parser being available, plus a JS driver). But I’m asking as if you’re instead just querying it periodically/in a Cron/etc, the HTTP interface for ClickHouse is likely a more straightforward approach right now.
AAh I didn't know. I guess an experiment with their PostgreSQL protocol is necessary
ASry I'm not sure I see why you think this is a problem.
ClickHouse for OLAP is very performant. I use it for about 600 dashboards (for different users), and yes queries every time (but cached with a worker). It's also what powers Cloudflare's Analytics Engine IIRC.
ClickHouse for OLAP is very performant. I use it for about 600 dashboards (for different users), and yes queries every time (but cached with a worker). It's also what powers Cloudflare's Analytics Engine IIRC.
AIt's got an HTTP endpoint to run queries, and honestly I might just stick to the Worker for caching + HTTP. I guess the most interesting part for me was the connection pooling.
MI am very aware of how ClickHouse performs. What I am trying to point out is that if you rely on native ClickHouse support you may be waiting a long time, and that CH’s baseline performance may be “fast” but isn’t commonly sub-second for anything but the most simple queries.
Instead, using a HTTP interface will likely get you moving in the meantime, or trying their Postgres interface (untested).
Instead, using a HTTP interface will likely get you moving in the meantime, or trying their Postgres interface (untested).
AYeah, I use their HTTP interface since Workers didn't have TCP support yet when we decided to use it and I couldn't be bothered with creating and maintaining a HTTP proxy for any other OLAP/time-series DB. Performance-wise, the queries seem to run fast enough (20-200ms, we really aren't the same size as CF), helped with caching, but the latency is spiky (I suspect its due to making new the connections but haven't investigated yet).
AAnyway, it's all good to know. Might try the Postgres interface some time later 
CWorkers have outbound TCP connections.
ANow, yes. Not last year.
DIt's not so supabase specific, actually. One of their core principles is using existing or their own open source software. In this case it's https://github.com/PostgREST/postgrest
MI mention pgrest in my comment.
DYeah I know, just wanted to throw out there that it's not super specific. Neon also has PostgREST support.
MNone of the MySQL drivers work (yet) in Workers. You will need a serverless driver and thus a MySQL provider that allows HTTP/WS access - e.g. https://docs.pingcap.com/tidbcloud/integrate-tidbcloud-with-cloudflare
KThere's https://github.com/nora-soderlund/cloudflare-mysql which is https://github.com/mysqljs/mysql adapted by a community member for the TCP Sockets API
KIt has the same downsides as Postgres w/o Hyperdrive
M@nora have you attempted to upstream the sockets support?
MYes.
MGitHub
fast mysqljs/mysql compatible mysql driver for node.js - GitHub - sidorares/node-mysql2: fast mysqljs/mysql compatible mysql driver for node.js
fast mysqljs/mysql compatible mysql driver for node.js - GitHub - sidorares/node-mysql2: fast mysqljs/mysql compatible mysql driver for node.jsMSame as Postgres libs to get SCRAM working. Does yours require
nodejsnode
JLooks like there's some exciting new developments in the Cloudflare dashboard for Hyperdrive 
https://dash.cloudflare.com/?to=/:account/workers/hyperdrive
https://dash.cloudflare.com/?to=/:account/workers/hyperdrive
PI have been spinning wheels on this for a good part of the day. I have HyperDrive (cockroachDB/pg) working with two workers, for whatever reason and I can't figure out what it is, when I run client.connect, it just breaks out of the routine. it doesn't throw an error.
The codes are super duper vanilla:
How do I track this down? It really feels like it has to be config related, but my wrangler files are identical between the three workers.
The codes are super duper vanilla:
How do I track this down? It really feels like it has to be config related, but my wrangler files are identical between the three workers.
MCan you share more details?
- what version of node-Postgres? Definitely the same with both?
- your binding config?
- your hyperdrive config ID?
MHi all. I'm using Hyperdrive with Postgres via
node-postgresclient.query()pool.query()Hey Matt!
Here is the binding config.. The weird thing is that all of the workers are hitting the exact same DB and instance.
I have attached screen shots of the Hyperdrive config.
Here is the binding config.. The weird thing is that all of the workers are hitting the exact same DB and instance.
I have attached screen shots of the Hyperdrive config.
P
MWhy does Wrangler tell me to put the PG connection string (for Hyperdrive use) in my
wrangler.toml.dev.varsWhat is the error message that tells you to do that?
MERROR: In development, you should use a local postgres connection string to emulate hyperdrive functionality. Please setup postgres locally and set the value of "HYPERDRIVE"'s "localConnectionString" to the postgres connection string in your wrangler.toml
MMy dev DB lives on Heroku, so it's a remote DB, not locally-hosted
Mcan you file a feature request for that? https://github.com/cloudflare/workers-sdk/issues/new?assignees=&labels=enhancement&projects=&template=feature-request.yaml&title=%F0%9F%9A%80+Feature+Request%3A
MYou don’t need to manage the pool in the client. Further, because Workers are typically short lived, your “pool” wouldn’t last any longer than your Worker invocation. Hyperdrive’s pooling is much, much more robust.
SLetsgoooo, finally! Thanks team
MWill do.
MGreat! Thanks.
MIs it possible to load one TOML in local dev and another for live? I'm aware that the TOML files supports environments, but I'm trying to circumvent this issue. Guess not if the file must be named exactly
wrangler.tomlyou can pass a path to the config file
MCreate, develop, and deploy your Cloudflare Workers with Wrangler commands.
MAh, interesting