This should work. - Cloudflare Developers

C

Hey! Are there plans to add hyperdrive support to the cloudflare terraform provider?

K

kian•1/31/24, 9:30 AM

It's publicly documented (https://developers.cloudflare.com/api/operations/create-hyperdrive) so it just needs to be added into cloudflare-go and cloudflare-terraform-provider.

K

kian•1/31/24, 9:31 AM

You can open an issue at https://github.com/cloudflare/terraform-provider-cloudflare/issues to request it

MMatt Silverlock This should work.

S

Sam•1/31/24, 3:21 PM

Can confirm

CColin hey y'all - wanted to talk about the `localConnectionString` in `wrangler.toml` ...

M

Matt SilverlockOP•1/31/24, 4:04 PM

We're going to introduce an env var that takes predecence over

wrangler.toml

wrangler.toml

localConnectionString

localConnectionString

localConnectionString

localConnectionString

C

carlos-alberto•1/31/24, 4:14 PM

I opened a PR at https://github.com/cloudflare/cloudflare-go/pull/1492 to add the support for Hyperdrive to cloudflare-go, hope it's okay, first

go

go

go

go attempt

GitHub

Add Hyperdrive CRUD support by carlos-alberto · Pull Request #1492 ...

Add support for managing Hyperdrive configs
Description
Resolves #1491
Mainly wrote by copying the existing queue.go
Has your change been tested?

Tested ListHyperdriveConfigs manually
Added tests
...

P

Paul•2/2/24, 8:48 PM

if i'm using hyperdrive to connect to a database that handles my user auth sessions, and i have caching enabled with 60 second max_age, and someone logs out... does that mean that for the next 60 seconds they might be pulling their old cached session?

JJames Unfortunately Pages Functions doesn’t support node_compat bundling, nor the hype...

P

Paul•2/3/24, 12:49 AM

This is tragic for me -- is this still the case?

PPaul This is tragic for me -- is this still the case?

J

James•2/3/24, 1:17 AM

Unfortunately so, yeah. There's no real way to get Hyperdrive on Pages Functions today - you will have to use Workers as far as I know.

PPaul if i'm using hyperdrive to connect to a database that handles my user auth sessi...

M

Matt SilverlockOP•2/3/24, 8:48 AM

Only if the query is exactly the same. I suggest reading https://developers.cloudflare.com/hyperdrive/configuration/query-caching/

Cloudflare Docs

Query caching · Hyperdrive

Hyperdrive automatically caches the most popular queries executed against your database, reducing the need to go back to your database (incurring …

PPaul This is tragic for me -- is this still the case?

M

Matt SilverlockOP•2/3/24, 8:49 AM

Hyperdrive is in beta. We’re working on Pages support. Hold tight.

MMatt Silverlock Only if the query is exactly the same. I suggest reading https://developers.clou...

P

Paul•2/3/24, 12:45 PM

Interesting, I wonder if I could add some sort of cache busting useless string to the query just to make sure I get fresh data. Is this a common approach?

PPaul Interesting, I wonder if I could add some sort of cache busting useless string t...

M

Matt SilverlockOP•2/3/24, 12:47 PM

Or as that guide states, use a non-caching Hyperdrive if you rely on non-cached session data.

M

Matt SilverlockOP•2/3/24, 12:47 PM

Cache busting with

random()

random()

random()

random() or some other non-deterministic function will have the same effect.

P

Paul•2/3/24, 12:48 PM

Sweet! Volatile functions too! Nice article

P

Paul•2/3/24, 12:49 PM

Cloudflare Pages support would be amazing. I was at it all day trying to get it to work. node_compat vs nodejsCompat. Bit of a nightmare

P

Paul•2/3/24, 12:50 PM

Is there any GitHub issue where I could track progress? Or any eta you might be able to share?

Jjonfanz 👋 (cc: @divided421) Jon from Prisma here. Accelerate does support Mongo. Agree ...

B

bryan•2/4/24, 1:17 AM

thanks Jon! is there a way to economically run accelerate just for connection pooling?

M

Mitya•2/5/24, 4:34 PM

I'm getting errors when trying to create a HD, saying the user/pass is probably wrong. The connection string is correct - I've tested it on my local machine. The DB is remote but unrestricted in terms of IP access. The password contains characters such as

_

_,

!

! and

^

^. Are these likely to be the problem?

M

Mitya•2/5/24, 4:35 PM

Command:

wrangler hyperdrive create "my-db" --connection-string="postgres://user:pass@host:5432/db"

wrangler hyperdrive create "my-db" --connection-string="postgres://user:pass@host:5432/db"

wrangler hyperdrive create "my-db" --connection-string="postgres://user:pass@host:5432/db"

wrangler hyperdrive create "my-db" --connection-string="postgres://user:pass@host:5432/db"

M

Mitya•2/5/24, 4:35 PM

(I've been able to successfully create HDs before, hence I'm thinking it's password characters.)

M

Mitya•2/5/24, 4:37 PM

Ah - it works if I create the HD via the dashboard instead (same connection string).

M

Mitya•2/5/24, 4:37 PM

(UX issue: When creating a HD via the dashboard, if you enter a bad connection string, there's no feedback given to the user.)

.

.Zero•2/8/24, 7:57 AM

how stable and ready for production is the connection pooling of hyperdrive?

.

.Zero•2/8/24, 7:57 AM

Can I throw this into production where we write around 25k lines per second? Up to 800Mbps disk writes?

C

carlos-alberto•2/8/24, 3:47 PM

Hey! I’ve submitted a PR to add terraform support for hyperdrive: https://github.com/cloudflare/terraform-provider-cloudflare/pull/3111

However in our testing we’ve noticed that the API docs are inconsistent, they state that

password

password

is a top level value (https://developers.cloudflare.com/api/operations/create-hyperdrive), in our testing we’ve found that it should be nested within

origin

origin

origin

origin.

Before I update my PR, and submit a new PR to the cloudflare-go repo, it would be good to clarify whether the API itself or the API docs are where the issue lies? I.e will the docs be updated to nest

password

password

in

origin

origin

origin

origin? Or the API updated to expose

password

password

at the top level?

GitHub

Add Hyperdrive Resource provider by carlos-alberto · Pull Request #...

Implements #3092
Requires cloudflare/cloudflare-go#1491 to be released.
Note: Not sure if the acctest is actually viable, as I believe DB creds need to actually be valid for the api request to work.

Cloudflare API Documentation

Interact with Cloudflare's products and services via the Cloudflare API

Ccarlos-alberto Hey! I’ve submitted a PR to add terraform support for hyperdrive: https://github...

M

Matt SilverlockOP•2/8/24, 4:08 PM

API docs are incorrect -

password

password

should be nested within the

origin

origin

origin

origin object - e.g. https://github.com/cloudflare/workers-sdk/blob/d637bd59a8ea6612d59ed4b73e115287615e617d/packages/wrangler/src/hyperdrive/client.ts#L23

GitHub

workers-sdk/packages/wrangler/src/hyperdrive/client.ts at d637bd59a...

Home to Wrangler, the CLI for Cloudflare Workers® - cloudflare/workers-sdk

MMatt Silverlock API docs are incorrect - `password` should be nested within the `origin` object ...

C

carlos-alberto•2/8/24, 4:09 PM

Perfect thank you, will get a new PR up in cloudflare-go and update the terraform one

Ccarlos-alberto Perfect thank you, will get a new PR up in cloudflare-go and update the terrafor...

M

Matt SilverlockOP•2/8/24, 4:11 PM

The types there are fairly stable (beta notwithstanding; always risk of some changes)

MMatt Silverlock The types there are fairly stable (beta notwithstanding; always risk of some cha...

C

carlos-alberto•2/8/24, 4:30 PM

Gotcha,

Updated https://github.com/cloudflare/terraform-provider-cloudflare/pull/3111

And opened https://github.com/cloudflare/cloudflare-go/pull/1501

Thanks for your help

M

mrwind•2/8/24, 11:34 PM

is hyperdrive working with mongodb (on mongo atlas)?

Mmrwind is hyperdrive working with mongodb (on mongo atlas)?

S

Sam•2/9/24, 7:42 AM

No, it's not

Mmrwind is hyperdrive working with mongodb (on mongo atlas)?

E

Emilio•2/9/24, 9:03 PM

We currently support Postgres, and plan to support MySQL soon.

Q

quambo•2/10/24, 11:51 AM

Anyone managed to hyperdrive working with remote database + kysely + local development

Using

pg@8.11.0

pg@8.11.0

pg@8.11.0

pg@8.11.0, initiating kysely like this:

const connectionString =
      env.LOCAL_DATABASE_URL || env.HYPERDRIVE.connectionString;
new Kysely<DB>({
    dialect: new PostgresDialect({
      pool: new Pool({
        max: 1,
        connectionString,
        ssl: {
          rejectUnauthorized: false,
        },
      }),
    }),
  });

const connectionString =
      env.LOCAL_DATABASE_URL || env.HYPERDRIVE.connectionString;
new Kysely<DB>({
    dialect: new PostgresDialect({
      pool: new Pool({
        max: 1,
        connectionString,
        ssl: {
          rejectUnauthorized: false,
        },
      }),
    }),
  });

const connectionString =
      env.LOCAL_DATABASE_URL || env.HYPERDRIVE.connectionString;
new Kysely<DB>({
    dialect: new PostgresDialect({
      pool: new Pool({
        max: 1,
        connectionString,
        ssl: {
          rejectUnauthorized: false,
        },
      }),
    }),
  });

const connectionString =
      env.LOCAL_DATABASE_URL || env.HYPERDRIVE.connectionString;
new Kysely<DB>({
    dialect: new PostgresDialect({
      pool: new Pool({
        max: 1,
        connectionString,
        ssl: {
          rejectUnauthorized: false,
        },
      }),
    }),
  });

Using

LOCAL_DATABASE_URL

LOCAL_DATABASE_URL

LOCAL_DATABASE_URL

LOCAL_DATABASE_URL for local development (connectionstring including

?sslmode=require

?sslmode=require

?sslmode=require

?sslmode=require), I'm able to setup the connection, but any query will yield

Error: Connection terminated unexpectedly

Error: Connection terminated unexpectedly

Error: Connection terminated unexpectedly

Error: Connection terminated unexpectedly.

Been trying to use

localConnectionString

localConnectionString

localConnectionString

localConnectionString instead of bypassing hyperdrive in local dev, but it transforms my connectionstring to some

hyperdrive.local

hyperdrive.local

hyperdrive.local

hyperdrive.local url.

Qquambo Anyone managed to hyperdrive working with remote database + kysely + local devel...

X

xatxat•2/10/24, 2:08 PM

i'm using the same approach with kysely + local development. works fine for me.

X

xatxat•2/10/24, 2:10 PM

i don't have

?sslmode=require

?sslmode=require

?sslmode=require

?sslmode=require in my connection string

Q

quambo•2/10/24, 2:34 PM

I just can't get it to work

trying postgres package, also no success, error

[wrangler:err] Error: internal error
    at cachedError (/x/node_modules/postgres/cf/src/query.js:170:23)
    at new Query (/x/node_modules/postgres/cf/src/query.js:36:24)
    at sql2 (/x/node_modules/postgres/cf/src/index.js:113:11)
    at Object.fetch (/x/index.ts:9:23)

[wrangler:err] Error: internal error
    at cachedError (/x/node_modules/postgres/cf/src/query.js:170:23)
    at new Query (/x/node_modules/postgres/cf/src/query.js:36:24)
    at sql2 (/x/node_modules/postgres/cf/src/index.js:113:11)
    at Object.fetch (/x/index.ts:9:23)

[wrangler:err] Error: internal error
    at cachedError (/x/node_modules/postgres/cf/src/query.js:170:23)
    at new Query (/x/node_modules/postgres/cf/src/query.js:36:24)
    at sql2 (/x/node_modules/postgres/cf/src/index.js:113:11)
    at Object.fetch (/x/index.ts:9:23)

[wrangler:err] Error: internal error
    at cachedError (/x/node_modules/postgres/cf/src/query.js:170:23)
    at new Query (/x/node_modules/postgres/cf/src/query.js:36:24)
    at sql2 (/x/node_modules/postgres/cf/src/index.js:113:11)
    at Object.fetch (/x/index.ts:9:23)

index.ts

import postgres from 'postgres';

export default {
  async fetch(request: Request, env: any) {
    const connectionString =
      env.DATABASE_URL || env.HYPERDRIVE.connectionString;
    console.log('fetch', request.url, env.DATABASE_URL);
    const sql = postgres(connectionString);
    const abc = await sql`SELECT * FROM tenant LIMIT 10`;
    console.log('abc', abc);

    return new Response(JSON.stringify({ ok: true }));
  },
};

import postgres from 'postgres';

export default {
  async fetch(request: Request, env: any) {
    const connectionString =
      env.DATABASE_URL || env.HYPERDRIVE.connectionString;
    console.log('fetch', request.url, env.DATABASE_URL);
    const sql = postgres(connectionString);
    const abc = await sql`SELECT * FROM tenant LIMIT 10`;
    console.log('abc', abc);

    return new Response(JSON.stringify({ ok: true }));
  },
};

import postgres from 'postgres';

export default {
  async fetch(request: Request, env: any) {
    const connectionString =
      env.DATABASE_URL || env.HYPERDRIVE.connectionString;
    console.log('fetch', request.url, env.DATABASE_URL);
    const sql = postgres(connectionString);
    const abc = await sql`SELECT * FROM tenant LIMIT 10`;
    console.log('abc', abc);

    return new Response(JSON.stringify({ ok: true }));
  },
};

import postgres from 'postgres';

export default {
  async fetch(request: Request, env: any) {
    const connectionString =
      env.DATABASE_URL || env.HYPERDRIVE.connectionString;
    console.log('fetch', request.url, env.DATABASE_URL);
    const sql = postgres(connectionString);
    const abc = await sql`SELECT * FROM tenant LIMIT 10`;
    console.log('abc', abc);

    return new Response(JSON.stringify({ ok: true }));
  },
};

package.json

{
  "name": "x",
  "dependencies": {
    "postgres": "^3.4.3"
  }
}

{
  "name": "x",
  "dependencies": {
    "postgres": "^3.4.3"
  }
}

{
  "name": "x",
  "dependencies": {
    "postgres": "^3.4.3"
  }
}

{
  "name": "x",
  "dependencies": {
    "postgres": "^3.4.3"
  }
}

wrangler.toml

name = "xyz"
compatibility_flags = ["nodejs_compat"]

name = "xyz"
compatibility_flags = ["nodejs_compat"]

name = "xyz"
compatibility_flags = ["nodejs_compat"]

name = "xyz"
compatibility_flags = ["nodejs_compat"]

Q

quambo•2/10/24, 2:35 PM

i've been trying for hours to get this to work, but i keep on getting odd errors, with different wrangler versions and ssl true/false

Q

quambo•2/10/24, 2:56 PM

wonder if its got to do with https://github.com/cloudflare/workerd/issues/1401

GitHub

🐛 BUG: Uncaught exception · Issue #1401 · cloudflare/workerd

Which Cloudflare product(s) does this pertain to? Wrangler or miniflare What version(s) of the tool(s) are you using? 3.5.0 What version of Node are you using? 18.16.1 What operating system are you...

Qquambo wonder if its got to do with https://github.com/cloudflare/workerd/issues/1401

M

Matt SilverlockOP•2/10/24, 5:54 PM

That’s an old issue that seems entirely unrelated.

Does your database accept TLS/SSL connections?

Qquambo I just can't get it to work 😦 trying postgres package, also no success, error `...

M

Matt SilverlockOP•2/10/24, 5:56 PM

Are your hyperdrive bindings configured too? When you create a Hyperdrive via the dash or wrangler, Hyperdrive itself connects to make sure it’s reachable before “saving” the configuration.

Q

quambo•2/10/24, 6:59 PM

I found the reason @Matt.. after enabling logs, i could see certificate issues with my crunchybridge pg and the cf worker. Using

NODE_EXTRA_CA_CERTS=./assets/certificate.pem wrangler dev

NODE_EXTRA_CA_CERTS=./assets/certificate.pem wrangler dev

NODE_EXTRA_CA_CERTS=./assets/certificate.pem wrangler dev

NODE_EXTRA_CA_CERTS=./assets/certificate.pem wrangler dev solved the issue.

EEmilio We currently support Postgres, and plan to support MySQL soon.

B

berkinovish•2/12/24, 3:50 PM

any ETA on when we can expect MySQL support?

Bberkinovish any ETA on when we can expect MySQL support?

M

Matt SilverlockOP•2/14/24, 11:13 AM

At this stage, closer to April/May.

M

moody 💭•2/14/24, 8:54 PM

hi, i'm running into issues with my rds database slowing down based on client write waits

M

moody 💭•2/14/24, 8:55 PM

is hyperdrive bandwidth throttled at all?

M

moody 💭•2/14/24, 8:55 PM

M

moody 💭•2/14/24, 9:00 PM

when this happens i see spikes in latency

I

Isaac McFadyen•2/14/24, 9:02 PM

According to the RDS docs this can also happen if there's high latency between the client and the database.

I

Isaac McFadyen•2/14/24, 9:02 PM

Are you accessing it via a Worker that's far from the DB?