is hyperdrive working with mongodb (on mongo atlas)?

Mmrwind is hyperdrive working with mongodb (on mongo atlas)?

S

Sam•2/9/24, 7:42 AM

No, it's not

Mmrwind is hyperdrive working with mongodb (on mongo atlas)?

E

Emilio•2/9/24, 9:03 PM

We currently support Postgres, and plan to support MySQL soon.

Q

quambo•2/10/24, 11:51 AM

Anyone managed to hyperdrive working with remote database + kysely + local development

Using

pg@8.11.0

pg@8.11.0

pg@8.11.0

pg@8.11.0, initiating kysely like this:

const connectionString =
      env.LOCAL_DATABASE_URL || env.HYPERDRIVE.connectionString;
new Kysely<DB>({
    dialect: new PostgresDialect({
      pool: new Pool({
        max: 1,
        connectionString,
        ssl: {
          rejectUnauthorized: false,
        },
      }),
    }),
  });

const connectionString =
      env.LOCAL_DATABASE_URL || env.HYPERDRIVE.connectionString;
new Kysely<DB>({
    dialect: new PostgresDialect({
      pool: new Pool({
        max: 1,
        connectionString,
        ssl: {
          rejectUnauthorized: false,
        },
      }),
    }),
  });

const connectionString =
      env.LOCAL_DATABASE_URL || env.HYPERDRIVE.connectionString;
new Kysely<DB>({
    dialect: new PostgresDialect({
      pool: new Pool({
        max: 1,
        connectionString,
        ssl: {
          rejectUnauthorized: false,
        },
      }),
    }),
  });

const connectionString =
      env.LOCAL_DATABASE_URL || env.HYPERDRIVE.connectionString;
new Kysely<DB>({
    dialect: new PostgresDialect({
      pool: new Pool({
        max: 1,
        connectionString,
        ssl: {
          rejectUnauthorized: false,
        },
      }),
    }),
  });

Using

LOCAL_DATABASE_URL

LOCAL_DATABASE_URL

LOCAL_DATABASE_URL

LOCAL_DATABASE_URL for local development (connectionstring including

?sslmode=require

?sslmode=require

?sslmode=require

?sslmode=require), I'm able to setup the connection, but any query will yield

Error: Connection terminated unexpectedly

Error: Connection terminated unexpectedly

Error: Connection terminated unexpectedly

Error: Connection terminated unexpectedly.

Been trying to use

localConnectionString

localConnectionString

localConnectionString

localConnectionString instead of bypassing hyperdrive in local dev, but it transforms my connectionstring to some

hyperdrive.local

hyperdrive.local

hyperdrive.local

hyperdrive.local url.

Qquambo Anyone managed to hyperdrive working with remote database + kysely + local devel...

X

xatxat•2/10/24, 2:08 PM

i'm using the same approach with kysely + local development. works fine for me.

X

xatxat•2/10/24, 2:10 PM

i don't have

?sslmode=require

?sslmode=require

?sslmode=require

?sslmode=require in my connection string

Q

quambo•2/10/24, 2:34 PM

I just can't get it to work

trying postgres package, also no success, error

[wrangler:err] Error: internal error
    at cachedError (/x/node_modules/postgres/cf/src/query.js:170:23)
    at new Query (/x/node_modules/postgres/cf/src/query.js:36:24)
    at sql2 (/x/node_modules/postgres/cf/src/index.js:113:11)
    at Object.fetch (/x/index.ts:9:23)

[wrangler:err] Error: internal error
    at cachedError (/x/node_modules/postgres/cf/src/query.js:170:23)
    at new Query (/x/node_modules/postgres/cf/src/query.js:36:24)
    at sql2 (/x/node_modules/postgres/cf/src/index.js:113:11)
    at Object.fetch (/x/index.ts:9:23)

[wrangler:err] Error: internal error
    at cachedError (/x/node_modules/postgres/cf/src/query.js:170:23)
    at new Query (/x/node_modules/postgres/cf/src/query.js:36:24)
    at sql2 (/x/node_modules/postgres/cf/src/index.js:113:11)
    at Object.fetch (/x/index.ts:9:23)

[wrangler:err] Error: internal error
    at cachedError (/x/node_modules/postgres/cf/src/query.js:170:23)
    at new Query (/x/node_modules/postgres/cf/src/query.js:36:24)
    at sql2 (/x/node_modules/postgres/cf/src/index.js:113:11)
    at Object.fetch (/x/index.ts:9:23)

index.ts

import postgres from 'postgres';

export default {
  async fetch(request: Request, env: any) {
    const connectionString =
      env.DATABASE_URL || env.HYPERDRIVE.connectionString;
    console.log('fetch', request.url, env.DATABASE_URL);
    const sql = postgres(connectionString);
    const abc = await sql`SELECT * FROM tenant LIMIT 10`;
    console.log('abc', abc);

    return new Response(JSON.stringify({ ok: true }));
  },
};

import postgres from 'postgres';

export default {
  async fetch(request: Request, env: any) {
    const connectionString =
      env.DATABASE_URL || env.HYPERDRIVE.connectionString;
    console.log('fetch', request.url, env.DATABASE_URL);
    const sql = postgres(connectionString);
    const abc = await sql`SELECT * FROM tenant LIMIT 10`;
    console.log('abc', abc);

    return new Response(JSON.stringify({ ok: true }));
  },
};

import postgres from 'postgres';

export default {
  async fetch(request: Request, env: any) {
    const connectionString =
      env.DATABASE_URL || env.HYPERDRIVE.connectionString;
    console.log('fetch', request.url, env.DATABASE_URL);
    const sql = postgres(connectionString);
    const abc = await sql`SELECT * FROM tenant LIMIT 10`;
    console.log('abc', abc);

    return new Response(JSON.stringify({ ok: true }));
  },
};

import postgres from 'postgres';

export default {
  async fetch(request: Request, env: any) {
    const connectionString =
      env.DATABASE_URL || env.HYPERDRIVE.connectionString;
    console.log('fetch', request.url, env.DATABASE_URL);
    const sql = postgres(connectionString);
    const abc = await sql`SELECT * FROM tenant LIMIT 10`;
    console.log('abc', abc);

    return new Response(JSON.stringify({ ok: true }));
  },
};

package.json

{
  "name": "x",
  "dependencies": {
    "postgres": "^3.4.3"
  }
}

{
  "name": "x",
  "dependencies": {
    "postgres": "^3.4.3"
  }
}

{
  "name": "x",
  "dependencies": {
    "postgres": "^3.4.3"
  }
}

{
  "name": "x",
  "dependencies": {
    "postgres": "^3.4.3"
  }
}

wrangler.toml

name = "xyz"
compatibility_flags = ["nodejs_compat"]

name = "xyz"
compatibility_flags = ["nodejs_compat"]

name = "xyz"
compatibility_flags = ["nodejs_compat"]

name = "xyz"
compatibility_flags = ["nodejs_compat"]

Q

quambo•2/10/24, 2:35 PM

i've been trying for hours to get this to work, but i keep on getting odd errors, with different wrangler versions and ssl true/false

Q

quambo•2/10/24, 2:56 PM

wonder if its got to do with https://github.com/cloudflare/workerd/issues/1401

GitHub

🐛 BUG: Uncaught exception · Issue #1401 · cloudflare/workerd

Which Cloudflare product(s) does this pertain to? Wrangler or miniflare What version(s) of the tool(s) are you using? 3.5.0 What version of Node are you using? 18.16.1 What operating system are you...

Qquambo wonder if its got to do with https://github.com/cloudflare/workerd/issues/1401

M

Matt Silverlock•2/10/24, 5:54 PM

That’s an old issue that seems entirely unrelated.

Does your database accept TLS/SSL connections?

Qquambo I just can't get it to work 😦 trying postgres package, also no success, error `...

M

Matt Silverlock•2/10/24, 5:56 PM

Are your hyperdrive bindings configured too? When you create a Hyperdrive via the dash or wrangler, Hyperdrive itself connects to make sure it’s reachable before “saving” the configuration.

Q

quambo•2/10/24, 6:59 PM

I found the reason @Matt.. after enabling logs, i could see certificate issues with my crunchybridge pg and the cf worker. Using

NODE_EXTRA_CA_CERTS=./assets/certificate.pem wrangler dev

NODE_EXTRA_CA_CERTS=./assets/certificate.pem wrangler dev

NODE_EXTRA_CA_CERTS=./assets/certificate.pem wrangler dev

NODE_EXTRA_CA_CERTS=./assets/certificate.pem wrangler dev solved the issue.

EEmilio We currently support Postgres, and plan to support MySQL soon.

B

berkinovish•2/12/24, 3:50 PM

any ETA on when we can expect MySQL support?

Bberkinovish any ETA on when we can expect MySQL support?

M

Matt Silverlock•2/14/24, 11:13 AM

At this stage, closer to April/May.

M

moody 💭•2/14/24, 8:54 PM

hi, i'm running into issues with my rds database slowing down based on client write waits

M

moody 💭•2/14/24, 8:55 PM

is hyperdrive bandwidth throttled at all?

M

moody 💭•2/14/24, 8:55 PM

M

moody 💭•2/14/24, 9:00 PM

when this happens i see spikes in latency

I

Isaac McFadyen•2/14/24, 9:02 PM

According to the RDS docs this can also happen if there's high latency between the client and the database.

I

Isaac McFadyen•2/14/24, 9:02 PM

Are you accessing it via a Worker that's far from the DB?

M

moody 💭•2/14/24, 9:15 PM

i don't think so, my db is only in us-east-1 and i am noticing latency querying it from east coast

M

moody 💭•2/14/24, 9:16 PM

it seems like a download bandwidth issue on hyperdrive's side

M

moody 💭•2/14/24, 9:16 PM

i don't think i'm throttling the 10gbps network connection of the rds instance

M

moody 💭•2/14/24, 9:16 PM

and it's def not latency because it's only my really data heavy query

M

moody 💭•2/14/24, 9:17 PM

which btw also relates to a very high data egress bill from aws. i'm considering moving the compute to ec2 but it's an effort i'd rather not make right now

M

moody 💭•2/14/24, 9:18 PM

since i'll have to create an https cert and load balancer and some auto scaling stuff all the infra to deploy that in a pipeline, etc.

U

Unsmart•2/14/24, 9:21 PM

Not really anything cloudflare can do about AWS having high bandwidth pricing take that one up with AWS. But cloudflare doesnt throttle bandwidth usage... how much data are getting back and how long does it take to download? And whats the comparison to downloading direct from RDS?

M

moody 💭•2/14/24, 9:21 PM

found the bandwidth chart

M

moody 💭•2/14/24, 9:23 PM

10MB/s peak, i'm not sure where these clientwrite spikes could be coming from but i do cache in-memory on the worker so maybe workers are getting recycled

M

moody 💭•2/14/24, 9:24 PM

these queries aren't crazy, it's like 3 numeric fields with maybe 10k rows, probably O(MBs), i would use wire compression if that was a thing because one of the columns has a lot of repeated values

UUnsmart Not really anything cloudflare can do about AWS having high bandwidth pricing ta...

M

moody 💭•2/14/24, 9:24 PM

kinda hard to measure this metric downloading locally since it's measured on rds

M

moody 💭•2/14/24, 9:25 PM

maybe the hyperdrive instances are getting overloaded during those periods?

Mmoody 💭kinda hard to measure this metric downloading locally since it's measured on rds

I

Isaac McFadyen•2/14/24, 9:25 PM

I think he just means, if you run the download time on a Worker that's deployed (Hyperdrive) versus on your local machine.

M

moody 💭•2/14/24, 9:25 PM

IIsaac McFadyen I think he just means, if you run the download time on a Worker that's deployed ...

M

moody 💭•2/14/24, 9:26 PM

well i don't think it's going to be different because these spikes are transient and have happened a few times in the past few weeks

M

moody 💭•2/14/24, 9:26 PM

and i can't measure timing on a worker anyway

M

moody 💭•2/14/24, 9:26 PM

but i could try next time it's spiking

U

Unsmart•2/14/24, 9:27 PM

If there wouldnt be a difference in speed from running on the worker vs not then its not a cloudflare issue. Thats all the test is supposed to see

M

moody 💭•2/14/24, 9:27 PM

i mean there's a lot of confounding factors in that test

M

moody 💭•2/14/24, 9:28 PM

e.g. my m1 pro is a lot faster than a worker presumably

U

Unsmart•2/14/24, 9:28 PM

The speed of the machine running doesnt matter as it would be network latency

M

moody 💭•2/14/24, 9:28 PM

ok, yeah i do need to write an endpoint just to test it though, because i do a lot of compute with the data

M

moody 💭•2/14/24, 9:29 PM

is there anything you can check on your side to see if there's an issue

M

moody 💭•2/14/24, 9:29 PM

cpu load on the hyperdrive compute

U

Unsmart•2/14/24, 9:29 PM

I dont work for cloudflare so no I cant

U

Unsmart•2/14/24, 9:29 PM

But I doubt they have an issue scaling it tbh

M

moody 💭•2/14/24, 9:29 PM

it's abeta service, it's very possible

M

moody 💭•2/14/24, 9:30 PM

i'm not doinga nything that weird and my rds bandwidth usage is orders of magnitude less than the cap

M

moody 💭•2/14/24, 9:30 PM

plus it's transient and not corresponding to any spikes in my other metrics, like cpu usage

Mmoody 💭e.g. my m1 pro is a lot faster than a worker presumably

I

Isaac McFadyen•2/14/24, 9:56 PM

A worker is run on CF servers which have 64 CPU cores and 256GB of RAM. A Worker is almost certainly faster than your M1 Pro, with the difference being that CF will cap the usage at a certain point

I

Isaac McFadyen•2/14/24, 9:57 PM

But Hyperdrive is an internal service, so I assume any throttling that might happen is relaxed because of that.