Who will win, docs aside or https://github.com/cloudflare/workerd/pull/1266/

K

kianOP•10/4/23, 6:24 PM

Kkian It'd be nice if Wrangler made a warning/error when you use local development w/ ...

U

Unsmart•10/4/23, 6:45 PM

Yeah ive noticed a lot with newer products people will attempt to use the bindings but get undefined because they have not updated wrangler... I dont think it would be too hard to look at keys not parsed and be like hey this isnt a valid key you might need to update or something... Or the remote/local thing also

UUnsmart Yeah ive noticed a lot with newer products people will attempt to use the bindin...

M

Matt Silverlock•10/4/23, 6:51 PM

cc @penalosa

UUnsmart Yeah ive noticed a lot with newer products people will attempt to use the bindin...

K

kianOP•10/4/23, 6:57 PM

I made https://github.com/cloudflare/workers-sdk/issues/2586 ages ago

GitHub

🚀 Feature Request: Expand validateAdditionalProperties to more conf...

Describe the solution The current usage of validateAdditionalProperties is only used for a subset of configuration fields, such as build or the top-level entries. This means that you can use the fo...

S

Sam•10/5/23, 1:14 PM

Quick questions - in the demo, a new connection is opened for each request (if I'm right). Won't this overload the PostgreSQL server when there're for example thousands of requests each second?

Also, I've created a Hyperdrive Id using wrangler. Should the dashboard in cloudflare change, or will it stay like this?

And last - in local development, I can't access the database (works in prod after wrangler deploy). I'm receiving the following error:

Cannot read properties of undefined (reading 'connectionString')

Cannot read properties of undefined (reading 'connectionString')

Cannot read properties of undefined (reading 'connectionString')

Cannot read properties of undefined (reading 'connectionString')

Do I need to add any parameters to the dev start up?

Thank you!

MMatt Silverlock Not yet, but very soon - we want to support local dev.

S

Sam•10/5/23, 1:23 PM

Ah nvm about the local dev.

SSam Quick questions - in the demo, a new connection is opened for each request (if I...

I

Isaac McFadyen•10/5/23, 1:42 PM

Hyperdrive is a connection pooler and avoids this exact thing. A few connections are opened from Hyperdrive, and then every request opens a new connection to Hyperdrive (but not the DB) so that your DB doesn't get overloaded.

I

Isaac McFadyen•10/5/23, 1:43 PM

Because connections are held open from Hyperdrive, the connections to Hyperdrive are very very quick and won't overload things.

IIsaac McFadyen Hyperdrive is a connection pooler and avoids this exact thing. A few connections...

S

Sam•10/5/23, 1:43 PM

Ahh ok, that's what I was thinking but I was not sure. Thanks for clarifying

S

Sam•10/5/23, 1:58 PM

For local dev, I'm trying this (pasting a string, if HYPERDRIVE is not available in the env), but I'm getting SSL issues. Is this happening for more people?

Or is my (managed by digital ocean) database cluster not set up correctly?

S

Sam•10/5/23, 2:08 PM

Also wrangler defaults to 3.0.0 after installing, should I update to 3.10.1?

Llithdew Ran into an issue just now trying to create a Hyperdrive over a PostgreSQL datab...

M

Matt Silverlock•10/5/23, 3:31 PM

Can you DM me your database URL? There’s not enough info in the issue to debug unfortunately.

SSam Also wrangler defaults to 3.0.0 after installing, should I update to 3.10.1?

M

Matt Silverlock•10/5/23, 3:31 PM

Yes

JJordan (UNCVRD)hey! I was wondering if this statement is accurate about being able to use Postg...

M

Matt Silverlock•10/5/23, 8:54 PM

You need to use the latest version of

Postgres.js

Postgres.js

Postgres.js

Postgres.js

we are waiting for them to publish
```
3.4.0
```
3.4.0
```
3.4.0
```
3.4.0

JJordan (UNCVRD)also...i was wondering if we'd be able to control the max number of connections ...

M

Matt Silverlock•10/5/23, 8:54 PM

It's not currently configurable: 20 is very, very low though and would impact performance (fewer conns in the pool == more waiting)

M

mattfreeman•10/6/23, 6:49 AM

Is the connection string returned by env.HYPERDRIVE.connectionString usable outside of workers, is it accessible on from workstation when connecting over Warp Teams into the same account ?

K

kianOP•10/6/23, 8:54 AM

Nope

N

Nikil•10/7/23, 3:07 PM

So am I understanding Hyperdrive correctly? It's basically the same idea as Cloudflare DNS w/ CDN-style caching but for DBs? That sounds cool...

Kkian Nope

M

mattfreeman•10/9/23, 5:53 AM

Any plans for this? It would be awesome use case to connect into a zero trust with Cloudflare Warps for team, point DataGrip at the hyperdrive endpoint and presume it gets hyperdriven from a near edge, and enjoy the acceleration of less chat between me and live DB 300ms away.

K

kianOP•10/9/23, 7:04 PM

I don't think there's a timeframe on it hyperdrive

Kkian It'd be nice if Wrangler made a warning/error when you use local development w/ ...

K

kianOP•10/9/23, 7:06 PM

So it turns out the docs callout I was thinking of is actually in Workers AI, and there isn't one for Hyperdrive. Oops

Kkian Who will win, docs aside or https://github.com/cloudflare/workerd/pull/1266/

K

kianOP•10/9/23, 7:13 PM

My bets have shifted over to the PR now meowlul

K

kianOP•10/9/23, 7:14 PM

I guess docs asides are pretty easy to miss - it'd be cool for Wrangler to do the warnings (like it does for Queues in remote mode currently).

K

kianOP•10/9/23, 7:14 PM

I guess that's hard for remote mode which can support them independently of a new Wrangler release, but local should be fine.

M

Max (@rozenmd)•10/9/23, 7:16 PM

Converted the hyperdrive example to work with Hono if anyone's curious:

import { Client } from 'pg';
import { Hono, MiddlewareHandler } from 'hono';

type Bindings = {
    HYPERDRIVE: Hyperdrive;
};

const app = new Hono<{
    Bindings: Bindings;
    Variables: {
        DB: Client;
    };
}>();

const dbMiddleware: MiddlewareHandler = async (c, next) => {
    c.set('DB', new Client({ connectionString: c.env.HYPERDRIVE.connectionString }));
    await next();
};

app.get('/', dbMiddleware, async (c) => {
    try {
        // Connect to our database
        await c.var.DB.connect();

        // Test query
        let result = await c.var.DB.query({ text: 'SELECT * FROM pg_tables' });

        // Return result rows as JSON
        return c.json(result);
    } catch (e) {
        console.log(e);
        return Response.json({ error: JSON.stringify(e) }, { status: 500 });
    }
});

export default app;

import { Client } from 'pg';
import { Hono, MiddlewareHandler } from 'hono';

type Bindings = {
    HYPERDRIVE: Hyperdrive;
};

const app = new Hono<{
    Bindings: Bindings;
    Variables: {
        DB: Client;
    };
}>();

const dbMiddleware: MiddlewareHandler = async (c, next) => {
    c.set('DB', new Client({ connectionString: c.env.HYPERDRIVE.connectionString }));
    await next();
};

app.get('/', dbMiddleware, async (c) => {
    try {
        // Connect to our database
        await c.var.DB.connect();

        // Test query
        let result = await c.var.DB.query({ text: 'SELECT * FROM pg_tables' });

        // Return result rows as JSON
        return c.json(result);
    } catch (e) {
        console.log(e);
        return Response.json({ error: JSON.stringify(e) }, { status: 500 });
    }
});

export default app;

import { Client } from 'pg';
import { Hono, MiddlewareHandler } from 'hono';

type Bindings = {
    HYPERDRIVE: Hyperdrive;
};

const app = new Hono<{
    Bindings: Bindings;
    Variables: {
        DB: Client;
    };
}>();

const dbMiddleware: MiddlewareHandler = async (c, next) => {
    c.set('DB', new Client({ connectionString: c.env.HYPERDRIVE.connectionString }));
    await next();
};

app.get('/', dbMiddleware, async (c) => {
    try {
        // Connect to our database
        await c.var.DB.connect();

        // Test query
        let result = await c.var.DB.query({ text: 'SELECT * FROM pg_tables' });

        // Return result rows as JSON
        return c.json(result);
    } catch (e) {
        console.log(e);
        return Response.json({ error: JSON.stringify(e) }, { status: 500 });
    }
});

export default app;

import { Client } from 'pg';
import { Hono, MiddlewareHandler } from 'hono';

type Bindings = {
    HYPERDRIVE: Hyperdrive;
};

const app = new Hono<{
    Bindings: Bindings;
    Variables: {
        DB: Client;
    };
}>();

const dbMiddleware: MiddlewareHandler = async (c, next) => {
    c.set('DB', new Client({ connectionString: c.env.HYPERDRIVE.connectionString }));
    await next();
};

app.get('/', dbMiddleware, async (c) => {
    try {
        // Connect to our database
        await c.var.DB.connect();

        // Test query
        let result = await c.var.DB.query({ text: 'SELECT * FROM pg_tables' });

        // Return result rows as JSON
        return c.json(result);
    } catch (e) {
        console.log(e);
        return Response.json({ error: JSON.stringify(e) }, { status: 500 });
    }
});

export default app;

Note that I've renamed "client" to "DB", otherwise its the same

M

meai1•10/10/23, 2:52 PM

I keep getting " not_entitled [code: 10000]" even though I can connect to the db via datagrip just fine. Do I need to create a token permission for hyperdrive or do I need a paid workers plan? It logs me into cloudflare via oauth so I dont know what I'm not entitled for

Mmeai1 I keep getting " not_entitled [code: 10000]" even though I can connect to the db...

C

Cyb3r-Jak3•10/10/23, 2:53 PM

Workers paid plan is required for hyperdrive

M

meai1•10/10/23, 2:53 PM

oh okay, thank you

A

alex (he/him)•10/11/23, 6:39 PM

if hyperdrive keeps connections in a warm pool, does that mean that serverless dbs e.g. neon wouldn't suspend to not pay for compute?

U

Unsmart•10/11/23, 6:41 PM

If they charge by connection hours then I would assume that would be the case yes

K

kianOP•10/11/23, 6:41 PM

There's an initial connection timeout of 15 seconds and an idle connection timeout of 10 minutes.

K

kianOP•10/11/23, 6:42 PM

Neon's default auto-suspend delay is 5 minutes

K

kianOP•10/11/23, 6:42 PM

I imagine Hyperdrive might let you configure the timeout periods down the line

A

alex (he/him)•10/11/23, 6:42 PM

Ok that means neon will stay online 10 minutes instead of the 5 minutes?

A

alex (he/him)•10/11/23, 6:42 PM

Or are idle connection not forcing the db to stay online?

C

Colin•10/11/23, 10:52 PM

hey y'all - wanted to ask if anyone had opinions regarding Neon w/ Neon's connection pooling vs Neon w/ Hyperdrive (aside from the "one is GA and one isn't")

C

Colin•10/11/23, 10:53 PM

It seems like the main Hyperdrive-specific features are query caching + analytics (not yet but soon)

C

Colin•10/11/23, 10:54 PM

does anyone have experience with it / have any thoughts i might be missing?

CColin hey y'all - wanted to ask if anyone had opinions regarding Neon w/ Neon's connec...

I

Isaac McFadyen•10/11/23, 10:55 PM

Hyperdrive will be more efficient. The reason for that is:

Hyperdrive:
Worker --(very low latency)--> Hyperdrive --(persistent connection)--> Neon, so connection establishment from Worker to Hyperdrive is fast
Neon:
Worker --(higher latency)--> Neon Pooler, so while Neon might handle connection pooling the connections will still take a longer amount of time to start

Workers are rapidly spun up/down based on requests, so initial connection time is very important.

I

Isaac McFadyen•10/11/23, 10:56 PM

However, your Neon DB might not sleep very well with inactivity on Hyperdrive. See the conversation above.

IIsaac McFadyen Hyperdrive will be more efficient. The reason for that is: **Hyperdrive:** Work...

C

Colin•10/11/23, 10:58 PM

I definitely understand this for the

pg

pg

driver, but if we're using

@neondatabase/serverless

@neondatabase/serverless

@neondatabase/serverless

@neondatabase/serverless or similar I'd expect their optimizations to reduce round trips would make it closer at least

C

Colin•10/11/23, 10:59 PM

i'm not sure how close that'd be to hyperdrive, if it's comparable at all

CColin I definitely understand this for the `pg` driver, but if we're using `@neondatab...

I

Isaac McFadyen•10/11/23, 10:59 PM

Ah - missed that you were using the Serverless driver. That should be fast to start since it's Websockets to the closest Neon node which then uses a long-lived TCP connection.

I

Isaac McFadyen•10/11/23, 11:01 PM

Check out https://neon.tech/demos/ping-thing if you want some preliminary numbers.

Ping Thing

A @neondatabase/serverless demo

Who will win, docs aside or https://github.com/cloudflare/workerd/pull/1266/

Similar Threads