Since D1 supports up to 50,000 databases, is there an API to dynamically create a db and binding for

Since D1 supports up to 50,000 databases, is there an API to dynamically create a db and binding for a worker? There is a rest API for this but I'm wondering if there is an accepted way of doing this from a worker.

kian•1/16/24, 7:57 PM

Not currently

Nnob Since D1 supports up to 50,000 databases, is there an API to dynamically create ...

Vy•1/16/24, 7:58 PM

not yet, it's something we're designing at the moment

nobOP•1/16/24, 8:20 PM

@kian @Vy Thanks for a quick reply! Is it ok if I use the rest api from a worker to do this for the time being?

jeremy•1/16/24, 8:29 PM

Hey, everyone. I am working through the Next.js tutorial. I am trying to set it up on Cloudflare Pages. I am at a step where I need to hook up a database. (I'll turn this into a thread and then ask my question.)

Nnob @kian @Vy Thanks for a quick reply! Is it ok if I use the rest api from a worker...

Max (@rozenmd)•1/16/24, 8:33 PM

Yep, that's how folks have been doing it

Max (@rozenmd)•1/16/24, 8:33 PM

rather, using the api to update their bindings dynamically

nobOP•1/16/24, 8:37 PM

@Max (@rozenmd) Gotcha, thanks!

Ln_north•1/17/24, 12:27 AM

I'm using Next.js created with create-next-app. When I start Next with commands like

yarn dev

yarn dev

, I want to use a local D1 created with npx wrangler d1 somecommand --localnpx wrangler d1 somecommand --local in api/foo/bar/route.tsapi/foo/bar/route.ts. However, even after referring to the following page, I can't seem to bind it properly in my local environment.
https://developers.cloudflare.com/pages/framework-guides/deploy-a-nextjs-site/#use-bindings-in-your-nextjs-application
Is this an issue unique to my environment? Is there any way to fix this?

Deploy a Next.js site · Cloudflare Pages docs

Next.js is an open-source React framework for creating websites and applications. In this guide, you will create a new Next.js application and deploy …

Vvillainy has anyone tried inserting a very large table into D1? i just get stuck in this ...

Arma•1/17/24, 10:22 AM

Yeah my database is about 150 MB in one big table, I inserted everything with wrangler d1 executewrangler d1 execute like you did

Arma•1/17/24, 10:23 AM

I made a SQL file with CREATE TABLE ... ; INSERT ... ; INSERT ... ; etc

Arma•1/17/24, 10:23 AM

a few million statements and a few hundred gigabytes

Arma•1/17/24, 10:24 AM

It worked fine, wrangler batched the statements and ran them without error

Arma•1/17/24, 10:24 AM

it took a while, as you would expect

Arma•1/17/24, 10:25 AM

The "Mapping SQL input into an array of statements" step took less than a minute

Arma•1/17/24, 10:26 AM

but it used a LOT of memory

Arma•1/17/24, 10:26 AM

was not an issue on my machine (16GB) but maybe something that can be improved by the wrangler devs

Arma•1/17/24, 10:27 AM

Make sure you separate your statements with a newline, and terminate each statement with a

CCyb3r-Jak3 What does the meta of the D1 result give for timing?

Arma•1/17/24, 10:49 AM

I've got the "cold start" again. Here are the details:

The whole request took 3960ms from the client end. I made one read on KV and executed two queries on D1. Only the first D1 query was slow, it took 3240ms. The KV read took 32ms and the second D1 query took 33ms. All of them measured in the worker with Date.now() before and after the

await

await

.

meta of the slow query:

{"served_by":"v3-prod","duration":0.6271,"changes":0,"last_row_id":0,"changed_db":false,"size_after":144568320,"rows_read":1,"rows_written":0}

{"served_by":"v3-prod","duration":0.6271,"changes":0,"last_row_id":0,"changed_db":false,"size_after":144568320,"rows_read":1,"rows_written":0}

Arma•1/17/24, 10:59 AM

I guess if you're not using the database for a while, it gets unloaded, and the whole thing needs to be read from disk and loaded on the next query?

AArma I guess if you're not using the database for a while, it gets unloaded, and the ...

Max (@rozenmd)•1/17/24, 11:53 AM

pretty much, yeah

Arma•1/17/24, 2:56 PM

Maybe I should split up my databases then

Arma•1/17/24, 2:57 PM

and put the big fat table on its own database, this way I can query other tables without waiting as much

Arma•1/17/24, 3:00 PM

And to avoid making Clouflare load n gigabytes of data when I just want to query a table of 2 KB

Bonadio•1/17/24, 4:55 PM

Hi, I am trying to simulate some stored procedure in D1 using batch. I was able to create a "TRIGGER" that RAISE an error. Is it ok to use the "TRIGGER" on D1?

kian•1/17/24, 6:25 PM

So you don't get an error at all, or you do but it's vague?

pinky•1/17/24, 7:23 PM

Hi, is it safe to create cached orm that is using bindings from previous fetch, something like this?

const db = getDrizzle(d1: D1Database) => {
    if (!db.cached) {
        db.cached = drizzle(d1);
    }
    return db;
}

export default {
    async fetch(request: Request, env: Env) {
        const db = getDrizzle(env.DB);
        ...
    }
}

const db = getDrizzle(d1: D1Database) => {
    if (!db.cached) {
        db.cached = drizzle(d1);
    }
    return db;
}

export default {
    async fetch(request: Request, env: Env) {
        const db = getDrizzle(env.DB);
        ...
    }
}

Ppinky Hi, is it safe to create cached orm that is using bindings from previous fetch, ...

Matt Silverlock•1/17/24, 8:11 PM

There's no long-running connection to cache, and so I'd consider this an anti-pattern (I assume you're trying to reduce latency)

MMatt Silverlock There's no long-running connection to cache, and so I'd consider this an anti-pa...

pinky•1/17/24, 9:57 PM

Should every worker run be treated as an isolated environment so nothing should be cached into module scope for reuse?

Ppinky Should every worker run be treated as an isolated environment so nothing should ...

Matt Silverlock•1/17/24, 10:06 PM

You should assume it may not be retained as invocations may not share the same underlying context.

But that’s orthogonal to my advice: there should be no measurable performance benefit to trying to “memoize” a binding like that - D1 doesn’t have the heavy connection setup that a traditional database does. Same goes for other bindings.

MMatt Silverlock You should assume it may not be retained as invocations may not share the same u...

pinky•1/17/24, 10:17 PM

Hmm ok, but let's say there is a trpc implementation and it has a bunch of routers that are loaded dynamically to avoid loading a big chunk of javascript for no reason. The most procedures do some database operation. However, to follow the advice that the same running worker (I assume it's alive for some time it won't die after the first fetch) should not consider any caching, a route to a procedure would need to be loaded dynamically every time with the latest env.DB passed into drizzle or used directly for a db operation. Is that a suggested approach?

I'm trying to understand if I should think about a worker as a lightweight express server that can be restarted any time but for the period when it's not restarted it can leverage some scoped caching or if this approach can cause some trouble and everything should be created fresh on every request.

Ppinky Hmm ok, but let's say there is a trpc implementation and it has a bunch of route...

Matt Silverlock•1/17/24, 10:20 PM

Memoizing heavy compute operations or serialization makes sense.

Memoizing bindings (which is what you asked) does not.

MMatt Silverlock Memoizing heavy _compute_ operations or serialization makes sense. Memoizing bi...

pinky•1/17/24, 10:23 PM

Ok got it, thanks for the clarification

Ppinky Hmm ok, but let's say there is a trpc implementation and it has a bunch of route...

kian•1/17/24, 10:25 PM

Think of it like a spot instance that you're unlikely to hit again, since that would require you hitting the same Cloudflare server.

Even if you can use the global scope, it'd only really benefit the user who already hit that Worker up until their connection ends.

Canopy•1/18/24, 7:34 AM

Would absolutely love to be able to use a D1 with my nextjs project but working out how on earth I'm meant to do local development with that is hard as nails. If anyone has an example project where they've solved this problem lemme know, because it's consuming so much of my time trying to work this out.

Yiwan•1/18/24, 9:36 AM

Why do you need to do it locally? Set up dev environment on the actual cloud, wrangler can stream logs

JJoël Hello @Matt ! (from Prisma 👋 ) I found out that the types were recently changed...

Matt Silverlock•1/18/24, 11:26 AM

rows_written are rows affected, because index writes/modifications are… affected.

JJoël Hello @Matt ! (from Prisma 👋 ) I found out that the types were recently changed...

Matt Silverlock•1/18/24, 11:27 AM

The rest we haven’t yet documented as it may be subject to change, which is why that PR is still in draft

Rubi•1/18/24, 2:55 PM

GA when

Vy•1/18/24, 3:16 PM

targetting GA early April

MMatt Silverlock This is an interim solution, for what it's worth. As we've called out before, we...

Victor•1/18/24, 6:27 PM

With the GA announcement ^, is there an update on the "group bindings"? Interested for multi tenancy bindings for D1

VVictor With the GA announcement ^, is there an update on the "group bindings"? Interest...

Matt Silverlock•1/18/24, 6:39 PM

No update yet (and we announced the GA timeline back in Sept 2023, for what it's worth!)

Max (@rozenmd)•1/18/24, 7:07 PM

you'll find early april is at the end of Q1

Unsmart•1/18/24, 7:07 PM

big if true

Gagan Suie•1/18/24, 10:50 PM

Reporting a VERY weird bug. Anytime we add the word "What" then the remaining text turns into an object:
The topic column has TEXT: "What is your favorite clip?"

const result = await env.DB.prepare(
            SELECT * 
            FROM weekly 
            WHERE weekNumber = ?1
        )
            .bind(weekNumber)
            .first()

const result = await env.DB.prepare(
            SELECT * 
            FROM weekly 
            WHERE weekNumber = ?1
        )
            .bind(weekNumber)
            .first()

result {"_id":1,"weekNumber":3,"topic":"What [object Object]

Gagan Suie•1/18/24, 10:54 PM

row was added via the "Add data" button in cloudflare D1 dashboard.

Since D1 supports up to 50,000 databases, is there an API to dynamically create a db and binding for

Similar Threads

Similar Threads

Similar Threads