is it possible to bind multiple D1 dbs to a single worker? And is it possible to bind a single D1 db

Mmoonchav is it possible to bind multiple D1 dbs to a single worker? And is it possible to...

I

Isaac McFadyen•8/18/24, 7:26 PM

Yes and yes

I

Isaac McFadyen•8/18/24, 7:26 PM

Just bind it like normal in your wrangler.toml but specify multiple of them

M

moonchavOP•8/18/24, 7:27 PM

oh sweet, thank you! I should have just rolled the dice and tried it before asking

I

Isaac McFadyen•8/18/24, 7:27 PM

No problem :lul:

V

veeque•8/19/24, 1:47 PM

in my

wrangler.toml

wrangler.toml

file I have this:

[[d1_databases]]
binding = "db"
database_name = "prod-main"
database_id = "[uuid1]"
preview_database_id = "[uuid2]"

[[d1_databases]]
binding = "db"
database_name = "prod-main"
database_id = "[uuid1]"
preview_database_id = "[uuid2]"

[[d1_databases]]
binding = "db"
database_name = "prod-main"
database_id = "[uuid1]"
preview_database_id = "[uuid2]"

[[d1_databases]]
binding = "db"
database_name = "prod-main"
database_id = "[uuid1]"
preview_database_id = "[uuid2]"

(it's a sveltekit app) however the preview deployments still use the production database... I thought that setting the

preview_database_id

preview_database_id

to the uuid of the preview database would suffice, but apparently it doesn't. In the D1 documentation I could only find https://developers.cloudflare.com/d1/configuration/environments/ but the config used looks vastly different than mine. what do I have to do?

Vveeque in my `wrangler.toml` file I have this: ``` [[d1_databases]] binding = "db" dat...

H

Hard@Work•8/19/24, 1:51 PM

IIRC preview IDs aren't used for anything on pages. You need to add a

database_id

database_id

to the preview env

V

veeque•8/19/24, 1:51 PM

and that env is set to be the staging one by preview deployments without me having anything to do, right?

V

veeque•8/19/24, 1:58 PM

ah ok, i see now, ty

V

veeque•8/19/24, 1:59 PM

can a d1 db be renamed?

FFini Thanks, I appreciate it. It's a production system that's now been done for 2 day...

W

Walshy•8/19/24, 6:20 PM

You should be good now, apologies for the issue!

カ

カイル•8/20/24, 5:00 AM

Is it possible to use the D1 rest API but lockdown the API key to only that D1 database

WWalshy You should be good now, apologies for the issue!

D

Dean•8/20/24, 10:06 AM

What was the cause? Deploying a ton of databases and this has me concerned

カカイル Is it possible to use the D1 rest API but lockdown the API key to only that D1 d...

D

Dean•8/20/24, 10:20 AM

Better off using a worker with an API Key.

V

veeque•8/20/24, 11:25 AM

this is my

wrangler.toml

wrangler.toml

:

name = "[the name]"
compatibility_date = "2024-07-29"
pages_build_output_dir = ".svelte-kit/cloudflare"

[[env.production.d1_databases]]
binding = "db"
database_name = "prod-main"
database_id = "524d4c4f-410b-46f9-9a87-2dc4f6110b1c"

[[env.preview.d1_databases]]
binding = "db"
database_name = "dev-main"
database_id = "025df237-d405-4f57-859f-ad3a055fd74b"

name = "[the name]"
compatibility_date = "2024-07-29"
pages_build_output_dir = ".svelte-kit/cloudflare"

[[env.production.d1_databases]]
binding = "db"
database_name = "prod-main"
database_id = "524d4c4f-410b-46f9-9a87-2dc4f6110b1c"

[[env.preview.d1_databases]]
binding = "db"
database_name = "dev-main"
database_id = "025df237-d405-4f57-859f-ad3a055fd74b"

name = "[the name]"
compatibility_date = "2024-07-29"
pages_build_output_dir = ".svelte-kit/cloudflare"

[[env.production.d1_databases]]
binding = "db"
database_name = "prod-main"
database_id = "524d4c4f-410b-46f9-9a87-2dc4f6110b1c"

[[env.preview.d1_databases]]
binding = "db"
database_name = "dev-main"
database_id = "025df237-d405-4f57-859f-ad3a055fd74b"

name = "[the name]"
compatibility_date = "2024-07-29"
pages_build_output_dir = ".svelte-kit/cloudflare"

[[env.production.d1_databases]]
binding = "db"
database_name = "prod-main"
database_id = "524d4c4f-410b-46f9-9a87-2dc4f6110b1c"

[[env.preview.d1_databases]]
binding = "db"
database_name = "dev-main"
database_id = "025df237-d405-4f57-859f-ad3a055fd74b"

and when I try to run

wrangler d1 migrations apply prod-main --local

wrangler d1 migrations apply prod-main --local

(or even without

--local

--local

) it says:

 ⛅️ wrangler 3.72.0
-------------------

✘ [ERROR] Couldn't find a D1 DB with the name or binding 'prod-main' in wrangler.toml.

 ⛅️ wrangler 3.72.0
-------------------

✘ [ERROR] Couldn't find a D1 DB with the name or binding 'prod-main' in wrangler.toml.

 ⛅️ wrangler 3.72.0
-------------------

✘ [ERROR] Couldn't find a D1 DB with the name or binding 'prod-main' in wrangler.toml.

 ⛅️ wrangler 3.72.0
-------------------

✘ [ERROR] Couldn't find a D1 DB with the name or binding 'prod-main' in wrangler.toml.

is it because of the way I've set up the environment variables?

V

veeque•8/20/24, 11:26 AM

it works just fine when running on cloudflare itself

Vveeque this is my `wrangler.toml`: ```toml name = "[the name]" compatibility_date = "2...

H

Hard@Work•8/20/24, 11:26 AM

You probably want

wrangler d1 migrations apply prod-main -e production --local

wrangler d1 migrations apply prod-main -e production --local

V

veeque•8/20/24, 11:27 AM

yup, that was it, thanks!

V

veeque•8/20/24, 11:49 AM

Well now it seems like the fact that I have a

preview

preview

and a

production

production

environment variable has made it not work locally inside the svelte kit app this is my

app.d.ts

app.d.ts

interface Platform {
  env: {
    db: D1Database;
  };
}

interface Platform {
  env: {
    db: D1Database;
  };
}

interface Platform {
  env: {
    db: D1Database;
  };
}

interface Platform {
  env: {
    db: D1Database;
  };
}

and I use it like this:

const db = await drizzle(platform?.env.db, { schema });

const db = await drizzle(platform?.env.db, { schema });

const db = await drizzle(platform?.env.db, { schema });

const db = await drizzle(platform?.env.db, { schema });

I also tried adding a nested

production

production

variable in both

Platform

Platform

and where I use it:

env.production.db

env.production.db

but no dice

V

veeque•8/20/24, 11:50 AM

TypeError: Cannot read properties of undefined (reading 'prepare')

TypeError: Cannot read properties of undefined (reading 'prepare')

Vveeque Well now it seems like the fact that I have a `preview` and a `production` envir...

H

Hard@Work•8/20/24, 11:52 AM

It would just be

interface Platform {
  env: {
    db: D1Database;
  };
}

interface Platform {
  env: {
    db: D1Database;
  };
}

interface Platform {
  env: {
    db: D1Database;
  };
}

interface Platform {
  env: {
    db: D1Database;
  };
}

, not with environments

H

Hard@Work•8/20/24, 11:52 AM

Also note that

Unlike Workers Environments, production and preview are the only two options available via [env.<ENVIRONMENT>] on Pages

HHard@Work It would just be ```ts interface Platform { env: { db: D1Database; }; } ...

V

veeque•8/20/24, 11:53 AM

still doesn't work, same error

H

Hard@Work•8/20/24, 11:53 AM

Uh

V

veeque•8/20/24, 11:54 AM

export const GET: RequestHandler = async ({ params, platform }) => {
    if (platform === undefined) return new Response('err');
    if (params.user === undefined) return new Response('err');

    const db = await drizzle(platform?.env.db, { schema });

    const result = await db.insert(schema.users).values({ name: params.user });

    return new Response(JSON.stringify(result));
};

export const GET: RequestHandler = async ({ params, platform }) => {
    if (platform === undefined) return new Response('err');
    if (params.user === undefined) return new Response('err');

    const db = await drizzle(platform?.env.db, { schema });

    const result = await db.insert(schema.users).values({ name: params.user });

    return new Response(JSON.stringify(result));
};

export const GET: RequestHandler = async ({ params, platform }) => {
    if (platform === undefined) return new Response('err');
    if (params.user === undefined) return new Response('err');

    const db = await drizzle(platform?.env.db, { schema });

    const result = await db.insert(schema.users).values({ name: params.user });

    return new Response(JSON.stringify(result));
};

export const GET: RequestHandler = async ({ params, platform }) => {
    if (platform === undefined) return new Response('err');
    if (params.user === undefined) return new Response('err');

    const db = await drizzle(platform?.env.db, { schema });

    const result = await db.insert(schema.users).values({ name: params.user });

    return new Response(JSON.stringify(result));
};

this is the quick-and-botchy code i made for adding something to a table

V

veeque•8/20/24, 11:55 AM

in the

/api/create/[user]

/api/create/[user]

route

V

veeque•8/20/24, 11:57 AM

const result = await db.select().from(schema.users);

const result = await db.select().from(schema.users);

also doesn't query, same error

V

veeque•8/20/24, 11:57 AM

it seems to me like the sveltekit app is trying to load a db that wrangler provides in a different way than it expects to(?)

V

veeque•8/20/24, 12:00 PM

putting a console.log to see what platform contains it shows me

env: {},

env: {},

... so it's not loading anything

V

veeque•8/20/24, 12:02 PM

ok found the issue: in the

svelte.config.js

svelte.config.js

file i had to set

platformProxy.environment

platformProxy.environment

to

'production'

'production'

inside the adapter's config

Vveeque ok found the issue: in the `svelte.config.js` file i had to set `platformProxy.e...

H

Hello, I’m Allie!•8/20/24, 12:12 PM

Good you figured it out! I’d kind of stumped myself on that one

L

Leif•8/21/24, 8:16 AM

Hey guys, I want to use the 'database per tenant' architecture, but I'm a little blocked with how to implement it with D1. (I'm using Pages with the SvelteKit adapter).

My current understanding of D1 is that upon a worker creation, it creates a read only replica of the Sqlite file inside the instance, and upon writes it calls back to the main D1 instance with eventual consistency inside workers replicas.

This architecture is fine because my application will never have multiple users writing to the same row, so no concurrency issues for my app.

I have users creating 'communities', and I would like each community to be it's own database (mainly for data ownership and D1 storage constraint reasons).

But here are my current issues that I would really appreciate help with understanding (I haven't found much documentation on this):

1) How to dynamically create databases with the same schema (a community schema, but I can create hundreds of these databases).
2) How to handle migrations across all the databases of the same schema.
3) Most importantly, how to efficiently bind these potentially hundreds of databases with my workers.

I might be heading in the wrong direction with this architecture. I'd love some correction if necessary. Thanks in advance for your help!

Original message was deleted

L

Leif•8/21/24, 8:34 AM

Oh wow! I guess my entire understanding of D1 is just wrong then! Is there any documentation for dynamic bindings? Thanks for the response.

L

Leif•8/21/24, 8:35 AM

Ohhh thank you. So out of curiosity, how are people actually using D1 in production? The limitations seem pretty unfavorable for most production applications.

E

EvanPiro•8/21/24, 6:52 PM

Is there support for functions? I know the feature isn't available in sqlite but I have a codebase that is full of repeated queries in nodejs and would love to call them directly from the db.

J

Jose•8/21/24, 9:58 PM

Hello, i am getting this error in my D1, Internal error while starting up D1 storage caused object to be reset. [code: 7500], and returns the info failed to reach database, any ideias?

LLeif Ohhh thank you. So out of curiosity, how are people actually using D1 in product...

D

Dean•8/22/24, 1:58 AM

Right now using workers with svelte and binding via the API. Will eventually hit problems as I can only bind 5000 ish databases. Dynamic bindings should solve the problem before we get there.

JJose Hello, i am getting this error in my D1, Internal error while starting up D1 sto...

M

Max (@rozenmd)•8/22/24, 5:33 AM

what's your database id?

O

oof2win2•8/22/24, 6:47 AM

is there an intended usage path for local dev when binding two workers to the same db?

A

andreasb•8/22/24, 6:51 AM

Any plans from Cloudflare to do a bitemporal/immutable database offering, akin to XTDB, SQL Server 2022 Ledger, ImmuDB or similar?

S

Softbear•8/22/24, 8:19 AM

Hello I see some discrepancy in my D1 usage.

When I query my stats I see that this query is reading a lot of rows

But when I run it it works as expected (I have an index on the column)

Same for the explain query plan

It's 2B rows/day of reads ...

F

Fernando Barroso•8/22/24, 8:10 PM

Hello all! Has anyone here ever built an event store (for the purpose of event sourcing) on top of D1 by chance?

I

ian•8/23/24, 9:04 PM

why are metrics queries on D1 limited to 1 week?

Iian why are metrics queries on D1 limited to 1 week?

M

Max (@rozenmd)•8/23/24, 9:48 PM

looks like a bug

Aamcclosky Any updates on when there might be updates around cloning databases? or spinning...

M

Max (@rozenmd)•8/23/24, 9:48 PM

it becomes possible after read replication is released

DDean Right now using workers with svelte and binding via the API. Will eventually hit...

L

Leif•8/24/24, 5:57 AM

I'm very curious by what you mean by problems. Are you referring to memory constraints on your worker, D1 instance, or both?

Also another big question I have: When a worker binds with a D1 database, is it performing any edge specific optimizations, or just making network calls against a crntral database. I ask this question because if I'm always going to be making db calls to a centeal DB, whats the purpose of having my functions on the edge?

LLeif I'm very curious by what you mean by problems. Are you referring to memory const...

D

Dean•8/24/24, 5:58 AM

I’m talking about the max 5000 db binding limit on a worker.

L

Leif•8/24/24, 5:59 AM

Okay makes sense

LLeif I'm very curious by what you mean by problems. Are you referring to memory const...

M

Max (@rozenmd)•8/24/24, 1:29 PM

when your worker talks to D1, everything is in the edge

when you use the API/wrangler/dash, it involves a round trip to a core DB. there's work in progress to move the API to the edge

is it possible to bind multiple D1 dbs to a single worker? And is it possible to bind a single D1 db

Similar Threads

Similar Threads

Similar Threads