next to Dynamic Bindings, are there plans for sharding as well perhaps? - Cloudflare Developers

© 2026 Hedgehog Software, LLC

More

Communities Docs About Terms Privacy

Miloš

M

MilošOP•6/24/24, 5:15 PM

Asking because my app requires a lot of queries across users which I believe wouldn't be that easy with per user db setup

Disciple of Caoximeihao

DDisciple of Caoximeihao ```const insertedOptionValues = await c.env.DB.prepare(`INSERT INTO optionValues...

Vinodh

V

Vinodh•6/25/24, 4:55 AM

Use Drizzle ORM -> https://orm.drizzle.team/docs/insert#insert-multiple-rows

Miloš

MMiloš next to Dynamic Bindings, are there plans for sharding as well perhaps?

Hard@Work

H

Hard@Work•6/25/24, 10:53 AM

That one I'm not sure about, might be good to ask for an update in #community-call-questions if there isn't one already

Pavlo

P

Pavlo•6/25/24, 10:56 AM

Hello, what does this error mean and how to solve it? D1_ERROR: D1 exceeded its CPU time limit and was reset

Pavlo

PPavlo Hello, what does this error mean and how to solve it? D1_ERROR: D1 exceeded its ...

SteffTek

S

SteffTek•6/25/24, 11:39 AM

your D1 Query was so big, that your worker shut itself off

SteffTek

S

SteffTek•6/25/24, 11:40 AM

you have limited time to answer a worker request

vrballer

Vvrballer were you able to get this working (or possibly find some working alternative)

Softbear

S

Softbear•6/25/24, 2:49 PM

I'm using TablePlus

dqwqwdldowqqw

D

dqwqwdldowqqw•6/25/24, 4:42 PM

@Hard@Work

dqwqwdldowqqw

D

dqwqwdldowqqw•6/25/24, 4:42 PM

Can you help sorry for ping

dqwqwdldowqqw

D

dqwqwdldowqqw•6/25/24, 4:42 PM

Check #general-help latest.

dqwqwdldowqqw

Ddqwqwdldowqqw Check #general-help latest.

Hard@Work

H

Hard@Work•6/25/24, 4:46 PM

?pings

Flare

F

Flare•6/25/24, 4:46 PM

Please do not ping community members for non-moderation reasons. Doing so will not solve your issue faster and will make people less likely to want to help you.

Hard@Work

H

Hard@Work•6/25/24, 4:46 PM

Also, not sure what this has to do with D1

SteffTek

S

SteffTek•6/25/24, 5:14 PM

i've got this error multiple times while migrating my database. someone knows why this keeps showing up? i always killed the last migration but this time i have follow migrations and it's not that easy

SteffTek

S

SteffTek•6/25/24, 5:31 PM

i removed so many comments after finding a github issue, this is dumb lol

EisenbergEffect

EEisenbergEffect Looking at limits to D1, I see that a single DB is limted to 10GB and that an ac...

Sam

S

Sam•6/25/24, 5:52 PM

The 10gb per database not, but I think the account limit can be increased by contacting support

peshakoo

Ppeshakoo I wonder how many users I can fit into 10GB without sharding

SteffTek

S

SteffTek•6/25/24, 5:59 PM

Softbear

SSoftbear I'm using TablePlus

vrballer

V

vrballer•6/25/24, 10:23 PM

Thanks!

puerh

P

puerh•6/26/24, 12:24 PM

Is there any way to restrict people from just using the UI to add a record to a d1 database directly (or even KV) - but still giving them worker write access? Feels like a really annoying permissions gap if not

I just want a db that we can only add to through known sources rather than anyone with enough permissions in cloudflare

Original message was deleted

puerh

P

puerh•6/26/24, 6:51 PM

I think that's what I'm trying to work out

Ideally the only way to deploy something that touches the DB will be through Github and atleast then it's reviewed.
But atm it feels a little impossible to store anything that's even remotely sensitive as it's a complete wild west. But I don't think Cloudflare offers the sort of permissions model I'm after.

I'm a Cloudflare admin, and I don't really want the permission to add records to the database directly through the UI

Unsmart

U

Unsmart•6/26/24, 6:58 PM

I mean realistically no platform offers that kind of permission model. But for all platforms you basically need to do the permission side of stuff in github actions where the github action can deploy the worker and not the user

Unsmart

U

Unsmart•6/26/24, 6:59 PM

I dont think cloudflare offers a read only view of workers resources though unless I am just missing it so they wont be able to use UI to view any D1/KV/etc stuff

rampatra

R

rampatra•6/26/24, 8:16 PM

Can someone help me with this issue - https://github.com/cloudflare/cloudflare-docs/issues/15343 ?

GitHub

Need instructions for existing workers · Issue #15343 · cloudflare/...

Existing documentation URL(s) Doc link - https://developers.cloudflare.com/d1/get-started/ This page lists nicely the steps when creating a worker from scratch but what if someone already has a wor...

Need instructions for existing workers · Issue #15343 · cloudflare/...

rampatra

R

rampatra•6/26/24, 8:18 PM

I am trying to follow the instructions and set up this DB environment variable in my nextjs app but I am stuck at the very first step.

CleanShot_2024-06-26_at_21.16.582x.png

rampatra

R

rampatra•6/26/24, 8:25 PM

I think the doc is outdated. Let me try something.

surgery18

S

surgery18•6/27/24, 12:18 AM

Hey, my project I am working on (uses pages, a few workers, R2, durable objects, and queue) currently uses Mongodb. I was thinking of converting my DB and my entire code base to use D1 instead. Before I do that, I want to know a couple of things.

1) Can D1 be used for a commerical project? Or is it more for hobby/personal projects? I am thinking for amount of requests hitting the db and what it can handle.
2) I think this is the big one, would I notice a big difference in speed from mongo to D1. If I convert to D1 then the whole architecture would be on Cloudflare and in my mind that means it would be quicker.

Thanks

Disciple of Caoximeihao

D

Disciple of Caoximeihao•6/28/24, 9:55 AM


      SELECT DISTINCT *
      FROM variantValues vv
      JOIN options o ON o.optionId = vv.optionId
      JOIN optionValues ov ON ov.valueId = vv.valueId
      WHERE vv.variantId IN '(7, 8, 9, 10)'


      SELECT DISTINCT *
      FROM variantValues vv
      JOIN options o ON o.optionId = vv.optionId
      JOIN optionValues ov ON ov.valueId = vv.valueId
      WHERE vv.variantId IN '(7, 8, 9, 10)'

hello, how to not working?

Disciple of Caoximeihao

D

Disciple of Caoximeihao•6/28/24, 9:55 AM

help

Muudka

M

Muudka•6/28/24, 11:00 AM

Hello,

We have a SaaS application that utilizes a shared codebase. When onboarding a new user, we create a dedicated database for them, apply migration schemas, load default data, and insert the user's details into a metadata database. This metadata is then linked to the user.

We have a centralized logging system that directs users to the appropriate database by retrieving their connection details from the metadata database and passing them to Prisma. We are currently using PostgreSQL with Prisma.

We would like to know if this behavior can be replicated using workers. Specifically, do the worker and D1 need to be bound simultaneously? At present, we have over 1,000 users, and this system is functioning well for us. The application is not very complex, so migrating from PostgreSQL to SQLite would not be an issue.

Additionally, can the worker accept connections to a D1 database that it is not bound to?

oof2win2

O

oof2win2•6/28/24, 2:28 PM

so i have multiple apps that share the same database, such as apps/connector, apps/dashboard, but the issue is how do i make them share the same database locally as well? on remote wrangler deployments it works because the database is identified by its id but locally it doesnt work and say the schema isnt shared

Alexandru Rosianu

AAlexandru Rosianu Seeing similar errors: > Error: D1 storage operation exceeded timeout which cau...

Alec | Kriebel LLC.

A

Alec | Kriebel LLC.•6/28/24, 3:28 PM

Just got this D1_ERROR: Internal error while starting up D1 storage caused object to be reset.D1_ERROR: Internal error while starting up D1 storage caused object to be reset. error again this morning on a DB with 118.78kb of data.

JustinNoel

J

JustinNoel•6/28/24, 7:55 PM

I'm not associated with this in any way (not even a customer). However, if you're new to SQLite of wonder if it's up to your needs, this might help.
https://highperformancesqlite.com/

Home - High Performance SQLite

A comprehensive video course by database educator Aaron Francis. Learn everything you need to confidently use SQLite in production.

Home - High Performance SQLite

Friday Candour

FFriday Candour Just buy a vps.

surgery18

S

surgery18•6/28/24, 8:03 PM

Huh? VPS? That isn't what I was asking.

peshakoo

Ppeshakoo you would def notice a big difference in speed. Most of my queries take less tha...

surgery18

S

surgery18•6/28/24, 8:33 PM

Ok, that is what I was thinking. Gonna dread doing the conversion since it is ALOT of files to modify, But in the end it should be well worth it.

Baqir

B

Baqir•6/29/24, 5:42 AM

Hi folks, need some help for creating D1 binding using Drizzle. I have this code.

I want to get the DB dynamically based on the client who user ID. Is there a solution?

I have a solution which is to create a mapping like const mapping = { [tenantId1] : env.DB_TENANT_1const mapping = { [tenantId1] : env.DB_TENANT_1 and so on, but is there something better?

Thanks

Capture-2024-06-29_at_11.09.18.png

Baqir

BBaqir Hi folks, need some help for creating D1 binding using Drizzle. I have this code...

Baqir

B

Baqir•6/29/24, 5:52 AM

Think I figured it out. Needed to add this line in my env.d.ts

[key: string]: D1Database;

Thanks to Claude

JustinNoel

J

JustinNoel•6/29/24, 2:19 PM

Update: Solved by adding

--remote

--remote

. See discession below.

I'm not sure I understand how to do a migration.

Scenario:

I've never used a migration before. I've modified a few tables in my local db, done my development code to deal with these changes.
Now, I've pushed my code and it's active in the preview environment of CF pages.

Migration Attempt:

I created a migration file using wrangler d1 migrations create preview-say-my-name move-authentication-to-workoswrangler d1 migrations create preview-say-my-name move-authentication-to-workos.

I edited the migration file with:

ALTER TABLE customers ADD COLUMN auth_service_user_id TEXT;
DROP TABLE IF EXISTS `magiclinks`;
DROP TABLE IF EXISTS `sessions`;
DROP TABLE IF EXISTS `login_passcodes`;

ALTER TABLE customers ADD COLUMN auth_service_user_id TEXT;
DROP TABLE IF EXISTS `magiclinks`;
DROP TABLE IF EXISTS `sessions`;
DROP TABLE IF EXISTS `login_passcodes`;

When I run wrangler d1 migrations apply preview-say-my-namewrangler d1 migrations apply preview-say-my-name, i get:

Migration 0001_move-authentication-to-workos.sql failed with the following errors:
✘ [ERROR] no such table: customers

Questions:

It seems that my original database and all its data is not used in the migration. So, it seems my migrations file is required to contain all the original schema and inserts for the exiting data.

Is that correct? This sure makes life painful and sort of makes migrations useless.

Do I have to continue doing this for every migration or once I've done the first, will the system be smart enough to carry data forward?

Feedback:

There's just not enough explanation of this in the docs.

JustinNoel

JJustinNoel **Update:** Solved by adding `--remote`. See discession below. I'm not sure I ...

Hello, I’m Allie!

H

Hello, I’m Allie!•6/29/24, 2:32 PM

Just to confirm, is this local mode?

Hello, I’m Allie!

HHello, I’m Allie!Just to confirm, is this local mode?

JustinNoel

J

JustinNoel•6/29/24, 2:42 PM

No. preview/staging is what i'm attempting right now.

Original message was deleted

JustinNoel

J

JustinNoel•6/29/24, 2:47 PM

That did it! i was assuming it would have read that info from my

wrangler.toml

wrangler.toml

because it had to add it for wrangler d1 migrations create preview-say-my-name move-authentication-to-workoswrangler d1 migrations create preview-say-my-name move-authentication-to-workos.

Thanks so much!

akira.eth

Aakira.eth const env = getMiniflareBindings() anyone know why this isn't returning my db bi...

Matt Silverlock

M

Matt Silverlock•6/29/24, 3:41 PM

What version are you on? https://developers.cloudflare.com/d1/build-with-d1/local-development/#miniflarehttps://developers.cloudflare.com/d1/build-with-d1/local-development/#miniflare


      SELECT DISTINCT *
      FROM variantValues vv
      JOIN options o ON o.optionId = vv.optionId
      JOIN optionValues ov ON ov.valueId = vv.valueId
      WHERE vv.variantId IN '(7, 8, 9, 10)'


      SELECT DISTINCT *
      FROM variantValues vv
      JOIN options o ON o.optionId = vv.optionId
      JOIN optionValues ov ON ov.valueId = vv.valueId
      WHERE vv.variantId IN '(7, 8, 9, 10)'