as of wrangler 3.15.0 `wrangler d1` commands default to prod, and need to be passed `--local` to ru

J

jiangok•11/15/23, 4:37 PM

I use CF pages + d1. On local development,

pnpm run build & npx wrangler pages dev ./public --compatibility-date=2023-11-15 --d1=DB

pnpm run build & npx wrangler pages dev ./public --compatibility-date=2023-11-15 --d1=DB

pnpm run build & npx wrangler pages dev ./public --compatibility-date=2023-11-15 --d1=DB

pnpm run build & npx wrangler pages dev ./public --compatibility-date=2023-11-15 --d1=DB creates a d1 database under

.wrangler/state/v3/d1/miniflare-D1DatabaseObject

.wrangler/state/v3/d1/miniflare-D1DatabaseObject

.wrangler/state/v3/d1/miniflare-D1DatabaseObject

.wrangler/state/v3/d1/miniflare-D1DatabaseObject folder. This is different from CF worker which store database under

.wrangler/state/v3/{database_id}

.wrangler/state/v3/{database_id}

.wrangler/state/v3/{database_id}

.wrangler/state/v3/{database_id} which can be operated like

wrangler d1 execute $DATABASE_NAME  --file=./seed.sql --local

wrangler d1 execute $DATABASE_NAME --file=./seed.sql --local

wrangler d1 execute $DATABASE_NAME  --file=./seed.sql --local

wrangler d1 execute $DATABASE_NAME --file=./seed.sql --local which observes

wrangler.toml

wrangler.toml

. How can I use wrangler to operate the local databases generated by CF pages like those generated by CF workers? It will be great if CF pages & workers have consistent local development model for D1. Thanks for any hints.

MMax (@rozenmd)as of wrangler 3.15.0 `wrangler d1` commands default to prod, and need to be pa...

B

bkyerv•11/15/23, 5:49 PM

what is the difference between the db file in the

{database_id}

{database_id}

{database_id}

{database_id} folder and the one in the

miniflare-D1DatabaseObject

miniflare-D1DatabaseObject

miniflare-D1DatabaseObject

miniflare-D1DatabaseObject folder? which one is local and which one is prod?

Bbkyerv what is the difference between the db file in the `{database_id}` folder and the...

H

Hello, I’m Allie!•11/15/23, 5:58 PM

Neither of them are prod. AFAIK, the

miniflare

miniflare

miniflare

miniflare-prefixed SQLite files are used for internal storage in

wrangler

wrangler

, like the names of bindings, metrics, etc

Bbkyerv what is the difference between the db file in the `{database_id}` folder and the...

M

Max (@rozenmd)OP•11/15/23, 6:17 PM

miniflare-D1DatabaseObject uses miniflare to create the sqlite file, while the other uses better-sqlite3 directly. both are local/neither are prod. I don't know why they aren't in the same location tbh

C

Cyb3r-Jak3•11/15/23, 6:42 PM

I thought the newer way was with the miniflare prefix and stuff was migrated over

_

_tweak^prone•11/15/23, 9:42 PM

We have a product thats about to release around late december 2023 - I would really like to include D1 for this project. How likely is it that it would be safe to use D1 - and/or that its actually released late december 2023 ?

(I can quickly switch to another db - its not a "major issue")

__tweak^prone We have a product thats about to release around late december 2023 - I would rea...

M

Matt Silverlock•11/15/23, 10:15 PM

https://blog.cloudflare.com/d1-open-beta-is-here/

We’re setting a clear goal: we want to make D1 “generally available” (GA) for production use-cases by early next year (Q1 2024).

The Cloudflare Blog

D1: open beta is here

D1 is now in open beta, and the theme is “scale”: with higher per-database storage limits and the ability to create more databases, we’re unlocking the ability for developers to build production-scale applications on D1

MMatt Silverlock https://blog.cloudflare.com/d1-open-beta-is-here/ > We’re setting a clear goal: ...

_

_tweak^prone•11/15/23, 11:34 PM

nice - thank you

MMatt Silverlock wrt D1 in South America - we’re working on it.

T

tadeumaia•11/16/23, 10:35 AM

Hello. Any news on this or maybe some alpha/beta/waitlist? This is holding me to use D1 for a project going live soon. Going turso right now as they already have 2 POP right now, But would love to avoid the migration.

Ddmio_co Hey, question about how D1 works. We have 1 central place where db is hosted/sto...

M

Matt Silverlock•11/16/23, 11:28 AM

Per https://blog.cloudflare.com/d1-open-beta-is-here/

read replication is something we’re working on

The Cloudflare Blog

D1: open beta is here

D1 is now in open beta, and the theme is “scale”: with higher per-database storage limits and the ability to create more databases, we’re unlocking the ability for developers to build production-scale applications on D1

Ttadeumaia Hello. Any news on this or maybe some alpha/beta/waitlist? This is holding me to...

M

Matt Silverlock•11/16/23, 11:28 AM

No news yet. Next year.

B

busterroni•11/16/23, 3:27 PM

Hi, I'm working on a flight search engine and am considering D1. I'm concerned that the 2GB database limit is too small to hold information for all upcoming flights -- there will be a couple million rows. We could also have one database per airport, to distribute the number of rows in each database. Do you think this is a good approach? Is D1 a good option for this kind of project?

Bbusterroni Hi, I'm working on a flight search engine and am considering D1. I'm concerned t...

C

Cyb3r-Jak3•11/16/23, 3:49 PM

Just curious, do you have an estimate of how large your database would be. It helps the team understand the needs better.

Ddmio_co KV? I presume data will not change too often

B

busterroni•11/16/23, 3:56 PM

The data will change often, prices and availability change

Bbusterroni The data will change often, prices and availability change

_

_tweak^prone•11/16/23, 4:04 PM

I might suspect that one DB per airport would result in rather a bit of a mess when doing lookups cross a few airports (?)

Bbusterroni Hi, I'm working on a flight search engine and am considering D1. I'm concerned t...

H

Hello, I’m Allie!•11/16/23, 4:46 PM

As a mockup, try pulling that data into an SQLite file, and see how big it is

H

Hello, I’m Allie!•11/16/23, 4:46 PM

You might be able to use certain techniques to minify it a tad

B

busterroni•11/16/23, 4:56 PM

It's going to be well over 2GB, there are millions of flights a year and a lot of updates

Bbusterroni It's going to be well over 2GB, there are millions of flights a year and a lot o...

S

Sam•11/16/23, 7:09 PM

Would recommend you using a Postgres cluster with CF Hyperdrive

Ttadeumaia Hello. Any news on this or maybe some alpha/beta/waitlist? This is holding me to...

J

João Castro•11/16/23, 9:08 PM

+1, it would be a huge improvement to have D1 in South America

U

Unsmart•11/17/23, 1:31 AM

D1 has never really had many downtime issues tbh beyond maybe weird edge cases for a very very few amount of people.

C

Cyb3r-Jak3•11/17/23, 2:52 AM

The API is always going to be slower. I would recommend making a proxy worker

C

Cyb3r-Jak3•11/17/23, 3:22 AM

The API is always going to be slower due to how the auth is handled

SSotashi Nokomata would be nice to have D1's HTTP API reduce the latency, for simple use cases dir...

M

Max (@rozenmd)OP•11/17/23, 9:16 AM

actually working on that

V

VIO•11/17/23, 12:21 PM

I just hit the 2GB limit

V

VIO•11/17/23, 12:22 PM

is there anyway to get a dumb of my db

V

VIO•11/17/23, 12:22 PM

?

M

Max (@rozenmd)OP•11/17/23, 12:55 PM

https://github.com/nora-soderlund/cloudflare-d1-backups until we build it natively

M

maguayo•11/17/23, 1:44 PM

Is D1 down?
Error: D1_TYPE_ERROR: Type 'undefined' not supported for value 'undefined'

K

kian•11/17/23, 1:47 PM

That isn't an issue with D1,

undefined

undefined

has never been supported.

K

kian•11/17/23, 1:47 PM

https://developers.cloudflare.com/d1/platform/client-api/#type-conversion

Not supported. Queries with undefined values will return a D1_TYPE_ERROR

K

kian•11/17/23, 1:47 PM

Make sure you're not passing

undefined

undefined

to

bind

bind

Kkian That isn't an issue with D1, `undefined` has never been supported.

M

maguayo•11/17/23, 1:48 PM

The app was made 1 year ago, did not change anything, but it's failing today

K

kian•11/17/23, 1:50 PM

I can only recommend logging out the parameters you are passing to

bind

bind

and seeing which one is undefined, and going from there - the part of the D1 binding that throws that error hasn't changed functionally in a very long time.

J

jiangok•11/17/23, 5:17 PM

hi, my CF page production cannot find the d1 database while preview can:

[env.staging]
d1_databases = [
    { binding = "DB", database_name = "test1", database_id = "1111", preview_database_id = "DB" },
]

wrangler pages deploy ./public --project-name myproj --env staging  # works

[env.production]
d1_databases = [
    { binding = "DB", database_name = "test1-prod", database_id = "1111" },
]

wrangler pages deploy ./public --project-name myproj --env production --branch main # throw "Cannot read properties of undefined (reading 'prepare')"

[env.staging]
d1_databases = [
    { binding = "DB", database_name = "test1", database_id = "1111", preview_database_id = "DB" },
]

wrangler pages deploy ./public --project-name myproj --env staging  # works

[env.production]
d1_databases = [
    { binding = "DB", database_name = "test1-prod", database_id = "1111" },
]

wrangler pages deploy ./public --project-name myproj --env production --branch main # throw "Cannot read properties of undefined (reading 'prepare')"

[env.staging]
d1_databases = [
    { binding = "DB", database_name = "test1", database_id = "1111", preview_database_id = "DB" },
]

wrangler pages deploy ./public --project-name myproj --env staging  # works

[env.production]
d1_databases = [
    { binding = "DB", database_name = "test1-prod", database_id = "1111" },
]

wrangler pages deploy ./public --project-name myproj --env production --branch main # throw "Cannot read properties of undefined (reading 'prepare')"

[env.staging]
d1_databases = [
    { binding = "DB", database_name = "test1", database_id = "1111", preview_database_id = "DB" },
]

wrangler pages deploy ./public --project-name myproj --env staging  # works

[env.production]
d1_databases = [
    { binding = "DB", database_name = "test1-prod", database_id = "1111" },
]

wrangler pages deploy ./public --project-name myproj --env production --branch main # throw "Cannot read properties of undefined (reading 'prepare')"

Any idea? Thanks very much!

M

Matt Silverlock•11/17/23, 5:22 PM

Pages doesn’t use

wrangler.toml

wrangler.toml

in production: https://developers.cloudflare.com/pages/platform/functions/bindings/#d1-databases

Bindings · Cloudflare Pages docs

A binding enables your Pages Functions to interact with resources on the Cloudflare developer platform. Use bindings to integrate your Pages Functions …

E

Edu•11/17/23, 6:16 PM

hello, most probably you are already tracking this; do you have an issue or place to track progress on the branching mechanism?

EEdu hello, most probably you are already tracking this; do you have an issue or plac...

I

Isaac McFadyen•11/17/23, 6:17 PM

Teams at Cloudflare don't really track progress on internal things externally, for the same reason they don't have release dates on things - if they need to push it back or rework for whatever reason they don't want to promise things a certain way or date

I

Isaac McFadyen•11/17/23, 6:17 PM

It's definitely being worked on or planned internally but we don't really have a way to track externally

CCyb3r-Jak3 Only if you are enterprise. The recommended way would be to run an HTTP proxy wo...

Ɇ

Ɇ₦ĐɆⱤV₳₥₱łⱤɆ•11/17/23, 6:19 PM

gross

as of wrangler 3.15.0 `wrangler d1` commands default to prod, and need to be passed `--local` to ru

Similar Threads