It won’t be soon. What is your goal? To backup outside? To run locally? Something else?

C

chientrm.com•8/27/23, 1:53 PM

Wish for the increment of 10 D1 databases limit

J

JThiel•8/27/23, 3:05 PM

Hey there, kind of cross-posting this as I posted this already in #next-on-pages. Yet I am unsure if this is actually a D1 or Pages / Next issue. We encouter the following error on a deployed app only. No issues locally with Wrangler (3.5.0) and the same app code:

"Error: D1_TYPE_ERROR: Type 'object' not supported for value 'Tue Sep 26 2023 12:48:07 GMT+0000 (Coordinated Universal Time)'
    at D1PreparedStatement.bind (bundledWorker-0.17018918700489616.mjs:898:17)
    at s.executeQuery (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:12:44988)
    at __next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:48052
    at Ii.provideConnection (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:83082)
    at async at.executeQuery (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:47999)
    at async _e.execute (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:19677)
    at async _e.executeTakeFirst (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:19900)
    at async _e.executeTakeFirstOrThrow (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:19972)
    at async createSession (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:91101)
    at async Ue.<computed> (__next-on-pages-dist__/webpack/71992532242772f20ff0087800cdd46e.js:56:78937)"

"Error: D1_TYPE_ERROR: Type 'object' not supported for value 'Tue Sep 26 2023 12:48:07 GMT+0000 (Coordinated Universal Time)'
    at D1PreparedStatement.bind (bundledWorker-0.17018918700489616.mjs:898:17)
    at s.executeQuery (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:12:44988)
    at __next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:48052
    at Ii.provideConnection (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:83082)
    at async at.executeQuery (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:47999)
    at async _e.execute (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:19677)
    at async _e.executeTakeFirst (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:19900)
    at async _e.executeTakeFirstOrThrow (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:19972)
    at async createSession (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:91101)
    at async Ue.<computed> (__next-on-pages-dist__/webpack/71992532242772f20ff0087800cdd46e.js:56:78937)"

"Error: D1_TYPE_ERROR: Type 'object' not supported for value 'Tue Sep 26 2023 12:48:07 GMT+0000 (Coordinated Universal Time)'
    at D1PreparedStatement.bind (bundledWorker-0.17018918700489616.mjs:898:17)
    at s.executeQuery (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:12:44988)
    at __next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:48052
    at Ii.provideConnection (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:83082)
    at async at.executeQuery (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:47999)
    at async _e.execute (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:19677)
    at async _e.executeTakeFirst (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:19900)
    at async _e.executeTakeFirstOrThrow (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:19972)
    at async createSession (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:91101)
    at async Ue.<computed> (__next-on-pages-dist__/webpack/71992532242772f20ff0087800cdd46e.js:56:78937)"

"Error: D1_TYPE_ERROR: Type 'object' not supported for value 'Tue Sep 26 2023 12:48:07 GMT+0000 (Coordinated Universal Time)'
    at D1PreparedStatement.bind (bundledWorker-0.17018918700489616.mjs:898:17)
    at s.executeQuery (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:12:44988)
    at __next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:48052
    at Ii.provideConnection (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:83082)
    at async at.executeQuery (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:47999)
    at async _e.execute (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:19677)
    at async _e.executeTakeFirst (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:19900)
    at async _e.executeTakeFirstOrThrow (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:19972)
    at async createSession (__next-on-pages-dist__/webpack/9b3bdf9eee839442fe37ed586efaff1d.js:1:91101)
    at async Ue.<computed> (__next-on-pages-dist__/webpack/71992532242772f20ff0087800cdd46e.js:56:78937)"

SQL Schema

CREATE TABLE "Session"
(
    "id"           INTEGER  NOT NULL PRIMARY KEY AUTOINCREMENT UNIQUE,
    "sessionToken" TEXT     NOT NULL,
    "userId"       TEXT     NOT NULL,
    "expires"      DATETIME NOT NULL,
    FOREIGN KEY ("userId") REFERENCES "User" ("id") ON DELETE RESTRICT ON UPDATE CASCADE
);

CREATE TABLE "Session"
(
    "id"           INTEGER  NOT NULL PRIMARY KEY AUTOINCREMENT UNIQUE,
    "sessionToken" TEXT     NOT NULL,
    "userId"       TEXT     NOT NULL,
    "expires"      DATETIME NOT NULL,
    FOREIGN KEY ("userId") REFERENCES "User" ("id") ON DELETE RESTRICT ON UPDATE CASCADE
);

CREATE TABLE "Session"
(
    "id"           INTEGER  NOT NULL PRIMARY KEY AUTOINCREMENT UNIQUE,
    "sessionToken" TEXT     NOT NULL,
    "userId"       TEXT     NOT NULL,
    "expires"      DATETIME NOT NULL,
    FOREIGN KEY ("userId") REFERENCES "User" ("id") ON DELETE RESTRICT ON UPDATE CASCADE
);

CREATE TABLE "Session"
(
    "id"           INTEGER  NOT NULL PRIMARY KEY AUTOINCREMENT UNIQUE,
    "sessionToken" TEXT     NOT NULL,
    "userId"       TEXT     NOT NULL,
    "expires"      DATETIME NOT NULL,
    FOREIGN KEY ("userId") REFERENCES "User" ("id") ON DELETE RESTRICT ON UPDATE CASCADE
);

DB Type for Kysely (with D1Dialect)

export interface Database {
...
    Session: {
        id: GeneratedAlways<string>
        userId: string
        sessionToken: string
        expires: string
    };
...
}

export interface Database {
...
    Session: {
        id: GeneratedAlways<string>
        userId: string
        sessionToken: string
        expires: string
    };
...
}

export interface Database {
...
    Session: {
        id: GeneratedAlways<string>
        userId: string
        sessionToken: string
        expires: string
    };
...
}

export interface Database {
...
    Session: {
        id: GeneratedAlways<string>
        userId: string
        sessionToken: string
        expires: string
    };
...
}

Any obvious fails on our end?

MMatt Silverlock It won’t be soon. What is your goal? To backup outside? To run locally? Somethin...

M

mukhtharcm•8/27/23, 3:11 PM

It was to run locally.

I have kind of complex setup for my dev.

I'm using kysely for using d1,

when developing locally, I connect it to a local sqlite file using better-sqlite

M

mukhtharcm•8/27/23, 3:11 PM

I was thinking, if we could download the db, I can get the production data on local

M

mukhtharcm•8/27/23, 3:12 PM

I think it was available before. I remember downloading it in an old prject

D

dominik-mukrecki•8/28/23, 11:05 AM

Hi, any plans to add postgres compatibility mode into D1? Regards

Ddominik-mukrecki Hi, any plans to add postgres compatibility mode into D1? Regards 🙂

M

Matt SilverlockOP•8/28/23, 11:53 AM

No plans. None of the “Postgres compatibility” modes work very well — there are too many differences. ORMs don’t work well. You will find yourself frustrated debugging the differences. You can’t just copy queries over.

Ddominik-mukrecki Hi, any plans to add postgres compatibility mode into D1? Regards 🙂

M

Matt SilverlockOP•8/28/23, 11:53 AM

Instead: what about Postgres do you actually need?

MMatt Silverlock Instead: what about Postgres do you actually need?

D

dominik-mukrecki•8/28/23, 12:00 PM

I asked because of Postgres management is sometimes frustrated and there is no ideal managed or self-hosted solution. Drop-in replacement will be great for many applications, something like Supabase, where the legacy application could be connected directly to the DB and changes are monitored in the realtime, with the ability to track changes for example.

Ddominik-mukrecki I asked because of Postgres management is sometimes frustrated and there is no i...

M

Matt SilverlockOP•8/28/23, 12:40 PM

Broadcasting real time changes doesn’t require Postgres though — D1 could certainly do that at some point

E

EvanPiro•8/28/23, 5:36 PM

How unreliable is D1 currently? I'm currently developing an application that will be live in the next few weeks and am strongly considering using D1 for the backend and dealing with alpha issues as they come.

EEvanPiro How unreliable is D1 currently? I'm currently developing an application that wil...

M

Matt SilverlockOP•8/28/23, 5:44 PM

It’s alpha, but… we’re pretty close to beta. Most folks don’t encounter any hard errors: we’re addressing the remaining transient ones we see. Lots of folks running it day to day. But I have the responsibility of saying “beta”.

E

EvanPiro•8/28/23, 6:00 PM

That's great to hear! I'm trying to move all my go-to solutions from AWS to Cloudflare. The next one on my list is Cognito. Do you have a guide or some docs on such a migration / feature parity with AWS?

EEvanPiro That's great to hear! I'm trying to move all my go-to solutions from AWS to Clou...

M

Matt SilverlockOP•8/28/23, 6:02 PM

What are you using Cognito for? For internal users: Cloudflare Access (see the public docs) is the go-to. We don’t have an “external” user-facing auth product but Clerk (clerk.dev) is fantastic.

MMatt Silverlock What are you using Cognito for? For internal users: Cloudflare Access (see the p...

E

EvanPiro•8/28/23, 6:06 PM

I'm using it for exactly what Clerk does : ). Thanks for the pointer!

E

EvanPiro•8/28/23, 6:11 PM

Is there a solution similar to AWS CloudFormation where I can export / import / diff all of the config of my cloudflare account? Ideally I'd like to have all the settings in my account as versioned YML and rely on CI to sync changes.

E

EvanPiro•8/28/23, 6:12 PM

Apologies if this is getting off topic

U

Unsmart•8/28/23, 6:15 PM

Cloudflare doesnt really have any custom thing for that. Best bet is either terraform or pulumi... pulumi is probably easier to use but not sure how much it supports

MMatt Silverlock It’s alpha, but… we’re pretty close to beta. Most folks don’t encounter any hard...

R

Rabid•8/28/23, 7:33 PM

What kind of reasonable limits do you forsee? I know the limit is going to increase from 500 MB in beta, but do you see this increasing to something like 1 GB? 10 GB? 100 GB? I don't need it to be enormous, but just want to make sure I'm not going to kneecap myself in the future if my app gets a decent amount of users.

RRabid What kind of reasonable limits do you forsee? I know the limit is going to incr...

C

chientrm.com•8/28/23, 7:34 PM

you can self-hosted D1

Cchientrm.com you can self-hosted D1

H

Hello, I’m Allie!•8/28/23, 9:55 PM

I mean, if you are going to self-host anyway, then you might as well use a DB built for larger sizes

H

Hello, I’m Allie!•8/28/23, 9:56 PM

You might be able to push 100 GB through local D1, but why risk it when you can just spin up Postgres or MySQL

R

Rabid•8/28/23, 10:21 PM

Yeah if I was going to self host I would just use postgres, or just stick with RDS

U

Unsmart•8/28/23, 11:00 PM

self hosting sucks just use planetscale if u need more then use d1 when its ga

HHello, I’m Allie!I mean, if you are going to self-host anyway, then you might as well use a DB bu...

C

chientrm.com•8/29/23, 1:52 AM

Then you don't need to rewrite queries

J

jared•8/29/23, 4:13 PM

Hi. Is there a way to call external endpoints (or Cloudflare Worker APIs) on D1 table changes (insert, update, delete)? Something like Firebase Functions for Firestore or Webhooks for Supabase (for those that are familiar).

Jjared Hi. Is there a way to call external endpoints (or Cloudflare Worker APIs) on D1 ...

C

chientrm.com•8/29/23, 4:26 PM

https://github.com/chientrm/wrangler-proxy

GitHub

GitHub - chientrm/wrangler-proxy: Wrangler Proxy exposes Workers AP...

Wrangler Proxy exposes Workers API to outside and integrate to your favorite frameworks. Compatible with DrizzleORM. - GitHub - chientrm/wrangler-proxy: Wrangler Proxy exposes Workers API to outsid...

Cchientrm.com https://github.com/chientrm/wrangler-proxy

J

jared•8/29/23, 7:03 PM

How can I use this to watch for changes in the D1 tables? To be more clear, the goal is that every time there are inserts, updates, deletes in my tables, a function should be automatically triggered to send the corresponding data to an external endpoint.

Jjared How can I use this to watch for changes in the D1 tables? To be more clear, the ...

C

chientrm.com•8/29/23, 7:04 PM

AFAIK D1 doesn't have trigger

Jjared How can I use this to watch for changes in the D1 tables? To be more clear, the ...

M

Max (@rozenmd)•8/30/23, 6:19 AM

we don't support this yet, but it's cool and something we want to look into

Jjared How can I use this to watch for changes in the D1 tables? To be more clear, the ...

M

Mia•8/30/23, 9:24 AM

i suppose a workaround would be to have the code that's making the changes fire a webhook to another endpoint

I

itsmatteomanf•8/30/23, 9:24 AM

Any plans to expose rows read/written + time on console queries?

I

itsmatteomanf•8/30/23, 9:24 AM

And indexes on tables in the UI?

MMia i suppose a workaround would be to have the code that's making the changes fire ...

M

Mia•8/30/23, 9:26 AM

not a super useful response if you dont have control over that code

I

itsmatteomanf•8/30/23, 9:26 AM

I presume there is no way to see query history with stats, right?

MMia not a super useful response if you dont have control over that code

I

itsmatteomanf•8/30/23, 9:27 AM

D1 can be controlled only via Workers in your account, so something you control is always in the middle.

M

Mia•8/30/23, 9:27 AM

fair enough

M

Mia•8/30/23, 9:29 AM

i finally found a project to actually use d1 with so im rly excited

M

Mia•8/30/23, 9:29 AM

spent like 3 days reading documentation and messing around with postman to figure out some new apis

M

Mia•8/30/23, 9:33 AM

currently figuring out a diagram for the db & tables

M

Mia•8/30/23, 9:33 AM

figure out all the data i need to store and how to store it

Iitsmatteomanf Any plans to expose rows read/written + time on console queries?

M

Max (@rozenmd)•8/30/23, 11:53 AM

yes, i've been meaning to add a status bar-like thingy

Iitsmatteomanf And indexes on tables in the UI?

M

Max (@rozenmd)•8/30/23, 11:54 AM

can do that, yeah

MMax (@rozenmd)can do that, yeah

I

itsmatteomanf•8/30/23, 12:00 PM

Thank you! Might be cool to have, but not sure if SQLite has it, the full create table exposed.

E

Elliot Hesp•8/30/23, 12:55 PM

Reading this blog: https://blog.cloudflare.com/d1-turning-it-up-to-11/

It mentions:

Importantly, when we enable global read replication, you won’t have to pay extra for it, nor will replication multiply your storage consumption

Importantly, when we enable global read replication, you won’t have to pay extra for it, nor will replication multiply your storage consumption

Importantly, when we enable global read replication, you won’t have to pay extra for it, nor will replication multiply your storage consumption

Importantly, when we enable global read replication, you won’t have to pay extra for it, nor will replication multiply your storage consumption

Does D1 read globally? Or is it still region based?

The Cloudflare Blog

D1: We turned it up to 11

We've been heads down iterating on D1, and we've just shipped a major new version that's substantially faster, more reliable, and introduces Time Travel: the ability to restore a D1 database to any point in time.

It won’t be soon. What is your goal? To backup outside? To run locally? Something else?

Similar Threads

Similar Threads

Similar Threads