Is there any specific write/read per second limit on D1? Docs says nothing like that

MMatt Silverlock Changelog going out soon, but for those here: * D1's `.raw()`, `.all()` and `.r...

Is there a reason this couldn't have been a compat-date gated change? Seems quite a few people are running into this breaking change, and having a Worker in prod break when not expected is exactly what compat dates were designed to solve...

ⱤⱤØ₥₳₦Is there any specific write/read per second limit on D1? Docs says nothing like ...

Max (@rozenmd)•2/16/24, 2:02 PM

It's not a "limit" we set, as much as a beta architectural limitation - your D1 database currently runs from a single durable object instance, and there's only so much work a single thread can do per second before overloading

IIsaac McFadyen Is there a reason this couldn't have been a compat-date gated change? Seems quit...

James•2/16/24, 2:12 PM

Agreed, R2 and Vectorize had similar changes to their data structures and things and just introduced those with a compat date/flag. Feels weird for a product like D1 that’s even further along to not have done that.

IIsaac McFadyen Is there a reason this couldn't have been a compat-date gated change? Seems quit...

Max (@rozenmd)•2/16/24, 2:25 PM

We saw it as a bug fix - the correct behaviour was documented, and

.run()

.run()

used to work as documented

MMax (@rozenmd)We saw it as a bug fix - the correct behaviour was documented, and `.run()` used...

James•2/16/24, 2:30 PM

It still broke people’s workers though unfortunately, and especially if they’re not in discord or don’t know to manually check changelogs, they’re left confused and frustrated that code that they previously had working, no longer does, with no change of their own. Unless there were alerts sent to people impacted outside of that?

JJames It still broke people’s workers though unfortunately, and especially if they’re ...

Vy•2/16/24, 2:56 PM

We're looking into a resolution here to restore the previous

.run

.run

behvaior and I'll report back. Apologies and yes we should/will be applying compatibility dates to D1 breaking chnages

kenton•2/16/24, 3:00 PM

Yes, apologies everyone, this was a mistake. Changes like this should absolutely be guarded by a compat flag, even in beta/alpha, even if the behavior was undocumented, but we screwed up. We're working on restoring the previous behavior now and will find ways to make sure this doesn't happen again.

Rrenzor has anyone noticed when you delete a db row in the UI, it deletes the entire thi...

Max (@rozenmd)•2/16/24, 5:25 PM

is the primary key null?

Rrenzor it is possible. I have a compound primary key set via a drizzle schema, and have...

Max (@rozenmd)•2/16/24, 5:45 PM

It's a bug - it shouldn't be showing you the delete button here because deleting a row with more than one column as a PK isn't implemented yet

Max (@rozenmd)•2/16/24, 5:45 PM

it's basically just deleting by the first column in your compound key

Vy•2/16/24, 5:46 PM

Sharing that we've gone back to previous

.run

.run

behavior where it does return results. We will be differentiating .run (no results) vs .all (with results) behind a compatibility date in the future

VVy Sharing that we've gone back to previous `.run` behavior where it does return re...

kenton•2/16/24, 6:26 PM

To clarify,

.run

.run

now once returns results as it used to.

mllnd•2/16/24, 6:30 PM

hey, quick question - would it make more sense to set up D1 connection on the server side of a cloudflare pages app or move all db interactions on the cloudflare functions side?

Mmllnd hey, quick question - would it make more sense to set up D1 connection on the se...

Hello, I’m Allie!•2/16/24, 6:32 PM

On Pages, Functions is the server

Hello, I’m Allie!•2/16/24, 6:32 PM

So both

mllnd•2/16/24, 6:40 PM

Ah, I see... been trying to make sense of the documentation. So the Nuxt.js project's compiler is responsible for mapping all server-side API routes into CF compatible Functions?

Mmllnd Ah, I see... been trying to make sense of the documentation. So the Nuxt.js proj...

Hello, I’m Allie!•2/16/24, 6:50 PM

Yes. When you are using a framework like Nuxt that has SSR/API routes, then you are not able to create Functions manually(or if you are, then it will break their stuff)

mllnd•2/16/24, 7:06 PM

Great, thanks for the clarification!

Original message was deleted

maitreya•2/17/24, 5:50 PM

getting similiar processing time for d1, one of the main reasons i havent moved yet

https://t3-app-d1.pages.dev/api/db-test/d1

oddly when using planetscale, i get higher latency in cloudflare workers than in vercel edge.

kian•2/17/24, 5:54 PM

A UK Worker to a WEUR DB hovers around 60ms to 100ms personally

itsmatteomanf•2/17/24, 6:08 PM

My DB fully imploded from a single DELETE query (it was 15d of logs, maybe, not sure what you would classify that) .

itsmatteomanf•2/17/24, 6:10 PM

Can't reach it anymore... id is

546787dd-35d5-4822-9b50-36fa0a4310c2

546787dd-35d5-4822-9b50-36fa0a4310c2

itsmatteomanf•2/17/24, 6:10 PM

Mind you, the column used on the

WHERE

WHERE

part has an index.

itsmatteomanf•2/17/24, 6:11 PM

And it was at max size, but I already deleted a bit of data to test the query worked, so wasn't technically anymore.

Iitsmatteomanf My DB fully imploded from a single DELETE query (it was 15d of logs, maybe, not ...

kenton•2/17/24, 8:05 PM

doh, the scheduling algorithm made a bad call and colocated your database with some other big chonkers and they ran out of disk space. We kicked the machine which should redistribute all the databases and are working on a more lasting fix to this issue. In any case your DB should be coming back now.

Kkenton doh, the scheduling algorithm made a bad call and colocated your database with s...

itsmatteomanf•2/17/24, 8:05 PM

Oh, thanks Kenton!

itsmatteomanf•2/17/24, 8:06 PM

It’s interesting it was broken after it was reduced in size ahah

kenton•2/17/24, 8:07 PM

heh, well the problem is "reducing" the size briefly increases the size due to writing all the changes to the WAL file

Kkenton heh, well the problem is "reducing" the size briefly increases the size due to w...

itsmatteomanf•2/17/24, 8:08 PM

Oh, yeah, true…

Kkenton doh, the scheduling algorithm made a bad call and colocated your database with s...

itsmatteomanf•2/17/24, 8:34 PM

Can confirm it's back! And the DELETE actually was successful before the error :lul:

Original message was deleted

kenton•2/17/24, 9:27 PM

arguably a bug, but not something we can easily fix. The sqlite DB contains some hidden metadata tables that we don't want to give applications direct access to, which means we use an authorizer callback to block access. If you try to enumerate all tables and query them all you will likely run afoul of this.

itsmatteomanf•2/17/24, 9:32 PM

Is there any specific way, or plans for a way, to see storage used by each table and, not sure if possible, each index?

Iitsmatteomanf Is there any specific way, or plans for a way, to see storage used by each table...

Max (@rozenmd)•2/17/24, 9:52 PM

SELECT SUM("pgsize") FROM "dbstat" WHERE name='TABLENAME';

Max (@rozenmd)•2/17/24, 9:52 PM

apparently

Max (@rozenmd)•2/17/24, 9:54 PM

SELECT name, SUM("pgsize") FROM "dbstat" GROUP BY name;

for all tables

MMax (@rozenmd)SELECT name, SUM("pgsize") FROM "dbstat" GROUP BY name; for all tables

itsmatteomanf•2/17/24, 9:54 PM

Was literally typing that out :lul:

itsmatteomanf•2/17/24, 9:54 PM

Thanks!

itsmatteomanf•2/17/24, 9:54 PM

Let me try it

Max (@rozenmd)•2/17/24, 9:54 PM

includes indexes too!

itsmatteomanf•2/17/24, 9:54 PM

Uhm...

no such table: dbstat

no such table: dbstat

kian•2/17/24, 9:55 PM

proobably isn't enabled in D1

itsmatteomanf•2/17/24, 9:55 PM

I'm not using any storage! :party_yay:

kian•2/17/24, 9:55 PM

it's a compile-time flag (

SQLITE_ENABLE_DBSTAT_VTAB

SQLITE_ENABLE_DBSTAT_VTAB

)

Kkian proobably isn't enabled in D1

itsmatteomanf•2/17/24, 9:55 PM

Yeah, I figured...

Is there any specific write/read per second limit on D1? Docs says nothing like that

Similar Threads

Similar Threads

Similar Threads