hey! am i remembering correctly that D1 is internally implemented on top of Durable Objects?

AAvi hey! am i remembering correctly that D1 is internally implemented on top of Dura...

So it used to be, and still is, partially. There were two backends:
Alpha: implemented on Durable Objects using the Durable Objects Storage API. This was quite slow and eventually deprecated and will no longer receive updates.
Beta and Production (what we have now): a whole new SQLite storage engine was implemented on top of Durable Objects, meaning it still uses DOs but the new engine. The SQLite engine is not yet exposed to users but they've said they eventually want to allow DOs to directly access it.

A Twitter thread by the Workers lead engineer on this topic: https://x.com/KentonVarda/status/1659551757796515846

Original message was deleted

Bọc Link VN•5/30/24, 8:22 AM

thank you for sharing

divby0•5/30/24, 10:26 AM

this makes the migrations fail since they are in the wrong order. How does wrangler decide which one to apply first?

divby0•5/30/24, 10:27 AM

like I can change the pattern for naming these, but I thought putting the date in the name was the best way to make it be in the right order automatically

Ddivby0 this makes the migrations fail since they are in the wrong order. How does wrang...

Max (@rozenmd)•5/30/24, 11:01 AM

https://github.com/cloudflare/workers-sdk/blob/main/packages%2Fwrangler%2Fsrc%2Fd1%2Fmigrations%2Fapply.tsx#L101

GitHub

workers-sdk/packages/wrangler/src/d1/migrations/apply.tsx at main ·...

Home to Wrangler, the CLI for Cloudflare Workers® - cloudflare/workers-sdk

workers-sdk/packages/wrangler/src/d1/migrations/apply.tsx at main ·...

Max (@rozenmd)•5/30/24, 11:01 AM

theres a migration create command that uses the expected naming

MMax (@rozenmd)https://github.com/cloudflare/workers-sdk/blob/main/packages%2Fwrangler%2Fsrc%2F...

divby0•5/30/24, 11:56 AM

not too advanced, but when I read correct, I can simply drop the dashes inbetween the numbers but the "_migration" is okay since it splits there?

Original message was deleted

divby0•5/30/24, 12:00 PM

Got it - okay I have just worked around it temporarily by adding the files into the migrations folder one by one until it's fixed

divby0•5/30/24, 12:00 PM

Thank you all very much!

Original message was deleted

Max (@rozenmd)•5/30/24, 12:12 PM

aight i'll cut a PR

Mrinank•5/30/24, 6:06 PM

does D1 binding act like a VPC? Some people suggest putting our databases inside a VPC for security reasons, so that the databases are not accessible through the internet.

Mrinank•5/30/24, 6:11 PM

and can we expose our database_id in git publically?

MMrinank does D1 binding act like a VPC? Some people suggest putting our databases inside...

Hello, I’m Allie!•5/30/24, 6:18 PM

All bindings are only accessible on your account, yes

Hello, I’m Allie!•5/30/24, 6:18 PM

So it is safe to publish to git

Suman!•5/31/24, 7:28 AM

Is anybody aware of this issue? Trying to use d1-dialect causes my tests to fail with-

TypeError: Cannot use require() to import an ES Module.

TypeError: Cannot use require() to import an ES Module.

https://github.com/aidenwallis/kysely-d1/issues/27 . Please help.

GitHub

Using D1Dialect causes worker tests to fail - Cannot use require() ...

Initializing D1Dialect causes my tests to not run as expected. private async initializeDb(db: D1Database) { this.db = new Kysely<Database>({ dialect: new D1Dialect({ database: db }), }); } Ha...

andrew•5/31/24, 7:56 AM

I'm excited about potentially building my next app fully native on CF. My only blocker is d1's storage limits. to work around i'd have to support db/customer. my question is making the migration story and dynamically creating dbs more ergonomic on the roadmap?

Aandrew 👋 I'm excited about potentially building my next app fully native on CF. My onl...

Max (@rozenmd)•5/31/24, 8:08 AM

it is indeed on the roadmap - i just (yesterday) drafted the specification for how dynamic bindings and migrations will fit together

Denni•5/31/24, 12:05 PM

Hey all! I'm having some trouble with my D1 database. It looks like it's using a cached result, both using the CLI and in my wrangler and I'm quite unsure why

Denni•5/31/24, 12:05 PM

Denni•5/31/24, 12:05 PM

This is the actual data, as you can see there are total_usages which are much high than what the cli is returning. Am I doing something wrong here? I simply want to get the highest total_usage from the database

Original message was deleted

Denni•5/31/24, 12:06 PM

--remote

--remote

is in there already

Denni•5/31/24, 12:06 PM

My deployed wrangler also returns the same incorrect result

Denni•5/31/24, 12:07 PM

It's also coincidentally the latest result from when I dumped the db yesterday and recreated it (the pack_id column was an INTEGER before, but it's usually above the max_safe_int)

DDenni It's also coincidentally the latest result from when I dumped the db yesterday a...

Max (@rozenmd)•5/31/24, 12:40 PM

are they definitely querying the same database id?

DDenni Hey all! I'm having some trouble with my D1 database. It looks like it's using a...

Matt Silverlock•5/31/24, 12:47 PM

wrangler

wrangler

and/or D1 don't cache results (at all). As Max said: can you confirm it's the same database ID here? Your screenshot shows it starting with

50a19e5e

50a19e5e

- is that the same DB you're looking at in the dash?

MMax (@rozenmd)are they definitely querying the same database id?

Denni•5/31/24, 12:48 PM

They're both using the same wrangler.toml I'd think

DDenni They're both using the same wrangler.toml I'd think

Matt Silverlock•5/31/24, 12:48 PM

Share it with us (redact anything - but IDs are safe to share) to be sure.

Denni•5/31/24, 12:49 PM

[[d1_databases]]
binding = "USAGE_DB"
database_name = "sticker-usage"
database_id = "50a19e5e-a287-41ea-b908-b27ed1cb4787"

[[d1_databases]]
binding = "USAGE_DB"
database_name = "sticker-usage"
database_id = "50a19e5e-a287-41ea-b908-b27ed1cb4787"

Denni•5/31/24, 12:49 PM

Denni•5/31/24, 12:49 PM

Both the same

Denni•5/31/24, 12:49 PM

The wrangler.toml is in the same directory as where I'm running the wrangler commands from

Matt Silverlock•5/31/24, 12:51 PM

There's no caching (concretely, absolutely, in no way is there result caching) , so I'm curious as to why this is showing different data.

Matt Silverlock•5/31/24, 12:51 PM

Are the row counts the same?

Denni•5/31/24, 12:51 PM

Yeah so am I, anything I can try here to make sure the right things are being queried? Some verbose output of the queries

Max (@rozenmd)•5/31/24, 12:52 PM

can you try run that query in the console tab?

MMatt Silverlock Are the row counts the same?

Denni•5/31/24, 12:52 PM

They are not

MMax (@rozenmd)can you try run that query in the console tab?

Denni•5/31/24, 12:53 PM

I'm sorry, the console tab? My browser console?

DDenni I'm sorry, the console tab? My browser console?

Matt Silverlock•5/31/24, 12:53 PM

In the D1 dashboard [from your screenshot]

Matt Silverlock•5/31/24, 12:53 PM

DDenni I'm sorry, the console tab? My browser console?

Max (@rozenmd)•5/31/24, 12:53 PM

Denni•5/31/24, 12:54 PM

Denni•5/31/24, 12:54 PM

This is the wrong count again

Denni•5/31/24, 12:55 PM

OH I think I get it, I'm querying using WHERE pack_id=... and the new results are numbers(?) so they are not included

Denni•5/31/24, 12:55 PM

If I remove the WHERE the counts do match

Denni•5/31/24, 12:55 PM

But cloudflare shows both data types as the sme even though they are different somehow

Max (@rozenmd)•5/31/24, 12:57 PM

ah that's the trouble with sqlite, even if you specify types, you can insert anything you want

MMax (@rozenmd)ah that's the trouble with sqlite, even if you specify types, you can insert any...

Denni•5/31/24, 1:01 PM

But now I'm hella confused why it is returning the rows where pack_id is a string with pack_id=371151432834875400pack_id=371151432834875400 but not where pack_id is a number with the same where clause

Denni•5/31/24, 1:01 PM

:denkpoes_animated:

Denni•5/31/24, 1:01 PM

I'm guessing because the number is larger than max_safe_int?

Denni•5/31/24, 1:05 PM

It's actually that value

hey! am i remembering correctly that D1 is internally implemented on top of Durable Objects?

Similar Threads

Similar Threads

Similar Threads