I'm not sure I even understand the whole complexity involved in a business like DoorDash but if some

I'm not sure I even understand the whole complexity involved in a business like DoorDash but if someone were to build an application with DoorDash's business model, I don't know how many but say they needed 10-15-20 tables with 7-10 columns... would D1 be able to handle that?

CiabbataConda•7/27/23, 7:39 PM

this is built with D1: https://northwind.d1sql.com/. here is the database: https://github.com/jpwhite3/northwind-SQLite3. you can find there a graph of its schema.

Matt Silverlock•7/27/23, 7:42 PM

Time Travel is now available in the latest

wrangler

wrangler

release (3.4.03.4.0): https://developers.cloudflare.com/d1/learning/time-travel/

Time Travel and backups · Cloudflare D1 docs

Time Travel is D1’s approach to backups and point-in-time-recovery, and allows you to restore a database to any minute within the last 30 days.

MMatt Silverlock Time Travel is now available in the latest `wrangler` release (`3.4.0`): https:/...

Matt Silverlock•7/27/23, 7:43 PM

Changelog: https://developers.cloudflare.com/d1/changelog/#2023-07-27

NNazCodeland I'm not sure I even understand the whole complexity involved in a business like ...

CiabbataConda•7/27/23, 7:43 PM

this is rather a question of whether SQLite can handle scale. see https://www.sqlite.org/whentouse.html, https://news.ycombinator.com/item?id=33975635.

NNazCodeland I'm not sure I even understand the whole complexity involved in a business like ...

Matt Silverlock•7/27/23, 7:48 PM

That is not a lot of tables. Schema complexity isn't really a problem. Making your queries efficient - D1 or not - is what is important.

NazCodelandOP•7/27/23, 7:48 PM

That schema png is exactly what I had in mind. I'll check those links out, thanks.

NazCodelandOP•7/27/23, 7:49 PM

I'll keep that in mind

NazCodelandOP•7/27/23, 8:24 PM

"To use SpatiaLite with SQLite, you need to install the SpatiaLite library and load it as an extension to SQLite."

Does D1 currently have the ability to add a library as an extension?

Braden•7/28/23, 12:18 AM

Are databases created via the dashboard on the experimental backend? Or is that just an option via the CLI? And how can you tell which backend you're using?

BBraden Are databases created via the dashboard on the experimental backend? Or is that ...

Chaika•7/28/23, 12:23 AM

It used to be an option when creating to enable experimental, but it looks like it was changed so that they're all experimental now. A really easy way to tell is if the dashboard has "Backups" or "Time Travel" option inside of the database. If it's time travel, it's running experimental.
Also if it has the metrics tab, only experimental has that.

BBraden Are databases created via the dashboard on the experimental backend? Or is that ...

Matt Silverlock•7/28/23, 12:39 AM

Braden•7/28/23, 1:12 AM

Thanks!

TigersWay•7/28/23, 2:09 AM

Is there any way to share/access a D1 Database from multiple CF accounts, from different workers or pages from different accounts?

TTigersWay Is there any way to share/access a D1 Database from multiple CF accounts, from d...

Cyb3r-Jak3•7/28/23, 2:28 AM

There is not. Resources like KV, R2, D1 are all scoped by account so you can only share between workers of the same account

CCyb3r-Jak3 There is not. Resources like KV, R2, D1 are all scoped by account so you can onl...

TigersWay•7/28/23, 2:30 AM

Confirmation of what I was afraid... Thank you for the answer

micromashor•7/28/23, 4:49 AM

D1 Dashboard seems to have trouble with BLOB columns, it appears to take the column's representation as a Uint8Array and concatenate all of the decimal values therein, resulting in this to represent a column containing [248, 15, 244, 44, 190, 134, 199, 60, 153, 194, 10, 101, 112, 55, 185, 241][248, 15, 244, 44, 190, 134, 199, 60, 153, 194, 10, 101, 112, 55, 185, 241]:

(please forgive my horrendous screenshot skills)

micromashor•7/28/23, 4:53 AM

the native sqlite3sqlite3 utility isn't any better (it just obliterates your terminal with binary data), so I don't have much of a suggestion, other than the behavior should probably have a length limit. Perhaps anything 32 bytes and fewer could be shown in a hex representation, and anything bigger should just show a <Binary Data><Binary Data> placeholder

Mmicromashor D1 Dashboard seems to have trouble with BLOB columns, it appears to take the col...

Max (@rozenmd)•7/28/23, 5:54 AM

yeah tbh it's a long standing bug, I wasn't really sure how folks would want binary data represented

micromashor•7/28/23, 5:54 AM

yeah it's tricky

micromashor•7/28/23, 5:55 AM

maybe a placeholder, and when you click on it, it brings up a hexdump?

itsmatteomanf•7/28/23, 10:17 AM

I am seeing two error messages for what seems to be same error, any specific reason for the difference?

itsmatteomanf•7/28/23, 10:17 AM

itsmatteomanf•7/28/23, 10:17 AM

It's most likely 2 different tables being referenced here, but would expect the same error message.

Iitsmatteomanf Click to see attachment

Matt Silverlock•7/28/23, 1:11 PM

Do you mean the duplicates (each of) "Foreign key mismatch - X referencing Y" twice, and "FOREIGN KEY constraint failed" twice?

Rre-thc Are we still limited at max size 100mb?

Matt Silverlock•7/28/23, 1:12 PM

Yes, for now. soontm

MMatt Silverlock Do you mean the duplicates (each of) "Foreign key mismatch - X referencing Y" tw...

itsmatteomanf•7/28/23, 1:17 PM

Wait, your message made me read those errors again… they are not the same thing. I don’t know why I read the one on top as another way to write a constraint violation. But it’s not.

itsmatteomanf•7/28/23, 1:17 PM

Not exactly sure what changed, it worked perfectly until a couple days back…

Iitsmatteomanf Not exactly sure what changed, it worked perfectly until a couple days back…

Matt Silverlock•7/28/23, 1:37 PM

Foreign keys are now enforced. Docs soon.

Iitsmatteomanf Not exactly sure what changed, it worked perfectly until a couple days back…

Matt Silverlock•7/28/23, 1:37 PM

d1-database

MMatt Silverlock Foreign keys are now enforced. Docs soon.

itsmatteomanf•7/28/23, 1:39 PM

Ohhh, so I had some violations somewhere, apparently, which I didn't realize. That's gonna be fun lul

Iitsmatteomanf Ohhh, so I had some violations somewhere, apparently, which I didn't realize. Th...

Matt Silverlock•7/28/23, 1:39 PM

WIP docs: https://github.com/cloudflare/cloudflare-docs/pull/9984

itsmatteomanf•7/28/23, 1:40 PM

I'm gonna need to do some countings and checking

itsmatteomanf•7/28/23, 2:34 PM

Can we get the analytics stats on the top-right of the D1 page to be formatted as the others are? I get those are precise, but if I 1M queries, I don't really care abot single digit changes lul

Colin•7/28/23, 3:46 PM

hey y'all - we're experiencing intermittent D1_ERROR: Network connection lostD1_ERROR: Network connection lost's in our Workers hitting D1.

New storage backend
Seems to be intermittent + random, there's no one query w/ 100% failure rate or anything - just scattered.
This is a higher-volume (I believe, compared to typical workloads) D1 DB, serving >50 queries/s

Does anyone have any idea if this is a D1 issue/an issue with our queries/DB, and if there's anything we can do? We're 100% aware D1 is still open alpha, just trying to get our success rate as close to 100% as possible

Colin•7/28/23, 3:47 PM

originally submitted a support ticket but was redirected here to chat with y'all

CColin hey y'all - we're experiencing intermittent `D1_ERROR: Network connection lost`'...

Matt Silverlock•7/28/23, 3:53 PM

What is the result set size / how many rows x bytes wide are you trying to return? Do you a database ID?

Colin•7/28/23, 3:55 PM

Sure, e707d84e-7dd9-4c4b-940a-63ea75a981fb is the ID. I’m on my phone right now so can’t be precise w/ result set size, but it’s definitely no more than 100 rows each being at most 512 bytes

CColin Sure, e707d84e-7dd9-4c4b-940a-63ea75a981fb is the ID. I’m on my phone right now ...

Matt Silverlock•7/28/23, 4:34 PM

Got it. Account ID would be great too. I don't see any immediate logs for e707d84e-7dd9-4c4b-940a-63ea75a981fbe707d84e-7dd9-4c4b-940a-63ea75a981fb

MMatt Silverlock Got it. Account ID would be great too. I don't see any immediate logs for `e707d...

Colin•7/28/23, 4:39 PM

Sure.
Account ID: cfd355bcff7e6af6838743ae944184b0cfd355bcff7e6af6838743ae944184b0
DB ID: e707d84e-7dd9-4c4b-940a-63ea75a981fbe707d84e-7dd9-4c4b-940a-63ea75a981fb

And some extras: Ray ID of a failed request: 7ecd59bfdc728ec87ecd59bfdc728ec8, timestamps are 16903819315021690381931502 (request) and 16903819365561690381936556 (D1 error)

Colin•7/28/23, 4:40 PM

Happy to add you on Discord + provide exact queries/source code if it'd help too

benjamin•7/28/23, 8:48 PM

Referencing this blog post (https://blog.cloudflare.com/whats-new-with-d1/), wondering about the plans for implementing procedures. I saw some references that this isn't in development/available (https://discord.com/channels/595317990191398933/789155108529111069/1104131881194246334).

Discord

Discord - A New Way to Chat with Friends & Communities

Discord is the easiest way to communicate over voice, video, and text. Chat, hang out, and stay close with your friends and communities.

codebam•7/29/23, 2:52 AM

does d1 support foreign keys?

Ccodebam does d1 support foreign keys?

Cyb3r-Jak3•7/29/23, 2:54 AM

Yeah it does. See WIP docs d1-database

CCyb3r-Jak3 Yeah it does. See WIP docs https://discord.com/channels/595317990191398933/99206...

codebam•7/29/23, 2:55 AM

thank you

codebam•7/29/23, 2:56 AM

what about auto increment? is that supported or do I just have to use random uuids?

Hyper•7/29/23, 3:51 AM

Hey guys. Im wondering when transactions will be supported on D1. Would love to be able to simplify database integrity maintenance

Ccodebam what about auto increment? is that supported or do I just have to use random uui...

Matt Silverlock•7/29/23, 11:43 AM

By default there is an automatic rowid column. You can also specify your own col_name INT PRIMARY KEYcol_name INT PRIMARY KEY which will also be auto incrementing. We haven’t yet published docs on this, but this applies: https://www.sqlite.org/rowidtable.html

HHyper Hey guys. Im wondering when transactions will be supported on D1. Would love to ...

Matt Silverlock•7/29/23, 11:43 AM

Do you mean JS transactions? We don’t have an ETA we can share.

MMatt Silverlock Do you mean JS transactions? We don’t have an ETA we can share.

Hyper•7/29/23, 12:07 PM

I’m a little confused. I thought transactions were not supported at all. Does this mean I can write transactions in SQL and they would work fine? For example if i’m using an orm that generates SQL of the transaction and executes it against D1, will that work normally?

I'm not sure I even understand the whole complexity involved in a business like DoorDash but if some

Similar Threads