D1 Dashboard seems to have trouble with BLOB columns, it appears to take the column's representation

D1 Dashboard seems to have trouble with BLOB columns, it appears to take the column's representation as a Uint8Array and concatenate all of the decimal values therein, resulting in this to represent a column containing

[248, 15, 244, 44, 190, 134, 199, 60, 153, 194, 10, 101, 112, 55, 185, 241]

[248, 15, 244, 44, 190, 134, 199, 60, 153, 194, 10, 101, 112, 55, 185, 241]

:

(please forgive my horrendous screenshot skills)

micromashorOP•7/28/23, 4:53 AM

the native

sqlite3

sqlite3

utility isn't any better (it just obliterates your terminal with binary data), so I don't have much of a suggestion, other than the behavior should probably have a length limit. Perhaps anything 32 bytes and fewer could be shown in a hex representation, and anything bigger should just show a

<Binary Data>

<Binary Data>

placeholder

Mmicromashor D1 Dashboard seems to have trouble with BLOB columns, it appears to take the col...

Max (@rozenmd)•7/28/23, 5:54 AM

yeah tbh it's a long standing bug, I wasn't really sure how folks would want binary data represented

micromashorOP•7/28/23, 5:54 AM

yeah it's tricky

micromashorOP•7/28/23, 5:55 AM

maybe a placeholder, and when you click on it, it brings up a hexdump?

itsmatteomanf•7/28/23, 10:17 AM

I am seeing two error messages for what seems to be same error, any specific reason for the difference?

itsmatteomanf•7/28/23, 10:17 AM

itsmatteomanf•7/28/23, 10:17 AM

It's most likely 2 different tables being referenced here, but would expect the same error message.

Iitsmatteomanf Click to see attachment

Matt Silverlock•7/28/23, 1:11 PM

Do you mean the duplicates (each of) "Foreign key mismatch - X referencing Y" twice, and "FOREIGN KEY constraint failed" twice?

Rre-thc Are we still limited at max size 100mb?

Matt Silverlock•7/28/23, 1:12 PM

Yes, for now. soontm

MMatt Silverlock Do you mean the duplicates (each of) "Foreign key mismatch - X referencing Y" tw...

itsmatteomanf•7/28/23, 1:17 PM

Wait, your message made me read those errors again… they are not the same thing. I don’t know why I read the one on top as another way to write a constraint violation. But it’s not.

itsmatteomanf•7/28/23, 1:17 PM

Not exactly sure what changed, it worked perfectly until a couple days back…

Iitsmatteomanf Not exactly sure what changed, it worked perfectly until a couple days back…

Matt Silverlock•7/28/23, 1:37 PM

Foreign keys are now enforced. Docs soon.

Iitsmatteomanf Not exactly sure what changed, it worked perfectly until a couple days back…

Matt Silverlock•7/28/23, 1:37 PM

d1-database

MMatt Silverlock Foreign keys are now enforced. Docs soon.

itsmatteomanf•7/28/23, 1:39 PM

Ohhh, so I had some violations somewhere, apparently, which I didn't realize. That's gonna be fun lul

Iitsmatteomanf Ohhh, so I had some violations somewhere, apparently, which I didn't realize. Th...

Matt Silverlock•7/28/23, 1:39 PM

WIP docs: https://github.com/cloudflare/cloudflare-docs/pull/9984

itsmatteomanf•7/28/23, 1:40 PM

I'm gonna need to do some countings and checking

itsmatteomanf•7/28/23, 2:34 PM

Can we get the analytics stats on the top-right of the D1 page to be formatted as the others are? I get those are precise, but if I 1M queries, I don't really care abot single digit changes lul

Colin•7/28/23, 3:46 PM

hey y'all - we're experiencing intermittent

D1_ERROR: Network connection lost

D1_ERROR: Network connection lost

's in our Workers hitting D1.

New storage backend
Seems to be intermittent + random, there's no one query w/ 100% failure rate or anything - just scattered.
This is a higher-volume (I believe, compared to typical workloads) D1 DB, serving >50 queries/s

Does anyone have any idea if this is a D1 issue/an issue with our queries/DB, and if there's anything we can do? We're 100% aware D1 is still open alpha, just trying to get our success rate as close to 100% as possible

Colin•7/28/23, 3:47 PM

originally submitted a support ticket but was redirected here to chat with y'all

CColin hey y'all - we're experiencing intermittent `D1_ERROR: Network connection lost`'...

Matt Silverlock•7/28/23, 3:53 PM

What is the result set size / how many rows x bytes wide are you trying to return? Do you a database ID?

Colin•7/28/23, 3:55 PM

Sure, e707d84e-7dd9-4c4b-940a-63ea75a981fb is the ID. I’m on my phone right now so can’t be precise w/ result set size, but it’s definitely no more than 100 rows each being at most 512 bytes

CColin Sure, e707d84e-7dd9-4c4b-940a-63ea75a981fb is the ID. I’m on my phone right now ...

Matt Silverlock•7/28/23, 4:34 PM

Got it. Account ID would be great too. I don't see any immediate logs for

e707d84e-7dd9-4c4b-940a-63ea75a981fb

e707d84e-7dd9-4c4b-940a-63ea75a981fb

MMatt Silverlock Got it. Account ID would be great too. I don't see any immediate logs for `e707d...

Colin•7/28/23, 4:39 PM

Sure.
Account ID:

cfd355bcff7e6af6838743ae944184b0

cfd355bcff7e6af6838743ae944184b0

DB ID:

e707d84e-7dd9-4c4b-940a-63ea75a981fb

e707d84e-7dd9-4c4b-940a-63ea75a981fb

And some extras: Ray ID of a failed request:

7ecd59bfdc728ec8

7ecd59bfdc728ec8

, timestamps are

1690381931502

1690381931502

(request) and

1690381936556

1690381936556

(D1 error)

Colin•7/28/23, 4:40 PM

Happy to add you on Discord + provide exact queries/source code if it'd help too

benjamin•7/28/23, 8:48 PM

Referencing this blog post (https://blog.cloudflare.com/whats-new-with-d1/), wondering about the plans for implementing procedures. I saw some references that this isn't in development/available (https://discord.com/channels/595317990191398933/789155108529111069/1104131881194246334).

Discord

Discord - A New Way to Chat with Friends & Communities

Discord is the easiest way to communicate over voice, video, and text. Chat, hang out, and stay close with your friends and communities.

codebam•7/29/23, 2:52 AM

does d1 support foreign keys?

Ccodebam does d1 support foreign keys?

Cyb3r-Jak3•7/29/23, 2:54 AM

Yeah it does. See WIP docs d1-database

CCyb3r-Jak3 Yeah it does. See WIP docs https://discord.com/channels/595317990191398933/99206...

codebam•7/29/23, 2:55 AM

thank you

codebam•7/29/23, 2:56 AM

what about auto increment? is that supported or do I just have to use random uuids?

Hyper•7/29/23, 3:51 AM

Hey guys. Im wondering when transactions will be supported on D1. Would love to be able to simplify database integrity maintenance

Ccodebam what about auto increment? is that supported or do I just have to use random uui...

Matt Silverlock•7/29/23, 11:43 AM

By default there is an automatic rowid column. You can also specify your own

col_name INT PRIMARY KEY

col_name INT PRIMARY KEY

which will also be auto incrementing. We haven’t yet published docs on this, but this applies: https://www.sqlite.org/rowidtable.html

HHyper Hey guys. Im wondering when transactions will be supported on D1. Would love to ...

Matt Silverlock•7/29/23, 11:43 AM

Do you mean JS transactions? We don’t have an ETA we can share.

MMatt Silverlock Do you mean JS transactions? We don’t have an ETA we can share.

Hyper•7/29/23, 12:07 PM

I’m a little confused. I thought transactions were not supported at all. Does this mean I can write transactions in SQL and they would work fine? For example if i’m using an orm that generates SQL of the transaction and executes it against D1, will that work normally?

HHyper I’m a little confused. I thought transactions were not supported at all. Does th...

Matt Silverlock•7/29/23, 1:10 PM

All queries are in an implicit transaction for D1.

HHyper I’m a little confused. I thought transactions were not supported at all. Does th...

Matt Silverlock•7/29/23, 1:11 PM

Some folks want a separate, callback-driven JS API that allows you to combine SQL + JS —> that part is further out.

Hyper•7/29/23, 6:29 PM

Gotcha