Hey everyone, when a D1 is created and the replicas distributed through the network. I wanna know if

Hey everyone, when a D1 is created and the replicas distributed through the network. I wanna know if that D1 is always up and running or if it only runs when a request has been made to it?

CiabbataConda•7/27/23, 6:27 PM

if I were to answer by intuition, given that D1 is based on SQLite it doesn't actually "run" anywhere – it's just a file on each node of the network. "Replication" of D1 is basically copying the database file across nodes, just as the static assets of a website would be. For the database to "run", I assume that a Worker reads the database file on request, if necessary, and works with that.
In essence, you could say that D1 "runs" only when a request is made. But it really never runs, as it is not a server. Pricing also reflects that (https://developers.cloudflare.com/d1/platform/pricing/): notice that the metrics by which usage is priced are reads, writes, and storage (which are characteristic to file usage), but not something like CPU time (which is characteristic to a server).

CiabbataConda•7/27/23, 6:28 PM

If somebody spots a mistake please correct me.

NazCodelandOP•7/27/23, 6:29 PM

Perfect that makes sense and thank you for all the other tid bits of information.

NNazCodeland Hey everyone, when a D1 is created and the replicas distributed through the netw...

Hello, I’m Allie!•7/27/23, 6:30 PM

D1 is not currently replicated, and iirc they hibernate after around 30 seconds of inactivity

NazCodelandOP•7/27/23, 6:33 PM

So, a typical PostgreSQL requires a server to be up and running at all times to serve records. But since D1 is built on SQLite - from what I understand, SQLite is serverless and doesn't require a server - it's just a file like CiabbataConda said.

just learning all this

NazCodelandOP•7/27/23, 6:34 PM

I intended to use PostgreSQL for the project I started building and now I am wondering if I can use D1 instead

NNazCodeland So, a typical PostgreSQL requires a server to be up and running at all times to ...

Hello, I’m Allie!•7/27/23, 6:50 PM

Sort of. Standard SQLite is just a file plus some code to read it. But how D1 works is that it is some code that Cloudflare runs for you, on top of that file. So yes, that service may need to be cold-booted occasionally if it hasn’t been called in a while, but as Skye pointed out, bootups occur in the order of a few milliseconds, so it shouldn’t be anything you need to factor into your plans

CiabbataConda•7/27/23, 6:51 PM

where do you get this info from? is there some documentation on the internals which I am incapable of finding?

CiabbataConda•7/27/23, 6:51 PM

i'd like to read more about this technology but god I can't find anything on it

CCiabbataConda where do you get this info from? is there some documentation on the internals wh...

Hello, I’m Allie!•7/27/23, 6:53 PM

Check out the blog: https://blog.cloudflare.com/introducing-d1/

The Cloudflare Blog

Announcing D1: our first SQL database

Today, we’re excited to announce D1, Cloudflare’s first SQL database, designed for Cloudflare Workers

NazCodelandOP•7/27/23, 6:53 PM

I'm gonna google more about the diff. between PostgreSQL and SQLite to understand more

Hello, I’m Allie!•7/27/23, 6:53 PM

And more on the beta improvements: https://blog.cloudflare.com/d1-turning-it-up-to-11/

The Cloudflare Blog

D1: We turned it up to 11

We've been heads down iterating on D1, and we've just shipped a major new version that's substantially faster, more reliable, and introduces Time Travel: the ability to restore a D1 database to any point in time.

CiabbataConda•7/27/23, 6:54 PM

of course they're blog posts...

CiabbataConda•7/27/23, 6:54 PM

thank you

NazCodelandOP•7/27/23, 7:37 PM

I'm not sure I even understand the whole complexity involved in a business like DoorDash but if someone were to build an application with DoorDash's business model, I don't know how many but say they needed 10-15-20 tables with 7-10 columns... would D1 be able to handle that?

CiabbataConda•7/27/23, 7:39 PM

this is built with D1: https://northwind.d1sql.com/. here is the database: https://github.com/jpwhite3/northwind-SQLite3. you can find there a graph of its schema.

Matt Silverlock•7/27/23, 7:42 PM

Time Travel is now available in the latest release (): https://developers.cloudflare.com/d1/learning/time-travel/

Time Travel and backups · Cloudflare D1 docs

Time Travel is D1’s approach to backups and point-in-time-recovery, and allows you to restore a database to any minute within the last 30 days.

MMatt Silverlock Time Travel is now available in the latest `wrangler` release (`3.4.0`): https:/...

Matt Silverlock•7/27/23, 7:43 PM

Changelog: https://developers.cloudflare.com/d1/changelog/#2023-07-27

NNazCodeland I'm not sure I even understand the whole complexity involved in a business like ...

CiabbataConda•7/27/23, 7:43 PM

this is rather a question of whether SQLite can handle scale. see https://www.sqlite.org/whentouse.html, https://news.ycombinator.com/item?id=33975635.

NNazCodeland I'm not sure I even understand the whole complexity involved in a business like ...

Matt Silverlock•7/27/23, 7:48 PM

That is not a lot of tables. Schema complexity isn't really a problem. Making your queries efficient - D1 or not - is what is important.

NazCodelandOP•7/27/23, 7:48 PM

That schema png is exactly what I had in mind. I'll check those links out, thanks.

NazCodelandOP•7/27/23, 7:49 PM

I'll keep that in mind

NazCodelandOP•7/27/23, 8:24 PM

"To use SpatiaLite with SQLite, you need to install the SpatiaLite library and load it as an extension to SQLite."

Does D1 currently have the ability to add a library as an extension?

Braden•7/28/23, 12:18 AM

Are databases created via the dashboard on the experimental backend? Or is that just an option via the CLI? And how can you tell which backend you're using?

BBraden Are databases created via the dashboard on the experimental backend? Or is that ...

Chaika•7/28/23, 12:23 AM

It used to be an option when creating to enable experimental, but it looks like it was changed so that they're all experimental now. A really easy way to tell is if the dashboard has "Backups" or "Time Travel" option inside of the database. If it's time travel, it's running experimental.
Also if it has the metrics tab, only experimental has that.

BBraden Are databases created via the dashboard on the experimental backend? Or is that ...

Matt Silverlock•7/28/23, 12:39 AM

Braden•7/28/23, 1:12 AM

Thanks!

TigersWay•7/28/23, 2:09 AM

Is there any way to share/access a D1 Database from multiple CF accounts, from different workers or pages from different accounts?

TTigersWay Is there any way to share/access a D1 Database from multiple CF accounts, from d...

Cyb3r-Jak3•7/28/23, 2:28 AM

There is not. Resources like KV, R2, D1 are all scoped by account so you can only share between workers of the same account

CCyb3r-Jak3 There is not. Resources like KV, R2, D1 are all scoped by account so you can onl...

TigersWay•7/28/23, 2:30 AM

Confirmation of what I was afraid... Thank you for the answer

micromashor•7/28/23, 4:49 AM

D1 Dashboard seems to have trouble with BLOB columns, it appears to take the column's representation as a Uint8Array and concatenate all of the decimal values therein, resulting in this to represent a column containing :

(please forgive my horrendous screenshot skills)

micromashor•7/28/23, 4:53 AM

the native utility isn't any better (it just obliterates your terminal with binary data), so I don't have much of a suggestion, other than the behavior should probably have a length limit. Perhaps anything 32 bytes and fewer could be shown in a hex representation, and anything bigger should just show a placeholder

Mmicromashor D1 Dashboard seems to have trouble with BLOB columns, it appears to take the col...

Max (@rozenmd)•7/28/23, 5:54 AM

yeah tbh it's a long standing bug, I wasn't really sure how folks would want binary data represented

micromashor•7/28/23, 5:54 AM

yeah it's tricky

micromashor•7/28/23, 5:55 AM

maybe a placeholder, and when you click on it, it brings up a hexdump?

itsmatteomanf•7/28/23, 10:17 AM

I am seeing two error messages for what seems to be same error, any specific reason for the difference?

itsmatteomanf•7/28/23, 10:17 AM

itsmatteomanf•7/28/23, 10:17 AM

It's most likely 2 different tables being referenced here, but would expect the same error message.

Iitsmatteomanf Click to see attachment

Matt Silverlock•7/28/23, 1:11 PM

Do you mean the duplicates (each of) "Foreign key mismatch - X referencing Y" twice, and "FOREIGN KEY constraint failed" twice?

Rre-thc Are we still limited at max size 100mb?

Matt Silverlock•7/28/23, 1:12 PM

Yes, for now. soontm

MMatt Silverlock Do you mean the duplicates (each of) "Foreign key mismatch - X referencing Y" tw...

itsmatteomanf•7/28/23, 1:17 PM

Wait, your message made me read those errors again… they are not the same thing. I don’t know why I read the one on top as another way to write a constraint violation. But it’s not.

itsmatteomanf•7/28/23, 1:17 PM

Not exactly sure what changed, it worked perfectly until a couple days back…

Iitsmatteomanf Not exactly sure what changed, it worked perfectly until a couple days back…

Matt Silverlock•7/28/23, 1:37 PM

Foreign keys are now enforced. Docs soon.

Iitsmatteomanf Not exactly sure what changed, it worked perfectly until a couple days back…

Matt Silverlock•7/28/23, 1:37 PM

d1-database

MMatt Silverlock Foreign keys are now enforced. Docs soon.

itsmatteomanf•7/28/23, 1:39 PM

Ohhh, so I had some violations somewhere, apparently, which I didn't realize. That's gonna be fun lul

Iitsmatteomanf Ohhh, so I had some violations somewhere, apparently, which I didn't realize. Th...

Matt Silverlock•7/28/23, 1:39 PM

WIP docs: https://github.com/cloudflare/cloudflare-docs/pull/9984

itsmatteomanf•7/28/23, 1:40 PM

I'm gonna need to do some countings and checking

itsmatteomanf•7/28/23, 2:34 PM

Can we get the analytics stats on the top-right of the D1 page to be formatted as the others are? I get those are precise, but if I 1M queries, I don't really care abot single digit changes lul

Colin•7/28/23, 3:46 PM

hey y'all - we're experiencing intermittent 's in our Workers hitting D1.

New storage backend
Seems to be intermittent + random, there's no one query w/ 100% failure rate or anything - just scattered.
This is a higher-volume (I believe, compared to typical workloads) D1 DB, serving >50 queries/s

Does anyone have any idea if this is a D1 issue/an issue with our queries/DB, and if there's anything we can do? We're 100% aware D1 is still open alpha, just trying to get our success rate as close to 100% as possible

Hey everyone, when a D1 is created and the replicas distributed through the network. I wanna know if

Similar Threads

Similar Threads

Similar Threads