It is currently not possible to download D1 databases that are using the new backend (=that have tim

It is currently not possible to download D1 databases that are using the new backend (=that have time travel). There is a community project that lets you do that though

zegevlierOP•10/19/23, 3:06 PM

I haven't used it myself, but here's a link: http://github.com/nora-soderlund/cloudflare-d1-backups

GitHub

GitHub - nora-soderlund/cloudflare-d1-backups: A package to dump a ...

A package to dump a copy of your experimental D1 database through a worker. - GitHub - nora-soderlund/cloudflare-d1-backups: A package to dump a copy of your experimental D1 database through a worker.

Zzegevlier It is currently not possible to download D1 databases that are using the new bac...

schwartzmj•10/19/23, 3:07 PM

alright thanks... any idea if its planned to be able to download your database at some time in the future? like, confirmed

zegevlierOP•10/19/23, 3:26 PM

I have not seen any confirmation about it, and a quick search also did not show any

OOld Man Maker Thanks for that information. I'll do some tests around this. As @Michał z typeof...

Matt Silverlock•10/19/23, 3:30 PM

Right now you can bind up to ~5000 databases per Worker (conservatively), so I suspect this isn't an immediate problem just yet / before we have a better solution.

MMatt Silverlock Right now you can bind up to ~5000 databases per Worker (conservatively), so I s...

Old Man Maker•10/19/23, 3:34 PM

Thanks Matt. Given that the binding is added, can I access it with a key?
E.g const result = env[binding_name].prepare(,,,,)?

Old Man Maker•10/19/23, 3:35 PM

The binding name will be a parameter passed into the worker

Old Man Maker•10/19/23, 4:00 PM

I'm actually thinking I might be going about this all wrong for this particular use case - in D1 we've got a hammer, so everything looks like a nail.
In this instance, I think I may be better off using KV as the storage mechanism
This has been a really interesting exercise in considerations and thanks to everyone who has put thoughts, ideas and solutions into it. Much appreciated

OOld Man Maker Thanks Matt. Given that the binding is added, can I access it with a key? E.g c...

Matt Silverlock•10/19/23, 4:07 PM

We're still designing the interface, but effectively being able to address databases/resources by name/ID at runtime.

itsmatteomanf•10/19/23, 4:38 PM

@Matt my D1 db seems to have vanished...

546787dd-35d5-4822-9b50-36fa0a4310c2

546787dd-35d5-4822-9b50-36fa0a4310c2

Iitsmatteomanf @Matt my D1 db seems to have vanished... `546787dd-35d5-4822-9b50-36fa0a4310c2` ...

Matt Silverlock•10/19/23, 4:44 PM

Account ID please

Iitsmatteomanf @Matt my D1 db seems to have vanished... `546787dd-35d5-4822-9b50-36fa0a4310c2` ...

Matt Silverlock•10/19/23, 4:45 PM

And does vanished mean: "it doesn't respond to queries", "it doesn't show up in the API", or something else?

MMatt Silverlock Account ID please

itsmatteomanf•10/19/23, 4:45 PM

275fcefacfae5b20fe129cbac8269aa4

275fcefacfae5b20fe129cbac8269aa4

MMatt Silverlock And does vanished mean: "it doesn't respond to queries", "it doesn't show up in ...

itsmatteomanf•10/19/23, 4:46 PM

Failed to reach database. Please try again later.

itsmatteomanf•10/19/23, 4:46 PM

Both UI and Workers (which returns

D1_ERROR: internal error

D1_ERROR: internal error

)

Iitsmatteomanf Both UI and Workers (which returns `D1_ERROR: internal error`)

Matt Silverlock•10/19/23, 4:47 PM

OK, so not "disappeared" - somehow stuck (still bad, but very different concern). Let us take a look.

MMatt Silverlock OK, so not "disappeared" - somehow stuck (still bad, but very different concern)...

itsmatteomanf•10/19/23, 4:48 PM

Thank you... it was growing in size, but shouldn't have hit the 2GB limit, at all. It seem to have started failing yesterday, like ~30 hrs ago.

Iitsmatteomanf Thank you... it was growing in size, but shouldn't have hit the 2GB limit, at al...

Matt Silverlock•10/19/23, 4:49 PM

Last I can see is 789MB

MMatt Silverlock Last I can see is 789MB

itsmatteomanf•10/19/23, 4:49 PM

Yeah, aligns with what I expected, yes.

Matt Silverlock•10/19/23, 4:49 PM

OK. Looking into why.

itsmatteomanf•10/19/23, 4:49 PM

Mostly in one single table.

itsmatteomanf•10/19/23, 4:49 PM

Thank you

niconiahi•10/19/23, 6:43 PM

any recommendation on how to visualize my d1 database locally? in preference something completely open source and simplistic. Thanks!

Iitsmatteomanf Mostly in one single table.

Matt Silverlock•10/19/23, 8:26 PM

Should be fixed now.

MMatt Silverlock Should be fixed now.

itsmatteomanf•10/19/23, 8:31 PM

Looks like it is! Thank you MeowHeartCloudflare

itsmatteomanf•10/19/23, 8:32 PM

Any insights on what happened, that you are willing to share?

Iitsmatteomanf Any insights on what happened, that you are willing to share? 🙂

Matt Silverlock•10/19/23, 8:33 PM

Not yet. We restored and are still working on the root cause.

SnowyYANG•10/19/23, 10:08 PM

d1 supports how many data type? when I create the table there is even no varchar, only text.

SSnowyYANG d1 supports how many data type? when I create the table there is even no varchar...

Cyb3r-Jak3•10/19/23, 10:11 PM

It should be all sqlite data types which are NULL, INTEGER, READ, TEXT and BLOB

SnowyYANG•10/19/23, 10:12 PM

so d1 is sqlite?

Cyb3r-Jak3•10/19/23, 10:12 PM

Yes

SnowyYANG•10/19/23, 10:12 PM

thanks

SnowyYANG•10/19/23, 10:17 PM

I have a large complex MySQL application (like erp), is it smart to migrate to D1?

Cyb3r-Jak3•10/19/23, 10:27 PM

How are you connecting to the MySQL DB? There isn't a connection string that you can use to connect to a D1 DB externally, done through a worker or HTTP

SnowyYANG•10/19/23, 11:07 PM

I originally plan to use PHP+MySQL. If I migrate MySQL to D1 then I have to migrate PHP to Worker.

SnowyYANG•10/20/23, 4:13 AM

tested. D1 performance is not good.

SSnowyYANG tested. D1 performance is not good.

Hello, I’m Allie!•10/20/23, 6:33 AM

Are you using the API? That is usually much slower

HHello, I’m Allie!Are you using the API? That is usually much slower

SnowyYANG•10/20/23, 6:51 AM

yes, I am using Page Functions to insert data into D1

SSnowyYANG yes, I am using Page Functions to insert data into D1

Hello, I’m Allie!•10/20/23, 7:00 AM

There, the performance also depends on how close/far away the DB is

SSnowyYANG tested. D1 performance is not good.

Santa DCON 🎄•10/20/23, 7:33 AM

You will never get faster than on-prem perf with a remote DB

Santa DCON 🎄•10/20/23, 7:35 AM

If DB perf is a requirement, you might want to look into on-prem. As far as I'm aware, D1 offers easy scalability (due to CF having 200 trillion servers, real number btw)

Rubi•10/20/23, 9:31 AM

hi ummm, I couldn't find current D1 limitation article? could anyone mind to share it

Rubi•10/20/23, 9:32 AM

oh, there is pinned message

Rubi•10/20/23, 9:32 AM

I see then, let me stay with MongoDB Atlas

itsmatteomanf•10/20/23, 10:03 AM

Is there a way to see per table sizes (and even index, possibly) in D1?

itsmatteomanf•10/20/23, 10:03 AM

I know

dbstat

dbstat

can exists, but doesn't look like it's there.

SSanta DCON 🎄You will never get faster than on-prem perf with a remote DB

Hello, I’m Allie!•10/20/23, 10:56 AM

I would note that D1 can(once replication goes live) theoretically be faster than on-prem if you still use Workers. As in, you would have to have/managed many DBs to be able to match the scale/locations of D1

HHello, I’m Allie!I would note that D1 can(once replication goes live) theoretically be faster tha...

Santa DCON 🎄•10/20/23, 10:57 AM

Ah yes, I was speaking from the perspective of him using php remote to D1

Samantas5855•10/20/23, 1:14 PM

We fully transitioned from mongodb to d1 yesterday and it's plenty fast! The python script that runs on our server updates the DB while our pages functions act as the client which return the results on our api. Considering that many requests happen every minute which receive the same output how could we cache the response for example one minute in order to not burn row reads?

SSamantas5855 We fully transitioned from mongodb to d1 yesterday and it's plenty fast! The pyt...

Cyb3r-Jak3•10/20/23, 1:27 PM

Use the cache API to cache responses. The function will still run but will reduce usage of D1

Jonathan•10/20/23, 4:26 PM

Hey all, I'm new here. I'm noticing really slow performance using D1 (cold and hot runs) in other countries like Germany and Australia. I'm getting ~1200ms on cold starts and ~700ms on hot. Is this normal for the beta?

It is currently not possible to download D1 databases that are using the new backend (=that have tim

Similar Threads

Similar Threads

Similar Threads