Is D1 sized fixed to maximum 2gb, or it can be extended on request? If not, we are going to design

Is D1 sized fixed to maximum 2gb, or it can be extended on request?

If not, we are going to design our application for multiple databases instead all tables in single database.

VVikash Is D1 sized fixed to maximum 2gb, or it can be extended on request? If not, we ...

itsmatteomanf•11/18/23, 1:42 PM

It's expected to grow, but no one, at least externally, knows by how much or when.

itsmatteomanf•11/18/23, 1:43 PM

If you can split it by database, I'd split it if possible.

itsmatteomanf•11/18/23, 1:43 PM

You'd get higher performance as each one is a separate instance.

VikashOP•11/18/23, 1:43 PM

I am planning my app, so I can for now.. The expected size of database is around 5gb

VikashOP•11/18/23, 1:49 PM

Also any suggestion/example for bulk import? I have around 500mb CSV file needs to be imported.

busterroni•11/18/23, 6:40 PM

I think there is a bug with deleting a table that is referenced by another table. You should probably get an error that the table could not be deleted, but instead you get a timeout after a bit.

busterroni•11/18/23, 7:04 PM

Also just got this when dropping a table

{'result': [], 'success': False, 'errors': [{'code': 7500, 'message': 'Internal error in D1 storage caused object to be reset.'}], 'messages': []}

{'result': [], 'success': False, 'errors': [{'code': 7500, 'message': 'Internal error in D1 storage caused object to be reset.'}], 'messages': []}

Anon•11/19/23, 7:32 PM

good, there will be a feature or a way to achieve pagination.

AAnon good, there will be a feature or a way to achieve pagination.

Cyb3r-Jak3•11/19/23, 11:30 PM

I mean you can already do pagination. You limit your query to X amount of results and for other pages you apply an offset

user•11/20/23, 5:59 AM

https://user.you-sk.id/plCW7Cg4qgki Is there any way to get more than 2gb?

uwu

owo

SSotashi Nokomata is there any way to restore a database without overwriting? do not see any opt...

Max (@rozenmd)•11/20/23, 8:55 AM

not yet, but we plan on adding fork/clone functionality

Rick•11/20/23, 10:43 AM

hey CF folks. Do we expect 50Gb + storage size for individual d1 databases in the future? I am not asking for any timelines, as I don't have any need right now, but I want to understand if I am locking myself into a tech choice that I may not be able to work with later. (I know that I can create multiple databases, but I fear the spreading my data over multiple databses will make things way too complicated for me)

RRick hey CF folks. Do we expect 50Gb + storage size for individual d1 databases in th...

user•11/20/23, 2:10 PM

^^ this

Uuser https://user.you-sk.id/plCW7Cg4qgki Is there any way to get more than 2gb?

Max (@rozenmd)•11/20/23, 2:25 PM

today? no, it's a hard limit we're working on increasing

MMax (@rozenmd)today? no, it's a hard limit we're working on increasing

user•11/20/23, 3:04 PM

Is there any sort of guess on when the hard limit might be increased?

user•11/20/23, 3:04 PM

I read into it and understand that it’s a difficult task to maintain for CF

user•11/20/23, 3:04 PM

Just curious if there’s any guesses about progress

Uuser Is there any sort of guess on when the hard limit might be increased?

Max (@rozenmd)•11/20/23, 3:15 PM

we don't guess/provide estimates - once we increase the limit there will likely be a large announcement about it

RRick hey CF folks. Do we expect 50Gb + storage size for individual d1 databases in th...

Matt Silverlock•11/20/23, 3:17 PM

Not in the next 6+ months (at least). Hyperdrive + a traditional RDBMS is likely going to be a better fit if you truly need 50GB in a single database (or have an existing DB you want to migrate) for the time being. Different use-cases.

MMax (@rozenmd)we don't guess/provide estimates - once we increase the limit there will likely ...

user•11/20/23, 3:21 PM

Alright, I assume the D1 role will be pongtsted?

clearwater4227•11/21/23, 2:07 AM

Hello all, I was looking to build with D1 but dont see a way to provision tables/schema through terraform. I just see creating databases... Am I missing something?

Cclearwater4227 Hello all, I was looking to build with D1 but dont see a way to provision table...

Cyb3r-Jak3•11/21/23, 2:33 AM

You typically wouldn't create the schema through terraform and better to use something like Drizzle (https://orm.drizzle.team/docs/quick-sqlite/d1) to manage it

Cloudflare D1 - Drizzle ORM

Drizzle ORM is a lightweight and performant TypeScript ORM with developer experience in mind.

clearwater4227•11/21/23, 12:04 PM

ok, interesting....so this looks like create/define schema in js and it will create schema/tables if they dont exist?

Cclearwater4227 Hello all, I was looking to build with D1 but dont see a way to provision table...

Max (@rozenmd)•11/21/23, 12:42 PM

Generally terraform is used to spin up infrastructure, I haven't heard of folks using it for SQL commands

clearwater4227•11/21/23, 1:03 PM

I use terraform for building out my bigquery database tables. I was going to move some of my tables to D1 for performance and cost reasons so I was looking at integrating that into my terraform setup.

clearwater4227•11/21/23, 1:05 PM

kind of build tables and workers for a few tables, setup terraform setup/deploy, migrate things over, and press go. Terraform is nice for schema management as well, at least for bigquery, as I can add and change columns and the bigquery terraform integration handles the changes and migrations.

Alexey•11/21/23, 4:16 PM

hey folks. I'm new to D1 (and databases in General).
My current problem is data conversion. It seems to me I need to manually convert JS data types to SQL data types and vice versa.
For example, for D1 BOOLEAN values I get

in my function code. I expect to have

true

true

false

false

in my function code instead.
I looked into some ORMs but didn't find that functionality for automatic types conversion.

What are the ways to handle this problem? How do you approach it?

AAlexey hey folks. I'm new to D1 (and databases in General). My current problem is data ...

Bram•11/21/23, 4:30 PM

SQLite does not have a boolean storage class, but is instead stored as 0 or 1. You can use an ORM for this, I recommend

drizzle

drizzle

, a relatively new ORM that has a lot of momentum and is build with edge in mind.

Bram•11/21/23, 4:34 PM

it let's you declare sql schema's directly in typescript, where you can hint the ORM that that the integer should be interpreted as a boolean on application level: https://orm.drizzle.team/docs/column-types/sqlite integer('id', { mode: 'boolean' })integer('id', { mode: 'boolean' }), same for timestamps, json etc

SQLite column types - Drizzle ORM

Drizzle ORM is a lightweight and performant TypeScript ORM with developer experience in mind.

Alexey•11/21/23, 4:35 PM

ok, interesting, will give drizzle a try. Thank for the suggestion @Bram

Unsmart•11/21/23, 5:32 PM

The main way would be to create a worker as a proxy

James•11/21/23, 5:53 PM

I think this is that same obscure issue we've seen a few times, cc @Matt: https://community.cloudflare.com/t/d1-database-is-crashed/583564

Cloudflare Community

D1 database is crashed

My D1 database is suddenly crashed. And it is not recovered. In D1 tab, there is a message “Failed to reach database. Please try again later”. What’s going on?

Cyb3r-Jak3•11/21/23, 6:49 PM

Old vs new backend. My guess is the first one hasn’t had the backend migrated.

RRed Spider Is it possible to bulk insert via the API? I have an array of items i want to in...

Hello, I’m Allie!•11/21/23, 7:53 PM

Depending on the size of the items, and whether they are rows in a single table, or multiple tables, you might be able to do a multi-insert

Hello, I’m Allie!•11/21/23, 7:54 PM

INSERT INTO table1 (column1,column2 ,..)
VALUES 
   (value1,value2 ,...),
   (value1,value2 ,...),
    ...
   (value1,value2 ,...);

INSERT INTO table1 (column1,column2 ,..)
VALUES 
   (value1,value2 ,...),
   (value1,value2 ,...),
    ...
   (value1,value2 ,...);

Hello, I’m Allie!•11/21/23, 7:54 PM

Something like this

EXtremeExploit•11/21/23, 8:17 PM

is D1 down?

EXtremeExploit•11/21/23, 8:18 PM

alrighty just wanted to make sure it wasnt only me, thankie

Unsmart•11/21/23, 8:18 PM

Seems fine for me... is this every query or just some?

EXtremeExploit•11/21/23, 8:18 PM

all of them

EXtremeExploit•11/21/23, 8:18 PM

even the site fails to fetch the data

Is D1 sized fixed to maximum 2gb, or it can be extended on request? If not, we are going to design

Similar Threads

Similar Threads

Similar Threads