Cloudflare Developers•2y ago

Is there any mention anywhere about the SQLite version of D1?

Ttt Is there any mention anywhere about the SQLite version of D1?

Max (@rozenmd)•2/2/24, 2:08 PM

https://github.com/cloudflare/workerd/blob/main/WORKSPACE#L67

MMax (@rozenmd)https://github.com/cloudflare/workerd/blob/main/WORKSPACE#L67

ttOP•2/2/24, 2:09 PM

Thanks!!

kgni•2/2/24, 2:28 PM

Hi there. Sorry in advance for the noob question.

Does binding values have to be unique when using the Wrangler CLI /

wrangler.toml

wrangler.toml

config?
E.G: if I have multiple environments, that I want to connect to different D1 databases.

I'm a bit confused since in the dashboard it seems like you create 1 binding per 1 database, but when using the

wrangler.toml

wrangler.toml

config file and the CLI, the binding name seems to be used more like an environment variable that you can then use in your worker.

Would the following 2 environments point to 2 different databases even though the binding names are the same?

[env.staging]
d1_databases = [
    { binding = "DB", database_name = "staging", database_id = "unique_db_id_1", migrations_dir = "migrations", preview_database_id = "DB_STAGING" },
]
workers_dev = true

[env.production]
d1_databases = [
    { binding = "DB", database_name = "production", database_id = "unique_db_id_2", migrations_dir = "migrations"},
]

[env.staging]
d1_databases = [
    { binding = "DB", database_name = "staging", database_id = "unique_db_id_1", migrations_dir = "migrations", preview_database_id = "DB_STAGING" },
]
workers_dev = true

[env.production]
d1_databases = [
    { binding = "DB", database_name = "production", database_id = "unique_db_id_2", migrations_dir = "migrations"},
]

neal•2/2/24, 3:56 PM

Hi! I'm using the D1 database for a game I'm working on https://twitter.com/nealagarwal/status/1752716375255101440

The database appears to be down after a large traffic spike. Is there anyone that could help debug?

Neal Agarwal (@nealagarwal) on X

The first version of Infinite Craft, an endless crafting game, is out now!

https://t.co/tb51ejeqsK

Twitter

•

1/31/24, 3:32 PM

Nneal Hi! I'm using the D1 database for a game I'm working on https://twitter.com/neal...

Unsmart•2/2/24, 3:59 PM

How many requests are you doing? It could simply be you are doing too many queries for the database to handle.

UUnsmart How many requests are you doing? It could simply be you are doing too many queri...

neal•2/2/24, 4:05 PM

It was about 500 requests per second during the traffic spike. But now the database appears to be down and all requests are hanging. I may need to migrate to a different database but I can't currently access the data

Unsmart•2/2/24, 4:06 PM

Ah yeah 500rps is very upper limits from what I have heard. Fwiw the app works fine for me rn most of the time only some queries fail

neal•2/2/24, 4:07 PM

The cached queries are still working, but anything that's not cached is failing because it needs to access the database unfortunately

Chaika•2/2/24, 4:07 PM

Using beta, not alpha DB right?

Chaika•2/2/24, 4:08 PM

can check via npx wrangler d1 info <db-name>npx wrangler d1 info <db-name>

neal•2/2/24, 4:08 PM

Yes using D1 that says it's in open beta

Chaika•2/2/24, 4:08 PM

there's still two different types of databases. If you made it a while ago, it may be alpha. Alpha had a lot worse performance

Chaika•2/2/24, 4:08 PM

500 rps is still pretty high though

CChaika can check via `npx wrangler d1 info <db-name>`

neal•2/2/24, 4:09 PM

It looks like this also doesn't respond

neal•2/2/24, 4:10 PM

Screen_Shot_2024-02-02_at_11.10.01_AM.png

neal•2/2/24, 4:11 PM

I created the database last month, so I believe it should be the beta version

Nneal I created the database last month, so I believe it should be the beta version

Chaika•2/2/24, 4:27 PM

Yea iirc they disabled new alpha dbs a few months ago
Under Workers & Pages -> D1, the database ID would be helpful (or you could just copy it from that error, it's the one at the end there hyphen seperated. Your Account ID would also be useful. It's under Workers & Pges overview or the /accounts/<accountid> in that error. I would OCR them but it's going to mess them up somewhere. They aren't private or anything but would let the D1 Team look into the DB

neal•2/2/24, 4:39 PM

My account id is "62cafb6b9c2ebd2e6970091b9ef0b6a8"

neal•2/2/24, 4:39 PM

Thanks for your help!

neal•2/2/24, 4:42 PM

Also database id is "eb589e60-ed90-4777-9685-f73fad7daaa3"

MMax (@rozenmd)make it a const in your worker if it never changes

bun•2/2/24, 4:47 PM

const you mean just create it as a variable?

Bbun const you mean just create it as a variable?

Max (@rozenmd)•2/2/24, 4:57 PM

yeah

bun•2/2/24, 4:58 PM

i already have a r2 + cache setup it works amazingly well. i have a 10ms latency on retrieving the cache of my 1 mb JSON file

Nneal Click to see attachment

Max (@rozenmd)•2/2/24, 4:58 PM

d1 info needs to hit your actual db to get data, looks like it's overloaded

Bbun i already have a r2 + cache setup it works amazingly well. i have a 10ms latency...

Max (@rozenmd)•2/2/24, 4:59 PM

as a const it'd be a 0ms latency

bun•2/2/24, 4:59 PM

yeah but that way i dont increase the bundle size

MMax (@rozenmd)d1 info needs to hit your actual db to get data, looks like it's overloaded

neal•2/2/24, 4:59 PM

is there anything I can do?

neal•2/2/24, 5:00 PM

I database doesn't load in the dashboard anymore either

Nneal is there anything I can do?

Max (@rozenmd)•2/2/24, 5:07 PM

i'm looking into it, but i would look for ways to shard the db across multiple d1 dbs (while we don't automatically do that for you yet)

MMax (@rozenmd)i'm looking into it, but i would look for ways to shard the db across multiple d...

neal•2/2/24, 5:15 PM

thank you!

Nneal thank you!

Max (@rozenmd)•2/2/24, 5:21 PM

looks like it recovered a bit - but yeah its reaching the limits of what a single d1 db can do in terms of read/writes before we add read replication and autoscaling

Max (@rozenmd)•2/2/24, 5:21 PM

would recommend sharding manually for now and caching anything and everything you can

neal•2/2/24, 5:21 PM

Ok thanks good to know, is there an easy way to duplicate the database?

Nneal Ok thanks good to know, is there an easy way to duplicate the database?

Max (@rozenmd)•2/2/24, 5:26 PM

not yet, we're working on adding db cloning/forking/dumping, but until then you can iterate through SELECT * FROM sqlite_master;SELECT * FROM sqlite_master; or use https://github.com/nora-soderlund/cloudflare-d1-backups

neal•2/2/24, 5:27 PM

Thanks, also do you think workers key-value would be able to handle the traffic better?

Nneal Thanks, also do you think workers key-value would be able to handle the traffic ...

Max (@rozenmd)•2/2/24, 5:30 PM

if you're just writing and then reading lots, KV would stand significantly more load, trouble is replication lag on updates

Max (@rozenmd)•2/2/24, 5:32 PM

https://developers.cloudflare.com/kv/reference/how-kv-works/

DDavid Renshaw ~~It seems that every time I do `wrangler dev` or `wrangler d1 execute` (regardl...

Max (@rozenmd)•2/2/24, 5:34 PM

max(indexed_column)

max(indexed_column)

should save some reads (i think)

DDavid Renshaw my database has ~280k rows, and I'm seeing that many reads show up in my "row me...

Max (@rozenmd)•2/2/24, 5:36 PM

the console doesn't run any queries unless you enter them - check out your network tab as you load the page

Paul•2/2/24, 5:50 PM

Once it’s GA, is there a limit to the size the database can be?

Chaika•2/2/24, 5:51 PM

will always be a limit, they'd rather you shard into mutiple small dbs then one big one -> d1-database

DDavid Renshaw It would be nice if I had a way to navigate to the "metrics" page without hittin...

Chaika•2/2/24, 5:52 PM

do you not have primary keys or any indexes on your tables? Shouldn't be a full scan just to count all the rows

[env.staging] d1_databases = [ { binding = "DB", database_name = "staging", database_id = "unique_db_id_1", migrations_dir = "migrations", preview_database_id = "DB_STAGING" }, ] workers_dev = true [env.production] d1_databases = [ { binding = "DB", database_name = "production", database_id = "unique_db_id_2", migrations_dir = "migrations"}, ]

Is there any mention anywhere about the SQLite version of D1?

Similar Threads

Similar Threads

Similar Threads