yeah with smart placement latency is low. however, it increases the request latency from 10 ms to 10

yeah with smart placement latency is low. however, it increases the request latency from 10 ms to 100-200 ms now. honestly i think there should be easier way to solve the problem im trying to deal with right now.

i have like less than 1 mb JSON data that i want to somehow have ready at all point of worker requests.

i tried getting it from r2 and caching but it didnt work for some reason.

do you think it would be better to just include the data in the bundle instead of using database for this?

Chaika•2/2/24, 12:49 AM

where are you/where are your users geographically? Are you in a region like South America/South Africa?

bunOP•2/2/24, 12:49 AM

no im in eastern europe

Chaika•2/2/24, 12:50 AM

oh so D1 should be close unless it was created with a location hint far away or for some reason created itself in a region far away

bunOP•2/2/24, 12:50 AM

i created it manually with eastern europe setting

Bbun yeah with smart placement latency is low. however, it increases the request late...

Chaika•2/2/24, 12:50 AM

The question about bundling it would come down to if it changed and if so how frequently

bunOP•2/2/24, 12:51 AM

the data wouldnt change

bunOP•2/2/24, 12:51 AM

at all

Chaika•2/2/24, 12:52 AM

then I think bundling would be a good idea. If you are on Workers Paid you get up to 10 mb, but if you go over 1 mb you get put in slower storage/slightly slower then first request, still super fast. If your worker is below 1 mb then your worker is ready to go on every machine

bunOP•2/2/24, 12:53 AM

1mb compressed ?

Chaika•2/2/24, 12:53 AM

yea gzip

bunOP•2/2/24, 12:54 AM

im on workers paid yes

bunOP•2/2/24, 12:54 AM

so everything below 1 mb does not affect performance ?

bunOP•2/2/24, 12:54 AM

then it affects only on initial request?

Chaika•2/2/24, 12:54 AM

D1 at the moment is a single location with read replicas eventually
R2 single location + cache (although you could cache any of these)
DOs are single locations
KV is made for read heavy stuff but two locations until cached
Worker Scripts less then 1 mb are everywhere
https://developers.cloudflare.com/workers/platform/storage-options/

Bbun so everything below 1 mb does not affect performance ?

Chaika•2/2/24, 12:55 AM

well if you were doing a lot in the global scope you could still have a cold starts

Chaika•2/2/24, 12:55 AM

should prob move to #workers-discussions if you wanted to ask more about that idea

bunOP•2/2/24, 12:55 AM

yeah sure

Bbun the data wouldnt change

Max (@rozenmd)•2/2/24, 7:09 AM

make it a const in your worker if it never changes

tt•2/2/24, 1:56 PM

Is there any mention anywhere about the SQLite version of D1?

Ttt Is there any mention anywhere about the SQLite version of D1?

Max (@rozenmd)•2/2/24, 2:08 PM

https://github.com/cloudflare/workerd/blob/main/WORKSPACE#L67

MMax (@rozenmd)https://github.com/cloudflare/workerd/blob/main/WORKSPACE#L67

tt•2/2/24, 2:09 PM

Thanks!!

kgni•2/2/24, 2:28 PM

Hi there. Sorry in advance for the noob question.

Does binding values have to be unique when using the Wrangler CLI /

wrangler.toml

wrangler.toml

config?
E.G: if I have multiple environments, that I want to connect to different D1 databases.

I'm a bit confused since in the dashboard it seems like you create 1 binding per 1 database, but when using the

wrangler.toml

wrangler.toml

config file and the CLI, the binding name seems to be used more like an environment variable that you can then use in your worker.

Would the following 2 environments point to 2 different databases even though the binding names are the same?

[env.staging]
d1_databases = [
    { binding = "DB", database_name = "staging", database_id = "unique_db_id_1", migrations_dir = "migrations", preview_database_id = "DB_STAGING" },
]
workers_dev = true

[env.production]
d1_databases = [
    { binding = "DB", database_name = "production", database_id = "unique_db_id_2", migrations_dir = "migrations"},
]

[env.staging]
d1_databases = [
    { binding = "DB", database_name = "staging", database_id = "unique_db_id_1", migrations_dir = "migrations", preview_database_id = "DB_STAGING" },
]
workers_dev = true

[env.production]
d1_databases = [
    { binding = "DB", database_name = "production", database_id = "unique_db_id_2", migrations_dir = "migrations"},
]

neal•2/2/24, 3:56 PM

Hi! I'm using the D1 database for a game I'm working on https://twitter.com/nealagarwal/status/1752716375255101440

The database appears to be down after a large traffic spike. Is there anyone that could help debug?

Neal Agarwal (@nealagarwal) on X

The first version of Infinite Craft, an endless crafting game, is out now!

https://t.co/tb51ejeqsK

Twitter

•

1/31/24, 3:32 PM

Nneal Hi! I'm using the D1 database for a game I'm working on https://twitter.com/neal...

Unsmart•2/2/24, 3:59 PM

How many requests are you doing? It could simply be you are doing too many queries for the database to handle.

UUnsmart How many requests are you doing? It could simply be you are doing too many queri...

neal•2/2/24, 4:05 PM

It was about 500 requests per second during the traffic spike. But now the database appears to be down and all requests are hanging. I may need to migrate to a different database but I can't currently access the data

Unsmart•2/2/24, 4:06 PM

Ah yeah 500rps is very upper limits from what I have heard. Fwiw the app works fine for me rn most of the time only some queries fail

neal•2/2/24, 4:07 PM

The cached queries are still working, but anything that's not cached is failing because it needs to access the database unfortunately

Chaika•2/2/24, 4:07 PM

Using beta, not alpha DB right?

Chaika•2/2/24, 4:08 PM

can check via

npx wrangler d1 info <db-name>

npx wrangler d1 info <db-name>

neal•2/2/24, 4:08 PM

Yes using D1 that says it's in open beta

Chaika•2/2/24, 4:08 PM

there's still two different types of databases. If you made it a while ago, it may be alpha. Alpha had a lot worse performance

Chaika•2/2/24, 4:08 PM

500 rps is still pretty high though

CChaika can check via `npx wrangler d1 info <db-name>`

neal•2/2/24, 4:09 PM

It looks like this also doesn't respond

neal•2/2/24, 4:10 PM

Screen_Shot_2024-02-02_at_11.10.01_AM.png

neal•2/2/24, 4:11 PM

I created the database last month, so I believe it should be the beta version

Nneal I created the database last month, so I believe it should be the beta version

Chaika•2/2/24, 4:27 PM

Yea iirc they disabled new alpha dbs a few months ago
Under Workers & Pages -> D1, the database ID would be helpful (or you could just copy it from that error, it's the one at the end there hyphen seperated. Your Account ID would also be useful. It's under Workers & Pges overview or the /accounts/<accountid> in that error. I would OCR them but it's going to mess them up somewhere. They aren't private or anything but would let the D1 Team look into the DB

neal•2/2/24, 4:39 PM

My account id is "62cafb6b9c2ebd2e6970091b9ef0b6a8"

neal•2/2/24, 4:39 PM

Thanks for your help!

neal•2/2/24, 4:42 PM

Also database id is "eb589e60-ed90-4777-9685-f73fad7daaa3"

MMax (@rozenmd)make it a const in your worker if it never changes

bunOP•2/2/24, 4:47 PM

const you mean just create it as a variable?

Bbun const you mean just create it as a variable?

Max (@rozenmd)•2/2/24, 4:57 PM

yeah

bunOP•2/2/24, 4:58 PM

i already have a r2 + cache setup it works amazingly well. i have a 10ms latency on retrieving the cache of my 1 mb JSON file

Nneal Click to see attachment

Max (@rozenmd)•2/2/24, 4:58 PM

d1 info needs to hit your actual db to get data, looks like it's overloaded

Bbun i already have a r2 + cache setup it works amazingly well. i have a 10ms latency...

Max (@rozenmd)•2/2/24, 4:59 PM

as a const it'd be a 0ms latency

bunOP•2/2/24, 4:59 PM

yeah but that way i dont increase the bundle size

MMax (@rozenmd)d1 info needs to hit your actual db to get data, looks like it's overloaded

neal•2/2/24, 4:59 PM

is there anything I can do?

neal•2/2/24, 5:00 PM

I database doesn't load in the dashboard anymore either

Nneal is there anything I can do?

Max (@rozenmd)•2/2/24, 5:07 PM

i'm looking into it, but i would look for ways to shard the db across multiple d1 dbs (while we don't automatically do that for you yet)

MMax (@rozenmd)i'm looking into it, but i would look for ways to shard the db across multiple d...

neal•2/2/24, 5:15 PM

thank you!

[env.staging] d1_databases = [ { binding = "DB", database_name = "staging", database_id = "unique_db_id_1", migrations_dir = "migrations", preview_database_id = "DB_STAGING" }, ] workers_dev = true [env.production] d1_databases = [ { binding = "DB", database_name = "production", database_id = "unique_db_id_2", migrations_dir = "migrations"}, ]

yeah with smart placement latency is low. however, it increases the request latency from 10 ms to 10

Similar Threads

Similar Threads

Similar Threads