Started getting a bunch of these again at 3AM CEST ```json { "message": "D1_ERROR", "cause": "Er

Started getting a bunch of these again at 3AM CEST

{
  "message": "D1_ERROR",
  "cause": "Error: D1 storage operation exceeded timeout which caused object to be reset."
}

{
  "message": "D1_ERROR",
  "cause": "Error: D1 storage operation exceeded timeout which caused object to be reset."
}

{
  "message": "D1_ERROR",
  "cause": "Error: D1 storage operation exceeded timeout which caused object to be reset."
}

{
  "message": "D1_ERROR",
  "cause": "Error: D1 storage operation exceeded timeout which caused object to be reset."
}

BBhadoo I really need more storage. Why Cloudflare isn't responding to requests. I've be...

Matt Silverlock•6/6/23, 10:34 AM

We can increase some limits during the alpha — storage isn’t one of them. From the Limits page:

Note that not all limits can be increased.

The original prototype backend just doesn’t perform well above 100MB. It’s pointless to increase it.

MMozzy Started getting a bunch of these again at 3AM CEST ```json { "message": "D1_ER...

Matt Silverlock•6/6/23, 10:34 AM

Which backend?

MMatt Silverlock Which backend?

MozzyOP•6/6/23, 10:35 AM

Experimental

MMozzy Experimental

Matt Silverlock•6/6/23, 10:35 AM

Gonna need more details than that. DB ID? What were you trying to do?

MMatt Silverlock Gonna need more details than that. DB ID? What were you trying to do?

MozzyOP•6/6/23, 10:37 AM

ID:

e68dccd4-7bf7-4860-ba18-17978966ec10

e68dccd4-7bf7-4860-ba18-17978966ec10

Error comes from first two POSTs here: https://github.com/leonlarsson/bfstats-api/blob/main/handlers/D1Handler.ts#L105
I'll try try dig up more info

MMozzy ID: `e68dccd4-7bf7-4860-ba18-17978966ec10` Error comes from first two POSTs here...

Matt Silverlock•6/6/23, 10:40 AM

Query looks pretty basic. Is this 100% of the time? 50%?

MozzyOP•6/6/23, 10:40 AM

Much much less

MozzyOP•6/6/23, 10:40 AM

Maybe 2%

MozzyOP•6/6/23, 10:40 AM

Looking into if it's happening due to the username

MozzyOP•6/6/23, 10:43 AM

Nope, nothing weird with the data. Just regular latin characters and short strings

MozzyOP•6/6/23, 10:45 AM

Actually seeing errors from https://github.com/leonlarsson/bfstats-api/blob/main/handlers/D1Handler.ts#L152 as well, which the simplest query imaginable

MozzyOP•6/6/23, 11:00 AM

Just got the errors again on POST /d1/users and /d1/outputs

Bbennycondon I'm running the command: ``` wrangler d1 execute DB --file=./migrations/16739100...

mukhtharcm•6/6/23, 12:14 PM

Did you got any solution?

Mmukhtharcm Did you got any solution?

Matt Silverlock•6/6/23, 12:20 PM

You should update to the latest wrangler first, especially if you’re not on v3+

MMozzy Nope, nothing weird with the data. Just regular latin characters and short strin...

Matt Silverlock•6/6/23, 12:23 PM

It won’t be the queries. Yours are trivial. Likely a bug on our side. The new storage subsystem is still, well, new

BBhadoo Also can i bind more than one database in wrangler.toml file and if yes, then ho...

Adi•6/6/23, 12:50 PM

Yes, this is how I've done it

[[ d1_databases ]]
binding = "__D1_BETA__DB_ONE"
database_name = "db-one"
database_id = "<someID>"
migrations_table = "d1_migrations"
migrations_dir = "d1_migrations/db-one"


[[ d1_databases ]]
binding = "__D1_BETA__DB_TWO"
database_name = "db-two"
database_id = "<someId>"
migrations_table = "d1_migrations"
migrations_dir = "d1_migrations/db-two"

[[ d1_databases ]]
binding = "__D1_BETA__DB_ONE"
database_name = "db-one"
database_id = "<someID>"
migrations_table = "d1_migrations"
migrations_dir = "d1_migrations/db-one"


[[ d1_databases ]]
binding = "__D1_BETA__DB_TWO"
database_name = "db-two"
database_id = "<someId>"
migrations_table = "d1_migrations"
migrations_dir = "d1_migrations/db-two"

Original message was deleted

Adi•6/6/23, 12:51 PM

your way is definitely cleaner than mine XD

Original message was deleted

Hello, I’m Allie!•6/6/23, 12:59 PM

I think it is because

wrangler

wrangler

has been, basically from the beginning, a tool that uses

toml

toml

. They are adding a system to parse JSON into an object that

wrangler

wrangler

can use that matches the

toml

toml

spec, but there may be some points that don't just work

Hello, I’m Allie!•6/6/23, 12:59 PM

And which config format is the best is an extremely hot topic.

Original message was deleted

Hello, I’m Allie!•6/6/23, 1:03 PM

Well,

wrangler

wrangler

used to be written in Rust, which is why it started out with

toml

toml

. And you can still run Workers entirely without writing JS(see #

workers-rs), plus

workerd

workerd

, which isn't JS at all. And they are moving toward JSON configs now

MMatt Silverlock You should update to the latest wrangler first, especially if you’re not on v3+

mukhtharcm•6/6/23, 1:05 PM

I'm already on 3.0.1

BBhadoo I'm using New Experimental. After 104.24 MB its not allowing more upload.

Matt Silverlock•6/6/23, 1:24 PM

Yes - 100MB is the limit for now. We aren’t increasing database limits yet. Hold tight.

MMozzy Experimental

Matt Silverlock•6/6/23, 2:42 PM

It's definitely not an experimental database (I just got online and checked:

e68dccd4-7bf7-4860-ba18-17978966ec10

e68dccd4-7bf7-4860-ba18-17978966ec10

is on the older alpha backend)

MMatt Silverlock It's definitely not an experimental database (I just got online and checked: `e6...

MozzyOP•6/6/23, 2:44 PM

:Hmm:

MMozzy :Hmm:

Matt Silverlock•6/6/23, 2:48 PM

Looked at the wrong ID, ha. I see ~11 storage timeouts in the last 24 hours on that DB. This is related to a larger issue we're investigating.

MozzyOP•6/6/23, 2:49 PM

Ahh okay

Vero•6/6/23, 6:40 PM

Hey there! It's feedback time!

We would love to hear from you. What features do you wish D1 had?

And what's your biggest challenge or frustration when using D1?

《

《 Sarge 》•6/6/23, 7:31 PM

My wish would be for more defined error messages. When executing a sql script. Often the error message is 'D1 Error' with no supporting information.

《

《 Sarge 》•6/6/23, 7:32 PM

Yeah, this is for wrangler sorry. Maybe wrong thread

Unsmart•6/6/23, 7:33 PM

That is valid feedback here also as its related to D1 still.

Original message was deleted

Matt Silverlock•6/6/23, 8:52 PM

We're going to lift the errors out of

error.cause

error.cause

into the top-level

error

error

soon. It might break a few folks who are exact string matching: no real way around this. Luck of an alpha

Original message was deleted

Matt Silverlock•6/6/23, 8:54 PM

Hard to really schedule. Outside of an alpha it'd be a compatibility date, but we have to then carry that forever, and I'm going to lean on the "alpha" part of "alpha" here. Likely within the week. We're going to keep

error.cause

error.cause

around until open beta, then trim that up.

mukhtharcm•6/7/23, 2:57 AM

Hi, is there any d1 orm except drizzle which support joins?

Mmukhtharcm Hi, is there any d1 orm except drizzle which support joins?

mukhtharcm•6/7/23, 3:31 AM

I asked this because of this problem https://github.com/cloudflare/workers-sdk/issues/3160

Tried kysely. It works

GitHub

🐛 BUG: when trying to left join 2 tables that have same name column...

Which Cloudflare product(s) does this pertain to? D1 What version of Wrangler are you using? 2.19.0 What operating system are you using? macOS Describe the Bug As per issue #555 we can see, that d1...

Hello, I’m Allie!•6/7/23, 7:48 AM

Anyone remember whether D1 Auto-Vacuums, or whether we should run them ourselves periodically?

HHello, I’m Allie!Anyone remember whether D1 Auto-Vacuums, or whether we should run them ourselves...

Matt Silverlock•6/7/23, 1:06 PM

Right now: neither. We need to implement auto-vacuum. You can't run it yourself. But we don't include the free list in the reported/restricted size. More of an "us" problem right now.

MMatt Silverlock Right now: neither. We need to implement auto-vacuum. You can't run it yourself....

Hello, I’m Allie!•6/7/23, 3:23 PM

Are there any other things you recommend doing to maximize performance? I’ve read about a couple of Pragmas you can set, but I’m not sure what applies/works with D1

Hello, I’m Allie!•6/7/23, 3:23 PM

Or, more like, is there something about Best Practices?

HHello, I’m Allie!Are there any other things you recommend doing to maximize performance? I’ve rea...

Matt Silverlock•6/7/23, 3:26 PM

Indexes! One good index goes a LONG way.

Matt Silverlock•6/7/23, 3:27 PM

There will be docs about all of this — just takes time to write.

shixzie•6/7/23, 8:44 PM

is there a way to specify the

migrations_dir

migrations_dir

with the cli? I'm working on a pages project and it doesn't have a

wrangler.toml

wrangler.toml

file. So I'm trying a script like

npx wrangler d1 migrations apply <DB> --migrations-dir=<dir>

npx wrangler d1 migrations apply <DB> --migrations-dir=<dir>

but it's not working.

Sshixzie is there a way to specify the `migrations_dir` with the cli? I'm working on a pa...

Matt Silverlock•6/7/23, 9:18 PM

Not currently, no. Pages does not support a config file.

MMatt Silverlock Not currently, no. Pages does not support a config file.

shixzie•6/7/23, 9:26 PM

Is that a planned feature? Any suggestions on what can I do to manage migrations meanwhile?

Sshixzie Is that a planned feature? Any suggestions on what can I do to manage migrations...

Matt Silverlock•6/8/23, 12:31 AM

It is planned. There is not currently a workaround. cc @nevika

Murder Chicken•6/8/23, 6:39 PM

Is there an easy way to get a prepared d1 statement as text (with the bindings applied) ?

MMurder Chicken Is there an easy way to get a prepared d1 statement as text (with the bindings a...

Matt Silverlock•6/8/23, 8:14 PM

Not at the moment, no: not something we yet expose.

Started getting a bunch of these again at 3AM CEST ```json { "message": "D1_ERROR", "cause": "Er

Similar Threads