Is it just me or are these errors bound to happen as long as I'm using D1? I've been getting a lot o

JJIYU Is it just me or are these errors bound to happen as long as I'm using D1? I've ...

T

TomK•10/30/24, 7:13 AM

You are not alone.

https://www.cloudflarestatus.com/incidents/h5n23gyb4my2

Increase in D1 Errors

N

naourass•10/30/24, 2:31 PM

I'm trying to apply a migration on a

preview

preview

preview

preview db, but the command is failing with

Exit Code 1

Exit Code 1

Exit Code 1

Exit Code 1 without any error message. Any idea what am I doing wrong or what could be the error cause here?

Update: Issue resolved, I had to create an independent preview database and pass its id to

preview_database_id

preview_database_id

preview_database_id

preview_database_id. Note that doing this will change the id of your local database too if any.

Here's my

wrangler.toml

wrangler.toml

wrangler.toml

wrangler.toml db config:

[[d1_databases]]
binding = "CORE_DB"
database_name = "phoenix-core-prod"
database_id = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
migrations_dir="database/migrations/core"
preview_database_id = "CORE_DB"

[[d1_databases]]
binding = "CORE_DB"
database_name = "phoenix-core-prod"
database_id = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
migrations_dir="database/migrations/core"
preview_database_id = "CORE_DB"

[[d1_databases]]
binding = "CORE_DB"
database_name = "phoenix-core-prod"
database_id = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
migrations_dir="database/migrations/core"
preview_database_id = "CORE_DB"

[[d1_databases]]
binding = "CORE_DB"
database_name = "phoenix-core-prod"
database_id = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
migrations_dir="database/migrations/core"
preview_database_id = "CORE_DB"

Here's the

npm

npm

npm

npm script:

"db:migrate:core:preview": "wrangler d1 migrations apply phoenix-core-prod --remote --preview"

"db:migrate:core:preview": "wrangler d1 migrations apply phoenix-core-prod --remote --preview"

"db:migrate:core:preview": "wrangler d1 migrations apply phoenix-core-prod --remote --preview"

"db:migrate:core:preview": "wrangler d1 migrations apply phoenix-core-prod --remote --preview"

And here's the command result:

N

naourass•10/30/24, 2:37 PM

PS. The migration was applied successfully on the local db (

--local

--local

--local

--local), and I've already created the phoenix-core-prod remote db:

N

naourass•10/30/24, 2:41 PM

And ofc I'm putting the right db id it's just redacted in the wrangler config above.

Nnaourass And ofc I'm putting the right db id it's just redacted in the wrangler config ab...

H

Hello, I’m Allie!•10/30/24, 2:50 PM

Note DB IDs(and the rest of your wrangler config) are not secret, as anyone would need access to your account to make use of it

N

naourass•10/30/24, 3:01 PM

Thanks for the note! Back to the issue, I'm note sure if I should create an independent preview db and pass its id to

preview_database_id

preview_database_id

preview_database_id

preview_database_id. Can anyone confirm?

N

naourass•10/30/24, 3:51 PM

Just tried that

and the migration command was applied successfully!

UUnsmart <https://developers.cloudflare.com/workers/wrangler/configuration/#d1-databases>...

J

JustinNoel•10/30/24, 4:27 PM

So how do you access these different bindings in your code? In the past, all environments could have the same DB binding name for all environments.

Now, do I have to do something like this?

let db = c.env.DB_DEV;

if (c.env.??? === "staging") {
  db = c.env.DB_STAGING;
}

if (c.env.??? === "production") {
  db = c.env.DB_PRODUCTION;
}

let db = c.env.DB_DEV;

if (c.env.??? === "staging") {
  db = c.env.DB_STAGING;
}

if (c.env.??? === "production") {
  db = c.env.DB_PRODUCTION;
}

let db = c.env.DB_DEV;

if (c.env.??? === "staging") {
  db = c.env.DB_STAGING;
}

if (c.env.??? === "production") {
  db = c.env.DB_PRODUCTION;
}

let db = c.env.DB_DEV;

if (c.env.??? === "staging") {
  db = c.env.DB_STAGING;
}

if (c.env.??? === "production") {
  db = c.env.DB_PRODUCTION;
}

JJustinNoel So how do you access these different bindings in your code? In the past, all en...

U

Unsmart•10/30/24, 4:29 PM

All environments can have the same binding name

UUnsmart All environments can have the same binding name

J

JustinNoel•10/30/24, 4:29 PM

✘ [ERROR] Processing wrangler.toml configuration:

    - DB assigned to multiple D1 Databases bindings.
    - Bindings must have unique names, so that they can all be referenced in the worker.
      Please change your bindings to have unique names.

✘ [ERROR] Processing wrangler.toml configuration:

    - DB assigned to multiple D1 Databases bindings.
    - Bindings must have unique names, so that they can all be referenced in the worker.
      Please change your bindings to have unique names.

✘ [ERROR] Processing wrangler.toml configuration:

    - DB assigned to multiple D1 Databases bindings.
    - Bindings must have unique names, so that they can all be referenced in the worker.
      Please change your bindings to have unique names.

✘ [ERROR] Processing wrangler.toml configuration:

    - DB assigned to multiple D1 Databases bindings.
    - Bindings must have unique names, so that they can all be referenced in the worker.
      Please change your bindings to have unique names.

U

Unsmart•10/30/24, 4:30 PM

That sounds like you’re reusing the same binding in the same environment

J

JustinNoel•10/30/24, 4:30 PM

This is how I used it in an earlier version of wrangler. Now, I can't.

[[d1_databases]]
    binding = "DB"
    database_name = "say-my-name"
    database_id = "EEB7B47A-96CF-11EF-9123-BE9F9F8BCB91"

[env.staging]
[[d1_databases]]
    binding = "DB"
    database_name = "preview-say-my-name"
    database_id = "a5e66eaa-c04a-4596-bc36-d2f4e33b7676"

[env.production]
[[d1_databases]]
    binding = "DB"
    database_name = "production-say-my-name"
    database_id = "7b52c889-26d0-4c6f-b4fc-48f2523db3f2"

[[d1_databases]]
    binding = "DB"
    database_name = "say-my-name"
    database_id = "EEB7B47A-96CF-11EF-9123-BE9F9F8BCB91"

[env.staging]
[[d1_databases]]
    binding = "DB"
    database_name = "preview-say-my-name"
    database_id = "a5e66eaa-c04a-4596-bc36-d2f4e33b7676"

[env.production]
[[d1_databases]]
    binding = "DB"
    database_name = "production-say-my-name"
    database_id = "7b52c889-26d0-4c6f-b4fc-48f2523db3f2"

[[d1_databases]]
    binding = "DB"
    database_name = "say-my-name"
    database_id = "EEB7B47A-96CF-11EF-9123-BE9F9F8BCB91"

[env.staging]
[[d1_databases]]
    binding = "DB"
    database_name = "preview-say-my-name"
    database_id = "a5e66eaa-c04a-4596-bc36-d2f4e33b7676"

[env.production]
[[d1_databases]]
    binding = "DB"
    database_name = "production-say-my-name"
    database_id = "7b52c889-26d0-4c6f-b4fc-48f2523db3f2"

[[d1_databases]]
    binding = "DB"
    database_name = "say-my-name"
    database_id = "EEB7B47A-96CF-11EF-9123-BE9F9F8BCB91"

[env.staging]
[[d1_databases]]
    binding = "DB"
    database_name = "preview-say-my-name"
    database_id = "a5e66eaa-c04a-4596-bc36-d2f4e33b7676"

[env.production]
[[d1_databases]]
    binding = "DB"
    database_name = "production-say-my-name"
    database_id = "7b52c889-26d0-4c6f-b4fc-48f2523db3f2"

U

Unsmart•10/30/24, 4:31 PM

Yeah right now everything is the same environment. You have to do:

[[env.<name>.d1_databases]]
…

J

JustinNoel•10/30/24, 4:32 PM

I don't understand that. Here's the docs: https://developers.cloudflare.com/d1/configuration/environments/

Cloudflare Docs

Environments | Cloudflare D1 docs

Environments are different contexts that your code runs in. Cloudflare Developer Platform allows you to create and manage different environments. Through environments, you can deploy the same project to multiple places under multiple names.

U

Unsmart•10/30/24, 4:33 PM

If you defined it exactly how they did there it would work also. Variable vs tables in toml

J

JustinNoel•10/30/24, 4:34 PM

I did. It didn't work. Again, those docs change the db binding name for each env.

I must frustrated about, is this changed. I don't see why it needed to.

U

Unsmart•10/30/24, 4:36 PM

It hasn’t changed. In toml a table has to be defined differently from the docs you linked that uses a variable to define it.

Table must be:
[[env.<name>.d1_databases]]
…

Variable can be:
[env.<name>]
d1_databases = [
…
]

J

JustinNoel•10/30/24, 4:38 PM

OK, I've tried again. I tried so many weird configurations that I must've made a mistake. But it is working now.

d1_databases = [
    { binding = "DB", database_name = "say-my-name", database_id = "EEB7B47A-96CF-11EF-9123-BE9F9F8BCB91" },
]

[env.staging]
d1_databases = [
    { binding = "DB", database_name = "preview-say-my-name", database_id = "a5e66eaa-c04a-4596-bc36-d2f4e33b7676" },
]

# This is a production environment
[env.production]
d1_databases = [
    { binding = "DB", database_name = "production-say-my-name", database_id = "7b52c889-26d0-4c6f-b4fc-48f2523db3f2" },
]

d1_databases = [
    { binding = "DB", database_name = "say-my-name", database_id = "EEB7B47A-96CF-11EF-9123-BE9F9F8BCB91" },
]

[env.staging]
d1_databases = [
    { binding = "DB", database_name = "preview-say-my-name", database_id = "a5e66eaa-c04a-4596-bc36-d2f4e33b7676" },
]

# This is a production environment
[env.production]
d1_databases = [
    { binding = "DB", database_name = "production-say-my-name", database_id = "7b52c889-26d0-4c6f-b4fc-48f2523db3f2" },
]

d1_databases = [
    { binding = "DB", database_name = "say-my-name", database_id = "EEB7B47A-96CF-11EF-9123-BE9F9F8BCB91" },
]

[env.staging]
d1_databases = [
    { binding = "DB", database_name = "preview-say-my-name", database_id = "a5e66eaa-c04a-4596-bc36-d2f4e33b7676" },
]

# This is a production environment
[env.production]
d1_databases = [
    { binding = "DB", database_name = "production-say-my-name", database_id = "7b52c889-26d0-4c6f-b4fc-48f2523db3f2" },
]

d1_databases = [
    { binding = "DB", database_name = "say-my-name", database_id = "EEB7B47A-96CF-11EF-9123-BE9F9F8BCB91" },
]

[env.staging]
d1_databases = [
    { binding = "DB", database_name = "preview-say-my-name", database_id = "a5e66eaa-c04a-4596-bc36-d2f4e33b7676" },
]

# This is a production environment
[env.production]
d1_databases = [
    { binding = "DB", database_name = "production-say-my-name", database_id = "7b52c889-26d0-4c6f-b4fc-48f2523db3f2" },
]

I think the documentation is very confusing. It really ought to show that the binding name is the same for all potential environments.

Thanks for your help.

K

kunal•10/30/24, 7:46 PM

Any idea on when D1 global read replication will be available?

We are testing a new integration, and our query read latency is ~500ms on average for primary key lookups, which is too slow for our use case. Considering setting up a bunch of globally replicated Postgres instances with Hyperdrive on top, but was hoping D1 might be a much easier way to get the same result

K

kunal•10/30/24, 7:47 PM

Also, is it possible to change the location of an existing D1 instance? The docs say it will move around on it's own, but mine seems stuck in a low traffic region (close to where I set it up from, rather than where the production traffic is coming from)

Kkunal Also, is it possible to change the location of an existing D1 instance? The docs...

C

Chaika•10/30/24, 7:52 PM

Where do the docs say that? The D1 instance stays within the region it is created in/starts in

Kkunal Also, is it possible to change the location of an existing D1 instance? The docs...

F

fry69•10/30/24, 7:54 PM

It says "smart placement" not "smart movement"

CChaika Where do the docs say that? The D1 instance stays within the region it is create...

K

kunal•10/30/24, 8:09 PM

I see. Maybe I confused the read replicas being dynamically provisioned with the leader getting dynamically moved.
So right now, D1 is purely in the single region chosen at creation (reads and writes), and the read replicas will allow global reads soon?

Kkunal I see. Maybe I confused the read replicas being dynamically provisioned with the...

I

Isaac McFadyen•10/30/24, 8:10 PM

That's correct, yup.

C

Chaika•10/30/24, 8:10 PM

Well, idk about "soon" for read replicas

H

Hello, I’m Allie!•10/30/24, 8:11 PM

:soontm:

C

Chaika•10/30/24, 8:11 PM

Depends on your definition. They talked about having it in Q3 and at the start of September having a private beta "soon", but haven't heard anymore and we're past q3

I

Isaac McFadyen•10/30/24, 8:12 PM

^ I wouldn't plan your application around read replicas

K

kunal•10/30/24, 8:23 PM

Got it, thanks for the information!

S

Sam•10/30/24, 10:26 PM

Hey all - was there a resurgence of https://www.cloudflarestatus.com/incidents/h5n23gyb4my2 earlier today (~16:30 UTC?)

Seeing quite a few D1 errors in my logs around that time & a few more recently

Increase in D1 Errors

T

TomK•10/31/24, 1:10 PM

D1 Now there are frequent errors, same scenario as yesterday.

J

Jeff•10/31/24, 2:21 PM

Hey guys, has D1 always returned bool values as strings?

JJeff Hey guys, has D1 always returned bool values as strings?

F

fry69•10/31/24, 2:22 PM

D1 is based on SQLite, everything is a string inside SQLite (except INTEGER)

J

Jeff•10/31/24, 2:23 PM

I am using the prisma adapter for it, maybe i never noticed this error but it crashes trying to parse the sqlite 'true' as bool

J

Jeff•10/31/24, 2:23 PM

https://www.sqlite.org/datatype3.html#boolean_datatype

JJeff https://www.sqlite.org/datatype3.html#boolean_datatype

F

fry69•10/31/24, 2:28 PM

Yeah, but SQLite and probably D1 is very lax about all that ->

Any column in an SQLite version 3 database, except an INTEGER PRIMARY KEY column, may be used to store a value of any storage class.

TTomK D1 Now there are frequent errors, same scenario as yesterday.

J

JIYUOP•10/31/24, 2:28 PM

I'm seeing less of it than yesterday, but it's still intermittent.

J

Jeff•10/31/24, 2:29 PM

I'll open a bug on prisma side then

J

JIYUOP•10/31/24, 2:40 PM

JST 23:29 Start Error: D1_ERROR: Network connection lost. - -

J

JIYUOP•10/31/24, 2:42 PM

I'm currently getting a lot of errors too.

J

JIYUOP•10/31/24, 2:46 PM

If this continues to happen, I'll have to seriously consider moving to another service. I'm currently running a storefront and it's seriously impacting sales and operations.

I

Isaac McFadyen•10/31/24, 2:52 PM

We escalated this again, thanks for the reports.

JJIYU I'm currently getting a lot of errors too.

I

Isaac McFadyen•10/31/24, 2:53 PM

Can you send your DB ID please? It's not sensitive info (no one can access your DB without account access) so you can send it here.

TTomK D1 Now there are frequent errors, same scenario as yesterday.

I

Isaac McFadyen•10/31/24, 2:53 PM

Mind sending your DB ID please?

JJIYU If this continues to happen, I'll have to seriously consider moving to another s...

J

James•10/31/24, 3:03 PM

With the sheer number of errors that occur with D1, I'd honestly recommend avoiding it for production use today. Some stats: https://canary.discord.com/channels/595317990191398933/992060581832032316/1300948096964100097

K

Kiki Y•10/31/24, 3:13 PM

Hi team, thank you for reporting the issue. I am Kiki from Cloudflare, we are checking this issue with internal team. Just to confirm, if anyone is still seeing the same error now?

J

JIYUOP•10/31/24, 3:15 PM

I've been using D1 since 2 years ago and it's a shame.
I've been really happy with the integration with Workers, but I think I'm going to have to move to supabase or something. - -
I'm afraid the move will be a pain. - -

J

JIYUOP•10/31/24, 3:16 PM

We are not experiencing any issues at this time.

J

JIYUOP•10/31/24, 3:18 PM

But what if this happens again someday? I'm sad.... - -

J

JIYUOP•10/31/24, 3:26 PM

JST: 00:18:45 ~ 00:20:00 again occured
Error: D1_ERROR: Network connection lost.
and
Error: D1_ERROR: Internal error while starting up D1 storage caused object to be reset.

Is it just me or are these errors bound to happen as long as I'm using D1? I've been getting a lot o

Similar Threads

Similar Threads

Similar Threads